Machine-learning-based tuning of encoding parameters for UGC video coding optimizations

In the era of COVID-19 pandemic, videos are very important to the billions of people staying and working at home. Two-pass video encoding allows for a refinement of parameters based on statistics obtained from the first pass. Given the variety of characteristics in user-generated content, there is opportunity to make this refinement optimal for this type of content. We show how we can replace the traditional models used for rate control in video coding with better prediction models with linear and nonlinear model functions. Moreover, we can utilize these first-pass statistics to further refine the traditional encoding recipes that are typically used for all input video sequences. Our work can provide much-needed bitrate savings for many different encoders, and we highlight it by testing on typical Facebook video content.