Hello, Habr. My name is Alexander Alpern, I am the CEO and founder of the Webinar Group. Today I would like to discuss such issues as the use of machine learning and artificial intelligence technologies in Internet broadcasting, the processing of media content and its delivery to the user.
What is it for? Increasing views, viewer loyalty, informational content of programs, as well as reducing the load on the network are just a part of the advantages provided by modern technologies.
Pandemic + traffic = problems
According to TeleGeography, which analyzed global Internet traffic exchange rates, traffic consumption skyrocketed in 2020. Thus, the global average increased from about 120 to 170 Tbps, at its peak - 300 Tbps.
In principle, the growth was already rather big, but last year all records were broken - the growth was 47%. The casket is easy to open - hundreds of millions of people were locked up at home, so the activity of using Internet messengers, online learning platforms, cloud gaming services and, of course, video platforms like YouTube has increased dramatically.
The most heavily loaded traffic exchange points were in Germany, Frankfurt (DE-CIX FRA) and in the Netherlands, Amsterdam (AMS-IX). In March, DE-CIX peer-to-peer center reached an all-time traffic peak of 9.1 Tbps .
In Russia, a similar situation was observed - in March-April 2020, online cinemas recorded a 2-4-fold increase in traffic. The reason is still the same - because of the pandemic and self-isolation, people were isolated in their own apartments and houses, so the Internet has become one of the ways to entertain themselves. Mobile traffic in the same period grew by 10-30%, primarily due to video chats and instant messengers.
It got to the point that Netflix and YouTube loweredvideo quality for European users. YouTube started showing videos in standard definition instead of HD, and Netflix dropped the resolution by 25%.
What to do?
Ideally, increase the capacity and throughput of the network infrastructure, both locally and globally. Everything is complicated here, since providers of the Internet, cellular communication and other telecommunication services cannot always invest significant resources.
The second option is to optimize your content. This refers not so much to reducing the quality of video or audio, as other methods. We will consider them below.
Optimizing compression of different videos
Netflix is the largest provider of media content. Several years ago, she introduced the practice of selecting different compression profiles for different videos, taking into account their characteristics.
Netflix has a wealth of video experience, including content compression. Using its own developments, the company trained a neural-like system that evaluates low-level video attributes, determining its class, and then finds the optimal parameters for each class.
Parameters such as the dynamics of the picture, the degree of clarity and the overall complexity of the plot are evaluated. In a matter of minutes, the system evaluates the dynamics of the plot, the degree of clarity and the overall saturation of the "picture". Based on the analysis data, the system makes a decision on video optimization. So, if the show or broadcast is not very dynamic, then the video quality can be reduced. If the picture changes quickly, the colors are saturated, and the plot is complex, then the quality either decreases slightly or remains at the same level.
Thus, the optimization of video transmission is achieved, the load on the network infrastructure is reduced. Work with video is carried out in such a way that the end result is invisible to content consumers. In other words, the audience just doesn't notice anything. For compression, the company usesVarious codecs and compression models, including 4K VMAF , so that the network is not congested even when streaming 4K video.
Selecting the video source with optimal quality
A high resolution, and therefore a large amount of transmitted data, does not mean good quality video delivered to the recipient. An example of this is the gif posted above.
Choosing a good quality video source is an urgent problem for distributors of TV channel packages. Ssimwave has come up with a machine learning solution that automatically selects the content source with the highest quality.
An example is the broadcast of the CNN channel. In one source, video parameters are 1080 @ 29.97i, MPEG-2, 40 Mbps, and in the other - with 720p60, H264, 22 Mbps. As mentioned above, high resolution does not at all mean good video quality for the recipient. The quality is influenced by a large number of factors, including compression and color transfer formats, dynamic range, transcoding procedures, delivery technologies and versions of subscriber players.
It is simply impossible to assess all this manually. But the neural network is able to cope with the task without any problems. Ssimwave was able to develop a solution that allows you to select the transmission source with the highest quality and minimum data volume in a matter of seconds.
Not by traffic alone
Machine learning, artificial intelligence and neural networks help not only optimize the volume of transmitted content. There are many other areas where technology is simply indispensable now.
Video catalog navigation
Many media content companies have catalogs - with games, videos, movies, and shows. According to Netflix, when choosing a movie to watch, subscribers are most likely to pay attention to movie poster icons. According to the company's creative director Nick Nelson, in 82% of cases, the choice of a movie is determined by the presented icon / poster.
Accordingly, the more successful the image, the higher the likelihood that the viewer will choose this particular piece of content. All this is relevant not only for video, but also for other digital goods.
To confirm or disprove Nelson's opinion, Accedo has partnered with AWS and British Television Corporation ITV to run an A / B test to identify factors that influence user choices. The conclusion is not too unexpected: users choose a movie, guided by their emotions when viewing screenshots. It takes less than 2 seconds to evaluate the screenshot.
Accordingly, machine learning and AI technologies can be used to select screenshots with optimal conversions. For films, such screenshots usually include an image of a hero whose face expresses emotions, and most often pictures with antagonists are chosen. Screenshots with groups of heroes are of less interest.
In addition, artificial intelligence helped form different collections for different categories / segments of viewers - both socially and geographically. Now it is not surprising that catalogs look different in different countries or regions of the same country. But just 10-20 years ago, the situation was different.
Increasing the information content of the video
It is about creating and structuring multimedia metadata for different videos - for example, sports videos. IBM Watson, an IBM supercomputer with built-in artificial intelligence, stood out in particular here. He can create visual descriptions in real time, transcribe audio, add editorial notes.
These skills are used while working with popular sporting events - for example, the World Cup, US Open, Super Bowl and others.
AI processes the broadcast stream, marking key points in it, adding notes and comments. Further, the already processed stream is either sent to the editors of the broadcasting program, or it is broadcast directly to the viewers.
Reduced customer churn
Machine learning and artificial intelligence are great at preventing customer churn. This happens if the interest of users in the content decreases. According to our data, uninteresting or overly complex content can cause churn for a quarter of users. This is a lot and this should not be allowed.
In order to reduce the churn rate, it is important to:
- understand what can cause churn
- automate analytics
- use forecasting tools
Analytics uses data such as user information, content viewability, user ratings, user interaction activity with support (here we can talk not only about videos, but also about online courses, as in our case) and other data. For example, on the We.Study platform , the system itself regularly "monitors" the course, providing specific recommendations for improving it, allowing you to track the behavior of participants and predict churn.
Based on the results of the analysis, certain actions can be taken - to change the content, the training program, if these are courses - to improve interaction with users.
Personalizing featured content
All YouTube users know what recommended content is. The more successful the recommendation, the higher the chance that the user will watch the video or view another type of content.
Illustrative case - the IBM Watson project and the Iris.tv platform. The partners managed to achieve optimal content personalization. To do this, the videos are first passed through a supercomputer that collects metadata. Then the platform analyzes this data to create new categories and titles of the film library.
In addition, with the help of machine learning, the platform was able to train to analyze the interests of specific users in order to guarantee to offer a video interesting to a person after he watched the next program.
What's next?
There are a lot of cases of using AI and machine learning, so only the most revealing points are indicated in the article. The general conclusion is that these technologies are beginning to be used systematically. Previously, they were used only as a test. Now hundreds and thousands of content companies are using machine learning to optimize content, debug business processes, attract new users, and retain old ones.
Over the next 3-5 years, the vast majority of media content providers will switch to the use of modern technologies, due to the fact that they allow solving the problems of both individual customers and the business as a whole.