Disney unveils AI-powered video compression research

Study: neural network market to grow 20.5% through 2024
University of California, Irvine and Disney researchers said they will continue to work toward a real, applicable version of the video compressor. (Pixabay)

Disney Research and the University of California, Irvine are showing off a new artificial intelligence-enhanced video compression model that’s in the early stages of development.

The new compression model has apparently yielded less distortion and smaller bits-per-pixel rates than current coding-decoding algorithms such as H.265 when trained on specialized video content. The researchers said they achieved comparable results on downscaled YouTube videos.

“Ultimately, every video compression approach works on a trade-off,” said research team leader Stephan Mandt, a UCI assistant professor of computer science who began the project while employed at Disney Research. “If I’m allowing for larger file sizes, then I can have better image quality. If I want to have a short, really small file size, then I have to tolerate some errors. The hope is that our neural network-based approach does a better trade-off overall between file size and quality.”

Sponsored by IBM

Webinar: Delighting Viewers with Content: Cloud Enabled Remote Production

Wednesday, June 24, 2020 | 12pm EST / 9am PST
IBM Aspera's multi-cloud roots enables broadcasters to move terabytes of video content at maximum speed, allowing production teams around the world to co-create, untethered by location. Hear how Fox Sports delighted millions of Super Bowl LIV and 2019 FIFA World Cup viewers by producing more content than ever, delivered to viewers across broadcast and web devices alike.

Mandt said the video compression still relies heavily on predictive capabilities. Current compression algorithms achieve this through engineered solutions like trying to compute the linear displacement of small, localized patches relative to their position on the previous frame.

RELATED: Disney recruits Microsoft for cloud production, distribution projects

“In contrast, deep neural networks take a datacentric approach and learn the video’s underlying dynamics by drawing on large datasets of video material,” wrote UCI in a news release. “These data-driven methods, enabled by advances in deep learning over the past decade, show promise for shrinking video file sizes in future generations of video compression codecs.”

The UCI/Disney Research team’s compression process first involves downscaling the dimensions of the video using a “variational autoencoder,” or a neural network that processes each video frame in a sequence of actions that results in a condensed array of numbers. The autoencoder then tries to undo this operation to ensure that the array contains enough information to restore the video frame. Then the algorithm attempts to guess the next compressed version of an image given what has gone before, relying on an AI-based technique called a “deep generative model.”

Mandt said this approach is not necessarily unique but what sets apart the Disney/UCI research is that the algorithm conducts an operation to encode frame content by rounding the autoencoder’s real-valued array to integers, which are easier to store than real numbers. Finally, the model applies lossless compression to the array, allowing for its exact restoration. During this process, the neural network informs the algorithm about which video frame to expect next, making the lossless compression aspect more efficient.

“The real contribution here was to combine this neural network-based deep generative video prediction model with everything else that belongs to compression algorithms, such as rounding and model-based lossless compression,” said Mandt. “Because the receiver requires a trained neural network for reconstructing the video, you might also have to think about how you transmit it along with the data. There are lots of open questions still. It’s a very early stage.”

Suggested Articles

It’s been almost a year since we last broke down the timeline for price increases at YouTube TV, AT&T TV Now and other virtual MVPDs.

Sling TV is responding to new price increases from competitors YouTube TV and fuboTV by offering its subscribers a one-year rate guarantee.

Peacock has reached a deal with ViacomCBS to carry CBS and Showtime series along with Paramount films when it launches.