Live streaming in the past year has garnered significant attention in the video industry, particularly with sports. In an interview with Fierce Video, Applause Chief Technology Officer Rob Mason shed some light on what goes into preparing a video live stream as well as the hurdles that arise from testing for large-scale events.
Applause helps companies including Amazon, Google, Facebook, among many others, test products and prepare for platform launches as well as other digital assets including AR/VR, bank transactions and streaming. Notably, Applause has a hand in testing live streamed events, such as for the NFL.
“Video isn’t just like a screenshot where you can take it and look at it. It’s movement and how those things move that matters,” said Mason. “Also, the way the world has evolved with ads and everything means there’s global ads and local ads and how those all get placed and sequenced in.”
Obviously, the quality of the stream is crucial, as companies need to ensure they have enough bandwidth to deliver high quality video to all their target markets worldwide. Viewers want quality from the moment they press ‘play’ to the minute they stop watching.
Another key consideration for stream testing, Mason said, is making sure video quality is consistent across a wide range of devices, including mobile and connected TVs.
"There are like five or six different set top boxes that are fighting for coverage, there’s also all the different versions and models of those,” he explained, citing Roku as an example. “And you have to work on all of those, because you don’t know what people are going to get or what they’re going to have in even one house.”
In the case of live streams, bandwidth and device compatibility still matter but there is also a focus on “the things that change during live testing,” such as ad placements, he said.
Advertising hurdles
When the experience is live, there are more things that can go wrong, Mason noted.
“Now you’re not just the testing community feeding off those streams but you’ve also got all the people around the world also feeding off the streams,” said Mason. “As you’ve got that extra load on the system, has that changed the experience at all? Maybe we had thousands of testers testing it but it’s not the same as five million people watching that football game."
Advertising introduces its own set of complications, as the types of ads vary depending on the market or geography. Mason noted a variety of factors that have to be considered when testing ad placements. For instance, if they’re relevant to the watcher, if the whole ad playing or if it’s cut off, or if ads for two competing companies are placed next to each other. Determining how many times an ad is seen in the target period is also critical.
“These are things that are very difficult to automate, because it’s a stream, it’s a long period of time and we’re not yet at the point of technology where you can have a computer watch a long video and tell you, ‘this ad came up four times,’” he said. “It seems basic, but these are the kinds of things that we are asked to check.”
While Applause isn’t involved with the delivery of the actual product, it’s tasked with ensuring everything runs smoothly once it’s live, said Mason.
“We’ll get access to streams before they go live and then we’ll also watch live streams if [the company] asks us to do that,” he said. “To ensure that pre- and post-production things are meeting expectations.”
Stream testing can also get tricky if the content is behind a paywall, as a platform needs to verify that the viewer can watch the event.
“Any time you’ve got a live stream you have to make sure people are allowed to see whatever they’re seeing, especially if it’s a video on-demand market,” Mason pointed out.
Crowdtesting
To enhance its testing capabilities, Applause leverages a global pool of volunteers who are trained to identify technical issues in streaming video. Crowdtesting is a key facet of Applause’s work because “you have to have real people watching [those streams],” said Mason.
For example, if someone likes to watch football and has access to different connected devices, then the company can assign them to test an NFL stream.
“A lot of the streaming testing we’re doing is structured testing. We’re saying, ‘watch this video or show at this time and then write down every ad you see, what time you saw it, what it was about, was it complete, whether there were any video glitches,” he explained. “In a way, we’re programming people, but they’re real people with real devices in the real world.”
How much testing goes into a stream or event depends on the company, as some “do not want you to see anything about it pre-production, because they’re holding things closely.”
While other companies “want to test thoroughly pre-production because they want to make sure every single device is working properly,” Mason said, noting Disney for instance is “very fanatical about the quality of pixel-perfect streams and placements.”
“The ones that do it well engage you pretty much pre- and post-production,” he added. And given how competitive streaming has become, the players in the space are using different standards for testing their technology.
“There’s a lot of different opinions on how fast a stream should forward when you press the forward button on your remote control and how that works in pauses and rewinds…everyone does it a bit differently,” Mason said.
What works for one streaming app won’t necessarily translate well to another platform. The framework for Roku, for instance, is “radically different” from that of Apple TV or the Amazon Fire streaming stick.
“These are completely different platforms, different languages, different approaches,” he continued. “In the mobile world, there are some technologies where you can write once and deploy it multiple times in mobile apps. That’s not true in this set top box world.”
Future of live streams
A few streamers are tackling the live event space. Netflix recently scored a multi-year deal to stream the Screen Actors Guild Awards and it will also live stream a new Chris Rock stand-up comedy special in March. YouTube, which secured the rights for the NFL Sunday Ticket, will host the sports package on its platform starting later this year.
“People are trying to engage you now and keep you engaged with good content,” said Mason. “There are some [streaming platforms] that have a lot of users in them, like Netflix…they’re not really known for live streaming, so there’s an opportunity there to branch off into that.”
More players will likely try to get into the live streaming game, but Mason expects the winners will be the bigger, more established streaming platforms “because it’s hard to do this at scale and have that content.”