Explore dynamic pre-rolls and mid-rolls in HLS

♪ ♪ Hi, everyone! I'm Prashant, an AVFoundation Engineer. Welcome to WWDC21. In recent years, the popularity of over-the-top streaming services have approached that of linear television. Depending on the market that you're in, your users might actually prefer free ad-supported content over ad-free content that they need to pay for. So if you're interested in including ads and other interstitials in your HLS streams, you've tuned in to the right session. Before we dive in, let's quickly review some of the existing mechanisms for inserting ads in HLS streams. You could perform server side ad insertion using DISCONTINUITY tags. This is a fairly static scheme in that the ad segments are stitched together with the content segments, and this stitching has to be done even before the playlist is vended to the user. As you might imagine, this doesn't allow for late binding of your ad assets where you'd like for an ad to be decisioned as the user is approaching the ad marker or even rebinding, say, when a user navigates to a portion of the video that they've already watched, they're going to see the same ad all over again. Since HLS operates at a segment granularity, transitions to and from ad pods need to happen at segment boundaries. If your ad marker happens to be present in the middle a content segment, you'd have to split up that segment in order to stitch in the ad. Oftentimes, the number of quality tiers would be different between your primary and interstitial assets. In order to stitch the ad together with all the primary quality tiers, the ads would need to be conditioned to match the content encoding. And you'd also want to make sure that the ads use the same codecs as the primary content. And in live streaming scenarios, your packager might have to do a lot of back-end bookkeeping. Your ads and interstitials would mostly be video-on-demand streams. But the packager would have to spool these out on a per-segment basis for the entirety of the ad break. You could also insert ads on the client side. Until now, we didn't have an official recommendation on how this can be done. One of the more popular approaches is to use a two-player scheme where one player would present the primary content while the other would present the interstitials. And transitions are achieved by coordinating playback between the two players and by managing player view hierarchies. While this works well, the issues here are mostly on the performance side. If buffering is not properly coordinated between the two players, pre-fetching your ad would impact the adaptive bit rate performance of the primary content stream. And features like Picture in Picture and AirPlay are really hard to do well with these custom player approaches. This year, we've addressed all these concerns and in general attempted to make ad insertion a lot more simple. We're introducing HLS interstitials where ads are treated as separate assets that can be scheduled onto a program timeline. Ads are no longer stitched in with DISCONTINUITY tags. Instead they remain as self-contained assets that can be referenced via their master playlist. As we'll soon see, this scheme is dynamic, allowing for late binding and even rebinding to your ad inventory. You're no longer constrained to segment boundaries. You could arbitrarily place ads anywhere on the program timeline. HLS interstitials in conjunction with AVKit offers built-in support for navigation restrictions on tvOS. We also have built-in support for AirPlay and Picture in Picture. All your scheduled ads and interstitials will be carried over the AirPlay session. We also coordinate buffering and the use of other system resources between the primary and interstitial players so as to achieve seamless transitions. Now let's take a look at a typical playback flow when we present some video-on-demand content along with ads using HLS interstitials.

The blue bar in this picture represents your main content, and the orange and green bars are the ads that you'd like to schedule during playback. Now, the primary would play up until the ad marker, at which point it would pause, and the first ad would start playing. Once the first ad plays out, the second ad would immediately follow. After the second ad, the primary would resume from where it left off. Now let's take a look at the buffering sequence. We start by buffering the primary up until the start time of the first ad. Now we'd pre-buffer the first ad, making it possible to achieve a smooth transition. After this has completely buffered, we'd pre-buffer the second ad. And after the second ad has completely buffered, we'd start buffering the primary again so that we switch back seamlessly. The playback flow for live scenarios is similar, except that we'd rejoin the primary after jumping forward by the ad duration. This is so that we remain in sync with the live edge. And this also adopts a similar buffering strategy such that seamless transitions are possible. So how would you schedule ads with HLS interstitials? You can now do server side ad insertion using DATERANGE tags. For this, we're introducing a new date range class with attributes. This playlist is carrying timing information through a PROGRAM-DATE-TIME tag. These tags are now mandatory since the ad schedule is now specified using dates. Here we see an ad lined up using the DATERANGE tag, with the class set to com.apple.hls.interstitial. The ID attribute uniquely identifies the event. START-DATE is where you'd like for the ad to start in the primary timeline. Here we see that the ad is scheduled at 5 seconds into playback. The DURATION attribute specifies the duration of the ad. The X-ASSET-URI attribute specifies the URI of the ad's master playlist. And the X-RESUME-OFFSET attribute specifies the offset from the START-DATE, where you'd like primary playback to resume. A resume offset of 0 would mean that the primary would resume from where it left off. If the resume offset attribute is absent, we'd rejoin the primary at an offset which equals the ad duration. This is probably what you'd want to do for live scenarios.

Sometimes you might wish to skip over in-stream ads as shown here. For that, you'd simply specify the resume offset to be the duration of the in-stream ad. You can schedule multiple ads using DATERANGE tags. Here we see the first ad scheduled at 5 seconds into the primary and the second one at 10 seconds. You can schedule back-to-back ads simply by specifying the same start time for each of them. Here we see that the first ad starts at 5 seconds into playback, and so does the second. The ads would be displayed in the order that they appear in the playlist. These approaches where you use the X-ASSET-URI attribute to reference your ad would require you to define your ad pod when you're adding the DATERANGE tag to the playlist. But you could defer that decision using the X-ASSET-LIST attribute. The X-ASSET-LIST points to a JSON object that contains the event schedule. The object has an ASSETS array that specifies the list of interstitials, with each entry specifying the URI of the master playlist and the duration. Note that this JSON object would be fetched only at buffering time, allowing for late binding to your ad inventory. Usually, ads would play out fully to the end. However, you can specify an end time for your ad using the X-PLAYOUT-LIMIT attribute. You can use this attribute to implement early return scenarios in live broadcasts. An early return is where you'd like to interrupt your ad break in order to return to the live feed. This could be due to a breaking news scenario, or maybe there was some sudden excitement on the sports field that you don't want your viewers to miss out on. So how would you use this to implement early return? Here we have a live playlist that contains six segments. The live edge is at the end of segment six. Now the playhead would usually be three target durations behind the live edge. In this case, it's at the end of segment three. At this point, a 15-second ad spot is decisioned to follow segment six. So the playlist update actually contains all six segments along with a DATERANGE tag carrying the ad schedule. In the next update, the playhead would've moved to the end of segment four, and the ad is still scheduled to follow segment six. The shaded segment seven here simply represents the ad break. In the following update, the playhead has moved to the end of segment five, and in the next update, we're at the beginning of the ad break. If you don't do anything, the ad would play out for 15 seconds. But if you want to schedule an early return, say, after 12 seconds, you'd simply specify the play-out limit to be that. Now your ad would play for 12 seconds before returning to the program. Oftentimes for contractual reasons, you'd want to prevent your user from jumping over or skipping through the ads. You can specify navigation restrictions using the X-RESTRICT attribute. X-RESTRICT with a value jump would prevent a user to seek from a time before the ad to a time after the ad. An X-RESTRICT with a value skip would prevent one from playing the ad at a rate different from what is desired. Note that these restrictions are enforced by the UI. On tvOS, they're enforced by AVKit. And they should be available to you if you use the AVPlayerViewController for your video presentation. When you're on other platforms or if you're not using AVKit, it's up to your application to enforce these restrictions. Now that we've seen how you could schedule server side ads using DATERANGE tags, let's see how you can monitor their progress at the client. For this, we're introducing two new AVFoundation objects: the AVPlayerInterstitialEventMonitor that notifies the client when an interstitial is scheduled or playing and the AVPlayerInterstitialEvent object which contains all the information necessary to place ads on a player item's timeline. The AVPlayerInterstitialEventMonitor has the following properties: a primary player that is playing your primary asset, a handle to the interstitial player that you can use to monitor playback of the ads, an events array which is an array of AVPlayerInterstitialEvent objects that represent the different interstitials set on the player. We'll get to the description of the AVPlayerInterstitialEvent object shortly. A handle to the currentEvent. So this would be valid when an interstitial is playing and null otherwise. And then we have the eventsDidChangeNotification which would fire when the event schedule changes, and finally the currentEvent- DidChangeNotification which would fire when we transition to and from interstitials. The AVPlayerInterstitialEvent object that describes an interstitial event has properties that are basically analogues of the DATERANGE attributes that we saw earlier. The primaryItem represents your primary asset on whose timeline you'd like to schedule the interstitials. The identifier is similar to the ID attribute and this uniquely identifies the event. The time and date fields specify the start time of the interstitial in media time and date respectively. This would be like the START-DATE attribute. We'd use a copy of the template items to create the interstitial player items that'd represent your ad pod. So this is similar to the ASSET-LIST attribute that we saw earlier. The restrictions property would specify the navigation restrictions for your interstitials. Then we have the resumptionOffset and playoutLimit properties that are similar to their DATERANGE counterparts. And finally we have a dictionary of userDefinedAttributes. So you can specify custom attributes in the DATERANGE tag, and these would be surfaced to your client application through the userDefinedAttributes property. For instance, you might include a beacon URL and other custom attributes for reporting ad playback metrics. Here's some example code that shows how you might use these APIs to update your UI when an interstitial is playing. You create an AVPlayer to play your primary content. So this content has interstitials scheduled using the DATERANGE tags. And then create an AVPlayerInterstitialEventMonitor and set it on the player. You then subscribe to the currentEvent- DidChangeNotification to inform your application when the player transitions in and out of interstitials. And when that notification fires, you simply update your UI. Sometimes you might want to schedule ads on the client side. For that, we're introducing the AVPlayer- InterstitialEventController that lets you programmatically set events on an AVPlayer. This object inherits from the AVPlayerInterstitialEventMonitor and so has a lot of properties in common. One thing to note is that while the events property is read-only at the monitor, it is a read-write property at the controller, thus letting you programatically schedule events. And this also has the cancelCurrentEvent API that lets you cancel the currently playing interstitial. In this sample, we see how one might schedule an ad pod containing two ads on an AVPlayer. You create an AVPlayer to play your primary asset. You then create an AVPlayer- InterstitialEventController and set it on the AVPlayer. You create an array of AVPlayer items that represent your ad pod. Then create an AVPlayerInterstitialEvent object where you specify the primaryItem to be the player's current item, as this represents your movie asset, the start time in this example, some 10 seconds into playback, and the templateItems to be the ad pod that we created earlier. And after the object is created, you simply set it on the events property of the controller. You might notice that just before we set the player on the AVPlayerViewController, we set the currentItem.translates- PlayerInterstitialEvents property to true. When this is set, AVKit would place navigation markers on the timeline and also enforce navigation restrictions on tvOS. Now let's cut to a demo where we go over AVKit integration with HLS interstitials on tvOS.

Here we have an ad scheduled to start at some 40 seconds into playback. And this ad has no navigation restrictions set. You can see that the ad marker is visible on the timeline.

As the ad is playing, a countdown timer appears over the playhead. And since this ad has no restrictions set, as I glide my hand over to the edge of the Apple TV remote, I'd get an option to skip. And doing so would cause us to skip out of the ad and go back to the primary. Now for this one, we have the same setup, but the ad has the skip restriction set.

So now when the ad plays, you will not see an option to skip out of the ad. You will have to wait for it to play out completely before returning to the primary. Tom Hanks in "Greyhound." Rated PG-13. Exclusively on Apple TV+. Thank you, Josh.

Here we have two ads as we see on this timeline. The first with no navigation restrictions and the second with a jump restriction. So when we attempt to jump past both the ads...

We see that the playhead snaps to the restricted ad marker while the ad plays.

We'll bring hell down from on high.

Tom Hanks in "Greyhound." Rated PG-13. Exclusively on Apple TV+. And after the ad plays out, we'd resume from the seek location.

To wrap up, you can now schedule server side ads using DATERANGE tags. You'd want to specify the resume offset to be 0 for video-on-demand streams such that you'd rejoin the program from where you left off. And you could skip this attribute for live streams so that we rejoin at the live edge. You can use the X-ASSET-LIST attribute for late binding of your ad assets, schedule early return in live broadcasts using the X-PLAYOUT-LIMIT attribute, and specify navigation restrictions with the X-RESTRICT attribute. To monitor ad playback on the client side, you'd use the AVPlayer- InterstitialEventMonitor. And you can programatically set ads using the AVPlayer- InterstitialEventController. Thank you for watching and enjoy the rest of WWDC21.