180VR streaming expectations

I have a VisionOS app that streams 180VR video to create immersive experiences, similar to the Apple TV+ immersive content. These videos are 8k60fps. When looking at "video encoding requirements" (1.25) in the hls authoring specifications, there are only specifications for up to 4k30fps for MV-HEVC. Using the "power of .75 rule" I think that 160000kbps could be close to the 8k60fps recommendation.

For launching my app, I did my best guess in creating the following multivariant playlist, with a generous low end and very generous high end, with 6 second segment durations for all variants. However, in practice, users seem to be to only be picking up the lower quality bandwidth content (80000kbps) even when speed tests are showing 5x that on the device. This leads to a lot artifacts in the content during playback, since its lower quality. If I hard code a higher bit rate variant (like 240000kbps) it does playback, but obviously has a bit more lag to start up.

Now that I have my Vision Pro, I've been able to see the Apple TV+ immersive content. I could be wrong, but it doesn't feel like its varying playback - when watch by tethering to my phone vs on high speed wifi, the content looks the same just a little slower to load on the phone.

I'm looking for 3 points of guidance:

  1. Are there hls recommendations for 8k60fps video? For both bitrates and target duration for the segments?
  2. Any guesses as to why I am not able to pick up the high bitrates? (this could simply be that the higher ends are still too high)
  3. While 180VR is just stereo video on a larger canvas, the viewing experience is quite different due to the immersion. Are there special recommendations for 180VR video? (Such as having only one variant and specified bitrate since a changing bitrate/video quality could be jarring to the viewer)

Example HLS multivariant playlist:

#EXTM3U
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=80632574,BANDWIDTH=82034658,VIDEO-RANGE=SDR,CODECS="mp4a.40.2,hvc1.1.60000000.L183.B0",RESOLUTION=4096x4096,FRAME-RATE=59.940,CLOSED-CAPTIONS=NONE
https://myurl.com/stream/t4096p_080000_kbps_t6/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=160782750,BANDWIDTH=162523567,VIDEO-RANGE=SDR,CODECS="mp4a.40.2,hvc1.1.60000000.L186.B0",RESOLUTION=4096x4096,FRAME-RATE=59.940,CLOSED-CAPTIONS=NONE
https://myurl.com/stream/t4096p_160000_kbps_t6/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=240941602,BANDWIDTH=243997537,VIDEO-RANGE=SDR,CODECS="mp4a.40.2,hvc1.1.60000000.L186.B0",RESOLUTION=4096x4096,FRAME-RATE=59.940,CLOSED-CAPTIONS=NONE
https://myurl.com/stream/t4096p_240000_kbps_t6/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=321061516,BANDWIDTH=325312545,VIDEO-RANGE=SDR,CODECS="mp4a.40.2,hvc1.1.60000000.H255.B0",RESOLUTION=4096x4096,FRAME-RATE=59.940,CLOSED-CAPTIONS=NONE
https://myurl.com/stream/t4096p_320000_kbps_t6/prog_index.m3u8
Up vote post of altonelli Down vote post of altonelli
565 views
Posted by
altonelli
Reply
Add a Comment

Replies

I haven't seen any specific recommendations from Apple, but for comparison, the "Highlining" video appears to be delivering 4320x4320, HRD10, 90fps content in fisheye projection at ~50Mbps for its best presentation.

Posted by
Michael Swanson
Up vote reply of Michael Swanson Down vote reply of Michael Swanson

  • Interesting, where are you seeing those numbers? Their streams seem exceptionally performant, but ~50Mbps seems pretty low for that content. The specifications for 3840x2160 HDR at 30fps is 34000 kbps - doubling that to get a similar resolution or tripling to get a similar frame rate greatly exceeds 50Mbps. (Not to cast too much doubt in that number - Apple has a much better production and distribution pipeline than an indie developer)

    —
    altonelli

  • Monitoring network traffic. As someone who spent 4+ years doing immersive video, those numbers don't seem that off to me. Of course, I haven't created any content at those specifications (yet), so YMMV.

    —
    Michael Swanson
Add a Comment

In terms of not picking up the high bitrates, please try specifying your own resolution limit by setting the .preferredMaximumResolution property.

Posted by
PuppyDog
Up vote reply of PuppyDog Down vote reply of PuppyDog
Add a Comment