Processing / tapping an HLS audio stream (or global app output)

I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). It doesn't seem like attaching an AVAudioMix configured with with an `audioTapProcessor` has any effect; none of the callbacks except `init` are being invoked. Is this a known limitation? If so, is this documented somewhere?


If the above is a limitation, what are my options using some of the other audio APIs? I looked into `AVAudioEngine` as well but it doesn't seem like there's any way I can configure any of the input node types to use an HLS stream. Am I wrong? Are there lower level APIs available to play HLS streams that provide the necessary hooks?


Alternatively, is there some generic way to tap into all audio being output by my app regardless of its source?


Thanks a lot!

Up vote post of c0delogic Down vote post of c0delogic
4.0k views
Posted by
c0delogic
Reply
Add a Comment

Accepted Reply

The MTAudioProcessingTap is not available with HTTP live streaming. I suggest filing an enhancement if this feature is important to you - and it's usually helpful to describe the type of app you're trying to design and how this feature would be used.

Posted by
theanalogkid
Up vote reply of theanalogkid Down vote reply of theanalogkid
Post marked as solved
Add a Comment

Replies

Bumping this & jumping on the notifications, as I too would really like to know if this is possible.

Posted by
JacobSyndeo
Up vote reply of JacobSyndeo Down vote reply of JacobSyndeo
Add a Comment

The MTAudioProcessingTap is not available with HTTP live streaming. I suggest filing an enhancement if this feature is important to you - and it's usually helpful to describe the type of app you're trying to design and how this feature would be used.

Posted by
theanalogkid
Up vote reply of theanalogkid Down vote reply of theanalogkid
Post marked as solved
Add a Comment

Hi theanalogkid,We are developing VR video player and recently we want to support spatial audio. Our player is based on AVPlayer to play HLS video (m3u8), which is 360 video with Ambisonics (https://en.wikipedia.org/wiki/Ambisonics). To process Ambisonics, we need a path to get PCM audio (MTAudioProcessingTAP) for hls stream, but it is not available as c0delogic reported. So is there plan to update this?

Posted by
gaudiolab
Up vote reply of gaudiolab Down vote reply of gaudiolab
Add a Comment
Hi, I am also working AVPlayer to play HLS audio (m3u8), and put 3D surround processing as like as gaudiolab, how can I put MTAudioProcessingTAP for HLS stream and let me know updates.
Posted by
JUN1OKAMURA
Up vote reply of JUN1OKAMURA Down vote reply of JUN1OKAMURA
Add a Comment
Same here.

We are developing an AR application and we want to capture the AR experience (HLS video with audio attached to an object).

As a hotfix we are capturing the audio over the speakers via the microphone and mixing it with a screen capture afterwards into one media file. Not very satisfying in terms of UX, quality and performance.
Posted by
bedrock
Up vote reply of bedrock Down vote reply of bedrock
Add a Comment
I am attempting to do visualizations (audio volume, visualized pitch and the like) in real time, and could use this ability.
Posted by
Eric_WVGG
Up vote reply of Eric_WVGG Down vote reply of Eric_WVGG
Add a Comment
Same here. We're trying to do simple capture of LPCM sample flowing into the hardware.
Posted by
austino222
Up vote reply of austino222 Down vote reply of austino222
Add a Comment

Any news??

Posted by
peurocs4
Up vote reply of peurocs4 Down vote reply of peurocs4
Add a Comment