Using AVAudioEngine to monitor audio in VisionOS

I am writing code to monitor the incoming audio levels in VisionOS. It works properly in the simulator, but gets an error on the device. Curious if anyone has any tips.

I took out some of the code so it's a bit shorter, as it fails in setupAudioEngine when I try to start the engine with this error:

Error starting audio engine: The operation couldn’t be completed. (com.apple.coreaudio.avfaudio error 561145187.)

Thanks in advance!

Here is my code:

class AudioInputMonitor: ObservableObject {
    private var audioEngine: AVAudioEngine?
    @Published var inputLevel: Float = 0

    init() {
        requestMicrophonePermission()
    }
    
    private func requestMicrophonePermission() {
        AVAudioApplication.requestRecordPermission { granted in
            DispatchQueue.main.async {
                if granted {
                    self.setupAudioSessionAndEngine()
                } else {
                    print("Microphone permission not granted")
                    // Handle the case where permission is not granted
                }
            }
        }
    }
    
    private func setupAudioSessionAndEngine() {
        do {
            let audioSession = AVAudioSession.sharedInstance()
            try audioSession.setCategory(.playAndRecord, mode: .measurement, options: [])
            try audioSession.setActive(true)
            
            self.setupAudioEngine()
        } catch {
            print("Failed to set up the audio session: \(error)")
        }
    }
    
    private func setupAudioEngine() {
        audioEngine = AVAudioEngine()
        
        guard let inputNode = audioEngine?.inputNode else {
            print("Failed to get the audio input node")
            return
        }

        let recordingFormat = inputNode.outputFormat(forBus: 0)
        inputNode.installTap(onBus: 0, bufferSize: 1024, format: recordingFormat) { [weak self] (buffer, _) in
            self?.analyzeAudio(buffer: buffer)
        }
        
        do {
            try audioEngine?.start()
        } catch {
            print("Error starting audio engine: \(error.localizedDescription)")
        }
    }
    
    private func analyzeAudio(buffer: AVAudioPCMBuffer) {
      // removed to be brief
    }
    
    func stopMonitoring() {
      // removed to be brief
    }
}

Accepted Reply

I figured out my own problem.

The Vision Pro hardware really doesn't like starting the audio engine during launch. I wanted to minimize the amount of UI in my app, and needed the audio input the whole time, so I figured just start as soon as I have permission. No go. But if I separate out the audioEngine?.start() into a startMonitoring() call, and call that after a timer fires from my ContentView.onAppear (or I guess I could add a button for the user to start it), all works great!

Replies

I figured out my own problem.

The Vision Pro hardware really doesn't like starting the audio engine during launch. I wanted to minimize the amount of UI in my app, and needed the audio input the whole time, so I figured just start as soon as I have permission. No go. But if I separate out the audioEngine?.start() into a startMonitoring() call, and call that after a timer fires from my ContentView.onAppear (or I guess I could add a button for the user to start it), all works great!