Resolving Delay in HEIF Image Processing During Screen Recording on macOS

Hello everyone,

I'm currently facing a challenging issue with my macOS application that involves HEIF image processing. The application uses an OperationQueue to handle HEIF compression tasks. However, I've observed a significant delay in processing when a screen recording is active. This delay doesn't occur under normal circumstances.

Here's a brief overview of the implementation:

  • The HEIF processing task is encapsulated within an Operation added to an OperationQueue.
  • The task involves using CIContext for image processing.
  • When screen recording is initiated, the operation's execution becomes unusually slow or gets delayed extensively.
  • After some research and community feedback, I learned that screen recording might be affecting the system's resource allocation, particularly impacting tasks that utilize GPU resources, like CIContext operations in my case.

To address this, I tried the following:

Switching to a custom dispatch queue with a .userInitiated QoS. Using GCD instead of OperationQueue. Despite these attempts, the issue persists during screen recording. It seems like the screen recording process is given higher priority by macOS, leading to resource reallocation and thus affecting my application's performance.

I'm looking for insights or suggestions on how to handle this scenario more effectively. Specifically, I am interested in:

  1. Understanding how screen recording impacts resource allocation in macOS.
  2. Exploring ways to ensure that my HEIF processing task is not severely impacted by other system processes like screen recording.
  3. Any best practices or alternative approaches for handling image processing tasks that are sensitive to system resource availability.

Here's a snippet of the HEIF processing function for reference:

import CoreImage

struct CommandResult: CustomStringConvertible {
    let output: String
    let error: Process.TerminationReason
    let status: Int32
    
    var description: String {
        return "error:\(error.rawValue), output:\(output), status:\(status)"
    }
}


func heif(at sourceURL: URL, to destinationURL: URL, as quality: Int = 75) -> CommandResult {
    let compressionQuality = CGFloat(quality) / 100.0
    guard let ciImage = CIImage(contentsOf: sourceURL) else {
        return CommandResult(output: "Load heic image failed \(sourceURL)", error: .exit, status: -1)
    }
    
    let context = CIContext(options: nil)

    let heifOptions = [kCGImageDestinationLossyCompressionQuality: compressionQuality] as! [CIImageRepresentationOption: Any]
    do {
        try context.writeHEIFRepresentation(of: ciImage,
                                            to: destinationURL,
                                            format: .RGBA8,
                                            colorSpace: ciImage.colorSpace!,
                                            options: heifOptions)
    } catch {
        return CommandResult(output: "Compress and write heic image failed \(sourceURL)", error: .exit, status: -1)
    }
    return CommandResult(output: "Compress and write heic image successfully \(sourceURL)", error: .exit, status: 0)
}

Thank you for your time and any assistance you can provide!