I am trying to take a screenshot of each running window on the screen independently (even if not in the foreground). I am using SCScreenshotManager's captureImageWithFilter and SCContentFilter's initWithDesktopIndependentWindow. When I do this for every SCWindow in SCShareableContent's windows, I do get all the actual windows, but also get many mostly blank windows or windows that no ordinary user would consider to be an actual window. These include dozens of windows with no title, some with "Focus Proxy" title, some for menu bar icons, the wallpaper, the desktop icons, etc. I've implemented a naive solution that filters all windows that have no title, owningApplications with no title or bundle ID in a hardcoded blocklist (e.g. "com.apple.controlcenter", "com.apple.wallpaper.agent") and that helps, but is far from robust and is naturally fragile. Is there a recommended way to distinguish actual application windows or an overall better approach here? Thanks!

I am capturing a screenshot with SCScreenshotManager's captureImageWithFilter. The resulting PNG has the same resolution as the PNG taken from Command-Shift-3 (4112x2658) but is 10x larger (14.4MB vs 1.35MB). My SCStreamConfiguration uses the SCDisplay's width and height and sets the color space to kCGColorSpaceSRGB. I currently save to file by initializing a NSBitmapImageRep using initWithCGImage, then representing as PNG with representationUsingType NSBitmapImageFileTypePNG, then writeToFile:atomically. Is there some configuration or compression I can use to bring down the PNG size to be more closely in-line with a Command-Shift-3 screenshot. Thanks!

I use ScreenCaptureKit, CoreVideo, CoreImage, CoreMedia frameworks to capture screenshots on macOS 14.0 and higher. Example of creating CGImageRef: CVImageBufferRef cvImageBufferRef = ..; CIImage* temporaryImage = [CIImage imageWithCVPixelBuffer:cvImageBufferRef]; CIContext* temporaryContext = [CIContext context]; CGImageRef imageRef = [temporaryContext createCGImage:temporaryImage fromRect:CGRectMake(0, 0, CVPixelBufferGetWidth(cvImageBufferRef), CVPixelBufferGetHeight(cvImageBufferRef))]; I have the next results of profiling with XCode Instruments Memory Leaks & Allocations: there is constantly increasing memory usage, but no memory leaks are detected, and there are many calls to create IOSurface objects, that have been never released. The most part of memory - All Anonymous VM - VM: IOSurface. The heaviest stack trace: [RPIOSurfaceObject initWithCoder:] [IOSurface initWithMachPort:] IOSurfaceClientLookupFromMachPort I don't have any of IOSurface objects created by myself. There are low-level calls to it. In Allocation List I can see many allocations of IOSurface objects, but there are no info about releasing it. Due to this info, how can I release them to avoid permanent increasing memory consumption?

I am using Screen Capture Kit to capture the windows and record it. But from macOS Sanoma onwards I see a wired behaviour when I try to capture the window which is in Full screen mode. The CMSampleBuffer returned by Screen capture kit has empty space at the top of the full screen window content. The ContentRect attachment in CMSampleBuffer includes this empty space. So there is no way to know what is the actual window content in the CMSampleBuffer. In the CaptureCample sample code provided by Apple it does not enumerate the Full screen windows. I made a change in that to enumerate full screen windows. The issue is reproduced in that also. Attaching the Image of showing the empty space. Has anybody encountered this issue?

I am working on ScreenCaptureKit sample with SCContentSharingPickerObserver. My Target is SwiftUI based and calling Objective-C class method. I added [MyTarget]-Bridging.h and [MyTarget]-Swift.h I got compile error of unknown class name SCContentSharingPickerObserver in [MyTarget]-Swift.h. But I do not know how to fix this error since [MyTarget]-Swift.h is Xcode generated file. I set macOS target is 14.0, swift language ver is 5 Anyone know how to fix this error or waiting for Xcode update?

I have old ScreenCaptureKit sample downloaded on Oct 2022. That sample worked on Oct 2022. But it does not work on Apr 2024 on Sonoma 14.4.1 M1 MacBook. It only shows black screen. I also download updated ScreenCaptureKit sample and test it. It works on Sonoma 14.4.1 M1 MacBook. I noticed latest sample have SCContentSharingPicker and other changes. I have my screen capture application based on old ScreenCaptureKit sample. My app only shows black screen. Do I have to add SCContentSharingPicker and SCContentSharingPickerObserver on my application for capturing screen on Sonoma? Old way of screen capture without SCContentSharingPicker is not supported anymore on Sonoma?

Hello! I'm trying to find my own window or SCWindow in a SCDisplay. This is so I can automatically change the SCDisplay to record when I drag a window from a display to another. Is there any way to check the windows that are contained in every SCDisplay? Thank you in advance!

Hi, I'm new to AVAudioEngine(and macOS programming in general). I'm trying to mix microphone audio with ScreenCaptureKit audio using AVAudioEngine without playing it back. I've created a AVAudioPlayerNode and scheduling buffers in my SCStream handler: playerNode.scheduleBuffer(samples) and have connected the playerNode to the mainMixerNode. audioEngine.connect(audioEngine.inputNode, to: audioEngine.mainMixerNode, format: micFormat) audioEngine.connect(playerNode, to: audioEngine.mainMixerNode, format: format) The problem is that mainMixerNode plays the audio to the speaker creating a feedback loop. How can I prevent the mixer output from being played back. Also: Is this the best way of mixing microphone input with some other input? I ran into AVAudioEngine's manual rendering mode, which seems like the way to go for mixing audio without playing it back. However, I couldn't figure out how to connect microphone input to the AVAudioEngine in manual rendering mode?

If someone in Apple WWDR sees this, please take the feedback to heart and report it up the chain: When you announce that a technology is being deprecated — such as CGDisplayStream — and also publish WWDC sessions about the intended replacement — ScreenCaptureKit — then you also need to give third-party developers a clear deadline by which this technology will be deprecated so that they can plan engineering efforts around implementing the new feature, and have ample time to communicate this to their customers. If it's important for third-party developers to get on board with this change, you should use every available means to communicate this to them, including multiple email alerts to their registered email address. Additionally, if you plan to make a BREAKING change in a framework that results in a wildly different user experience, you should probably hold that off until the summer release for the next major OS. What you should definitely NOT do is roll out a new privacy prompt in a mid-year release of macOS; or give your developers, customers, and AppleSeed program participants zero advance notice that this alert is coming, ignore your own Human Interface Guidelines when designing said prompt, and perform no user experience design testing (aka "putting on your customer hat") during a presumed internal alpha testing cycle to refine the experience and still find the most effective and least annoying way to present this additional prompt and spur change with your third-party developers. Oh, wait, you've done exactly all those things the wrong way with respect to ScreenCaptureKit. Right now, a host of Apple device administrators and client platform engineers are sending mountains of feedback to you, and they're also scrambling to contact third-party developers to let them know this is coming. Most of the vendors being discussed in private forums are said to be caught off guard by this change. We anticipate that users are not going to like this, and there is no way we can manage it with MDM or configuration profiles. In short, the current experience is a ghastly mess. WE, the administrators, will get blamed for this, not the third-party developers. WE will have to explain to our leadership why this experience is terrible and cannot be managed. Engineers need deadlines to help plan their work and prioritize tasks. In this case, vendors have had no firm deadline for this effort. There's already precedence for Apple announcing estimated deadlines for deprecations and feature removals. You do your developers and customers a great disservice by not communicating schedules to them. Please do better. P.S.: Feedback filed as FB13619326.

Hello, I'm playing with the ScreenCaptureKit. My scenario is very basic: I capture frames and assign them to a CALayer to display in a view (for previewing) - basically what Apple's sample app does. I have two screens, and I capture only one of them, which has no app windows on it. And my app excludes itself from capture. So, when I place my app on the screen which is not being captured, I observe that most didOutputSampleBuffer calls receive frames with Idle status (which is expected for a screen which is not updating). However, if I bring my capturing app (which, to remind, is excluded from capture) to the captured screen, the frames start coming with Complete status (i.e. holding pixel buffers). And this is not what I expect - from capture perspective the screen is still not updating, as the app is excluded. The preview which the app displays proves that, showing empty desktop. So it seems like updates to a CALayer triggers screen capture frame regardless of the app being included or excluded. This looks like a bug to me, and can lead to noticable waste of resources and battery when frames are not just displayed on screen, but also processed somehow or/and sent over a network. Also, I'm observing another issue due to this behavior, where capture hangs when queueDepth is set to 3 in this same scenario (but I'll describe it separately). Please, advise if I should file a bug somewhere, or maybe there is a rational explanation of this behavior. Thank you

I found that didOutputSampleBuffer would not be called for long time when the screen is no change. It sometimes make me confused for if there something wrong. In my design, it will change to other screen shot method, such as CGWindowListCreateImage, when long time no data. But this is not what I expected. I set the minimumFrameInterval to 30 but it seems no work. [config setMinimumFrameInterval:CMTimeMake(1, 30)]; Is there any settings that can let me get a didOutputSampleBuffer, even given a CMSampleBufferRef with status SCFrameStatusIdle, called atlest one time per second? Which would make me think it works fine without any exception.

I notice from macOS Sonoma System Settings, we have "Screen & System audio Recording". I'm an macOS app developer and want to request only Audio permission, I browse the document for a while and WWDC code demo, but still have no idea of how to request "System Audio Recording Only" permission? All the demo and doc I can find is request "Screen Recording & System Audio"

When I try to initialize SCContentFilter(desktopIndependentWindow: window), I get a very weird error: Assertion failed: (did_initialize), function CGS_REQUIRE_INIT, file CGInitialization.c, line 44. This is a brand new CLI project on Sonoma 14.2.1. Any ideas?

Hello, I develop an application called MiniMeters and am using ScreenCaptureKit to get the desktop audio on macOS. As of macOS 14.2, a few users began noticing that the volume of the incoming audio differed depending on the audio device connected. One user's Apogee Symphony Desktop is trimmed -14dB, another user's UAD Apollo Twin is -14dB as well, my MOTU M4 is -6dB, and my MacBook Pro's internal speakers show 0dB. This does not change with changing the output volume on the interface (obviously), nor digitally in the system. I cannot seem to find anything documenting this change. It also affects other applications that use ScreenCaptureKit such as OBS. Does anyone have any idea what that could correlate to so I could potentially compensate? Thanks.

I try capture screen on wingon. For start app on winlogon i register LaunchAgent to start on LoginWindow. Application is started, but callback getShareableContentExcludingDesktopWindows newer called, and application just stuck.

The old one is "x-apple.systempreferences:com.apple.preference.security?Privacy_ScreenCapture"

So I'm building a colour sampler tool, similar to ColorSlurp and since CGWindowListCreateImage (which I am using for below macOS 14.0) is deprecated since Sonoma, I am wondering what's the best approach to replacing it. The way I use CGWindowListCreateImage currently is to take a screenshot of a specified area around the mouse pointer every time the mouse moves. Which works perfectly fine without issues. Now I've tried replacing CGWindowListCreateImage with SCScreenshotManager.createImage which is an async function and as you might expect, running async functions on mouse movements doesn't quite work out that well. It is lagging behind, heavily. So my question would be what's the appropriate ScreenCaptureKit methods to replace the functionality already created with CGWindowListCreateImage? Should I create a SCStream instead? Then I am worried about the fact that I would need to stream the whole screen instead of just the area around the mouse pointer since updating stream configs is an async function as well. I'd greatly appreciate any sort of direction!

My existing code is working properly in iOS < 17 devices it records the iPhone screen and records audio as well simultaneously, but in iOS 17 devices the screen recording video is captured for only 2 seconds and then stops automatically, As its an extension, i don't have logs to debug the issue. I have tested the same code in other iPhones and OS less than 17, its working fine but in iOS 17 devices this issue is coming. @try { NSLog(@“initAssesWriter”); NSError *error = nil; CGRect screenRect = [[UIScreen mainScreen] bounds]; _videoWriter = [[AVAssetWriter alloc] initWithURL: _filePath fileType:AVFileTypeMPEG4 error:&error]; NSParameterAssert(_videoWriter); //Configure video NSDictionary* videoCompressionProps = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithDouble:2048*1024.0], AVVideoAverageBitRateKey, nil ]; NSDictionary* videoSettings = [NSDictionary dictionaryWithObjectsAndKeys: AVVideoCodecTypeH264, AVVideoCodecKey, [NSNumber numberWithInt:screenRect.size.width * 4], AVVideoWidthKey, [NSNumber numberWithInt:screenRect.size.height * 4], AVVideoHeightKey, videoCompressionProps, AVVideoCompressionPropertiesKey, nil]; _writerInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:videoSettings] ; _writerInput.expectsMediaDataInRealTime = YES; NSParameterAssert(_writerInput); NSParameterAssert([_videoWriter canAddInput:_writerInput]); [_videoWriter addInput:_writerInput]; AudioChannelLayout acl; bzero( &acl, sizeof(acl)); acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono; NSDictionary* audioOutputSettings = [NSDictionary dictionaryWithObjectsAndKeys: [ NSNumber numberWithInt: kAudioFormatMPEG4AAC], AVFormatIDKey, [ NSNumber numberWithInt: 1 ], AVNumberOfChannelsKey, [ NSNumber numberWithFloat: 44100.0 ], AVSampleRateKey, [ NSData dataWithBytes: &acl length: sizeof( AudioChannelLayout ) ], AVChannelLayoutKey, [ NSNumber numberWithInt: 64000 ], AVEncoderBitRateKey, nil]; _audioWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType: AVMediaTypeAudio outputSettings: audioOutputSettings ]; _audioWriterInput.expectsMediaDataInRealTime = YES; // seems to work slightly better NSParameterAssert(_audioWriterInput); NSParameterAssert([_videoWriter canAddInput:_audioWriterInput]); [_videoWriter addInput:_audioWriterInput]; [_videoWriter setMovieFragmentInterval:CMTimeMake(1, 600)]; [_videoWriter startWriting]; } @catch (NSException *exception) { } @finally { } -(void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType{ @try { if(!_isRecordingStarted){ [_videoWriter startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp(sampleBuffer)]; _isRecordingStarted = YES; [self saveFlurryLogs:@"Assest writer Start Recording" Details:@""]; NSLog(@"CMSampleBufferGetPresentationTimeStamp"); } } @catch (NSException *exception) { [self saveFlurryLogs:@"Recording Start Execption" Details:exception.description]; } @finally { } @try { switch (sampleBufferType) { case RPSampleBufferTypeVideo: // Handle video sample buffer if([_writerInput isReadyForMoreMediaData]){ [_writerInput appendSampleBuffer:sampleBuffer]; NSLog(@"writing matadata Video"); } break; case RPSampleBufferTypeAudioApp: // Handle audio sample buffer for app audio break; case RPSampleBufferTypeAudioMic: if([_audioWriterInput isReadyForMoreMediaData]){ [_audioWriterInput appendSampleBuffer:sampleBuffer]; NSLog(@"writing matadata Audio"); } // Handle audio sample buffer for mic audio break; default: break; } } @catch (NSException *exception) { [self saveFlurryLogs:@"Packet Write Execption" Details:exception.description]; } @finally { } }

Trying to integrate the new screencapturekit into our application. The stand alone test we made works fine, however when integrated, when we start the stream capture we get this error in the logs (ScreenCaptureKit) [ERROR] _SCStream_RemoteAudioQueueOperationHandlerWithError:1032 Error received from the remote queue -16665 Any insights what might be causing this? this is what we're passing addStreamOutput private let sampleQueue = DispatchQueue(label: Bundle.main.bundleIdentifier! + ".SampleQueue") self.stream = SCStream(filter: filter, configuration: self.streamConfig, delegate: self) do { try self.stream?.addStreamOutput(self, type: .screen, sampleHandlerQueue: self.sampleQueue) } We have the whole handlers and what not, pretty much verbatim from the apple provided sample

I am using ScreenCaptureKit to create a screenshot software, but I found that the screenshot captured by the new API, SCScreenshotManager.captureImage, is very blurry. This is my screenshot. It is so blurry. But I hope it's like this. My code is as follows. func captureScreen(windows: [SCWindow], display: SCDisplay) async throws -> CGImage? { let availableWindows = windows.filter { window in Bundle.main.bundleIdentifier != window.owningApplication?.bundleIdentifier } let filter = SCContentFilter(display: display, including: availableWindows) if #available(macOS 14.0, *) { let image = try? await SCScreenshotManager.captureImage( contentFilter: filter, configuration: SCStreamConfiguration.defaultConfig( width: display.width, height: display.height ) ) return image } else { return nil } } extension SCStreamConfiguration { static func defaultConfig(width: Int, height: Int) -> SCStreamConfiguration { let config = SCStreamConfiguration() config.width = width config.height = height config.showsCursor = false if #available(macOS 14.0, *) { config.captureResolution = .best } return config } }