Core Media

RSS for tag

Efficiently process media samples and manage queues of media data using Core Media.

Core Media Documentation

Posts under Core Media tag

25 Posts
Sort by:
Post marked as Apple Recommended
6.5k Views
I built an app which hosts a CMIOExtension. The app works, and it can activate the extension. The extension loads in e.g. Photo Booth and shows the expected video (a white horizontal line which moves down the picture). I have a couple of questions about this though. The sample Camera Extension is built with a CMIOExtension dictionary with just one entry, CMIOExtensionMachServiceName which is $(TeamIdentifierPrefix)$(PRODUCT_BUNDLE_IDENTIFIER) This Mach service name won't work though. When attempting to activate the extension, sysextd says that the extensions has an invalid mach service name or is not signed, the value must be prefixed with one of the App Groups in the entitlement. So in order to get the sample extension to activate from my app, I have to change its CMIOExtensionMachServiceName to <my team ID>.com.mycompany.my-app-group.<myextensionname> Is this to be expected? The template CMIOExtension generates its own video using a timer. My app is intended to capture video from a source, filter that video, then feed it to the CMIOExtension, somehow. The template creates an app group called "$(TeamIdentifierPrefix)com.example.app-group", which suggests that it might be possible to use XPC to send frames from the app to the extension. However, I've been unable to do so. I've used NSXPCConnection * connection = [[NSXPCConnection alloc] initWithMachServiceName:, using the CMIOExtensionMachServiceName with no options and with the NSXPCConnectionPrivileged option. I've tried NSXPCConnection * connection = [[NSXPCConnection alloc] initWithServiceName: using the extension's bundle identifier. In all cases when I send the first message I get an error in the remote object proxy's handler: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named <whatever name I try> was invalidated: failed at lookup with error 3 - No such process." According to the "Daemons and Services Programming Guide" an XPC service should have a CFBundlePackageType of XPC!, but a CMIOExtension is of type SYSX. It can't be both. Does the CMIOExtension loading apparatus cook up a synthetic name for the XPC service, and if so, what is it? If none, how is one expected to get pixel buffers into the camera extension?
Posted
by
Post not yet marked as solved
7 Replies
1.9k Views
Hi! I'm trying to move from CoreMedio I/O DAL Plug-In to CoreMedia I/O camera extensions, announced in macOS 12.3. I created a test extension, placed it inside my app bundle into Contents/Library/SystemExtensions and signed with codesigning certificate. But when I try to install my extension from inside my app, using this code (Swift): func requestActivation() { guard case .idle = status else { fatalError("Invalid state") } print("Requesting activation of extension \"\(extensionIdentifier)\"") let req = OSSystemExtensionRequest.activationRequest(forExtensionWithIdentifier: extensionIdentifier, queue: DispatchQueue.main) req.delegate = self OSSystemExtensionManager.shared.submitRequest(req) status = .requested } I'm getting an error: OSSystemExtensionErrorDomain error 8: Code Signature Invalid which is rather generic. Can anybody tell me what I am doing wrong? Or at least propose some steps to find it out? I'm posting here entitlements and codesign output for my extension and containing application for further information. kdg@admins-Mac-mini SystemExtensions % codesign -d --entitlements - ./com.visicom.VirtualCamera.avextension.systemextension Executable=/Applications/VirtualCamera.app/Contents/Library/SystemExtensions/com.visicom.VirtualCamera.avextension.systemextension/Contents/MacOS/com.visicom.VirtualCamera.avextension [Dict] [Key] com.apple.security.app-sandbox [Value] [Bool] true [Key] com.apple.security.application-groups [Value] [Array] [String] 6SUWV7QQBJ.com.visicom.VirtualCamera kdg@admins-Mac-mini /Applications % codesign -d --entitlements - ./VirtualCamera.app Executable=/Applications/VirtualCamera.app/Contents/MacOS/VirtualCamera [Dict] [Key] com.apple.developer.system-extension.install [Value] [Bool] true [Key] com.apple.security.app-sandbox [Value] [Bool] true [Key] com.apple.security.application-groups [Value] [Array] [String] 6SUWV7QQBJ.com.visicom.VirtualCamera [Key] com.apple.security.files.user-selected.read-only [Value] [Bool] true kdg@admins-Mac-mini SystemExtensions % codesign -dvvv ./com.visicom.VirtualCamera.avextension.systemextension Executable=/Applications/VirtualCamera.app/Contents/Library/SystemExtensions/com.visicom.VirtualCamera.avextension.systemextension/Contents/MacOS/com.visicom.VirtualCamera.avextension Identifier=com.visicom.VirtualCamera.avextension Format=bundle with Mach-O universal (x86_64 arm64) CodeDirectory v=20500 size=1553 flags=0x10700(hard,kill,expires,runtime) hashes=37+7 location=embedded Hash type=sha256 size=32 CandidateCDHash sha256=25bd80657bfd6e0ab95467146c7b532817e9e520 CandidateCDHashFull sha256=25bd80657bfd6e0ab95467146c7b532817e9e5209fd50b0cb7ceef40dcfb40e8 Hash choices=sha256 CMSDigest=25bd80657bfd6e0ab95467146c7b532817e9e5209fd50b0cb7ceef40dcfb40e8 CMSDigestType=2 CDHash=25bd80657bfd6e0ab95467146c7b532817e9e520 Signature size=9006 Authority=Developer ID Application: Visicom Media Inc. (6SUWV7QQBJ) Authority=Developer ID Certification Authority Authority=Apple Root CA Timestamp=7 Jul 2022, 21:49:32 Info.plist entries=23 TeamIdentifier=6SUWV7QQBJ Runtime Version=12.3.0 Sealed Resources version=2 rules=13 files=0 Internal requirements count=1 size=200 kdg@admins-Mac-mini /Applications % codesign -dvvv ./VirtualCamera.app Executable=/Applications/VirtualCamera.app/Contents/MacOS/VirtualCamera Identifier=com.visicom.VirtualCamera Format=app bundle with Mach-O universal (x86_64 arm64) CodeDirectory v=20500 size=1989 flags=0x10700(hard,kill,expires,runtime) hashes=51+7 location=embedded Hash type=sha256 size=32 CandidateCDHash sha256=31e15fbbd436a67a20c5b58c597d8a4796a67720 CandidateCDHashFull sha256=31e15fbbd436a67a20c5b58c597d8a4796a6772020308fb69f4ee80b4e32788b Hash choices=sha256 CMSDigest=31e15fbbd436a67a20c5b58c597d8a4796a6772020308fb69f4ee80b4e32788b CMSDigestType=2 CDHash=31e15fbbd436a67a20c5b58c597d8a4796a67720 Signature size=9006 Authority=Developer ID Application: Visicom Media Inc. (6SUWV7QQBJ) Authority=Developer ID Certification Authority Authority=Apple Root CA Timestamp=7 Jul 2022, 21:58:09 Info.plist entries=20 TeamIdentifier=6SUWV7QQBJ Runtime Version=12.3.0 Sealed Resources version=2 rules=13 files=4 Internal requirements count=1 size=188 Thanks in advance!
Posted
by
Post not yet marked as solved
0 Replies
613 Views
My current app implements a custom video player, based on a AVSampleBufferRenderSynchronizer synchronising two renderers: an AVSampleBufferDisplayLayer receiving decoded CVPixelBuffer-based video CMSampleBuffers, and an AVSampleBufferAudioRenderer receiving decoded lpcm-based audio CMSampleBuffers. The AVSampleBufferRenderSynchronizer is started when the first image (in presentation order) is decoded and enqueued, using avSynchronizer.setRate(_ rate: Float, time: CMTime), with rate = 1 and time the presentation timestamp of the first decoded image. Presentation timestamps of video and audio sample buffers are consistent, and on most streams, the audio and video are correctly synchronized. However on some network streams, on iOS, the audio and video aren't synchronized, with a time difference that seems to increase with time. On the other hand, with the same player code and network streams on macOS, the synchronization always works fine. This reminds me of something I've read, about cases where an AVSampleBufferRenderSynchronizer could not synchronize audio and video, causing them to run with independent and potentially drifting clocks, but I cannot find it again. So, any help / hints on this sync problem will be greatly appreciated! :)
Posted
by
Post not yet marked as solved
1 Replies
1k Views
Hi , We are getting big spikes of errors on very few programs in some live channels. [-16012:CoreMediaErrorDomain] [Error Domain=CoreMediaErrorDomain Code=-16012 "(null)"] When this error occurred , AVPlayer stops & users has to restart the playback. This is the error we are getting & this happens in some live programs. We have the same set up & uses same transcoders etc in all programs. Mostly we have very low error rate in player with live programs , but with this error , error rate can increase up to 80% of the users effecting pretty much all the users on apple devices. Does anyone know what this error actually means ? What is the context & what is the reason behind that ? It seems like this may be related subtitles & this occurs only when the subtitles are enabled. ( The subtitles are not embedded in the stream it is teletext ) Try to find in apple documents & online & nothing could be find unfortunately.
Posted
by
Post marked as solved
3 Replies
783 Views
I am currently writing a software product which involves a Camera Extension and a Cocoa application. I would like to share some files between the two components and as of my understanding this should be quite straightforward by putting both applications into the same App Group and then accessing the particular Group Container. However doing so, does result in both components accessing different locations for the Group Container. I am using the following piece of code to create a new folder inside the container: let directory = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: "group.235ADAK9D5.com.creativetoday.camskool")! let newDirectory = directory.appendingPathComponent("Mydir") try? FileManager.default.createDirectory(at: newDirectory, withIntermediateDirectories: false) If I run this I find that the Cocoa application is going to access the following Location and create the file there: /Users//Library/Group Containers//" Where as the Camera Extension will access the following Location and create the directory there: /private/var/db/cmiodalassistants/Library/Group Containers// If I create a file in one directory it does not appear in the other. I tried for both components to access the opposite directory but it results in an permission denied message. What am I doing wrong?
Posted
by
Post not yet marked as solved
1 Replies
561 Views
Im trying to load an image in a coremediaIO extension. Ive successfully loaded the image and draw it in 3 different ways. Once from the AppGroup, once from the extension bundle and another time as a base64 string. No problems each time the image loads and renders. However, when I call the image to be drawn in a pixelbuffer it loads the extension into the dock. Id love to be able to draw an image and the extension not appear in the dock but I cannot figure this out. Can render text in this timer loop no problem. Any suggestions other than sending the image across the sink stream? Essentially i want to render the app logo whenever the host app is not sending signal to the extension. func startStreaming() { guard let _ = _bufferPool else { return } _streamingCounter += 1 _timer = DispatchSource.makeTimerSource(flags: .strict, queue: _timerQueue) _timer!.schedule(deadline: .now(), repeating: 1.0/Double(kFrameRate), leeway: .seconds(0)) _timer!.setEventHandler { if self.sinkStarted { return } var err: OSStatus = 0 var pixelBuffer: CVPixelBuffer? err = CVPixelBufferPoolCreatePixelBufferWithAuxAttributes(kCFAllocatorDefault, self._bufferPool, self._bufferAuxAttributes, &pixelBuffer) if err != 0 { logger.debug("out of pixel buffers \(err)") } if let pixelBuffer = pixelBuffer { CVPixelBufferLockBaseAddress(pixelBuffer, []) let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer) let width = CVPixelBufferGetWidth(pixelBuffer) let height = CVPixelBufferGetHeight(pixelBuffer) let rgbColorSpace = CGColorSpaceCreateDeviceRGB() if let context = CGContext(data: pixelData, width: width, height: height, bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer), space: rgbColorSpace, bitmapInfo: CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.premultipliedFirst.rawValue) { let graphicsContext = NSGraphicsContext(cgContext: context, flipped: false) NSGraphicsContext.saveGraphicsState() NSGraphicsContext.current = graphicsContext let cgContext = graphicsContext.cgContext let dstRect = CGRect(x: 0, y: 0, width: width, height: height) cgContext.clear(dstRect) cgContext.setFillColor(NSColor.black.cgColor) cgContext.fill(dstRect) let imageWidth = 400 // You can adjust the width as needed let imageHeight = 400 // You can adjust the height as needed let imageOrigin = CGPoint(x: (width - imageWidth) / 2, y: (height - imageHeight) / 2) // Center the image if let decodedData = Data(base64Encoded: imageBaseString, options: .ignoreUnknownCharacters), let image = NSImage(data: decodedData), let cgImage = image.cgImage(forProposedRect: nil, context: nil, hints: nil) { cgContext.draw(cgImage, in: CGRect(origin: imageOrigin, size: NSSize(width: imageWidth, height: imageHeight))) } NSGraphicsContext.restoreGraphicsState() } CVPixelBufferUnlockBaseAddress(pixelBuffer, []) } if let pixelBuffer = pixelBuffer { var sbuf: CMSampleBuffer! var timingInfo = CMSampleTimingInfo() timingInfo.presentationTimeStamp = CMClockGetTime(CMClockGetHostTimeClock()) err = CMSampleBufferCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: pixelBuffer, dataReady: true, makeDataReadyCallback: nil, refcon: nil, formatDescription: self._videoDescription, sampleTiming: &timingInfo, sampleBufferOut: &sbuf) if err == 0 { self._streamSource.stream.send(sbuf, discontinuity: [], hostTimeInNanoseconds: UInt64(timingInfo.presentationTimeStamp.seconds * Double(NSEC_PER_SEC))) } } } _timer!.setCancelHandler {} _timer!.resume() } Culprit is here: if let decodedData = Data(base64Encoded: imageBaseString, options: .ignoreUnknownCharacters), let image = NSImage(data: decodedData), let cgImage = image.cgImage(forProposedRect: nil, context: nil, hints: nil) { cgContext.draw(cgImage, in: CGRect(origin: imageOrigin, size: NSSize(width: imageWidth, height: imageHeight))) } Doesnt matter how I load the image, the dock extension will trigger as soon as the draw loop is triggered or on extension init if I load the image then. Is this possible?
Posted
by
Post not yet marked as solved
2 Replies
948 Views
Our app gets rejected for referencing non-public symbol _CMTimebaseCreateWithMasterClock. We have confirmed that we are using the suggested CMTimebaseCreateWithSourceClock. When looking at our DerivedData, the compiler turns CMTimebaseCreateWithSourceClock into CMTimebaseCreateWithMasterClock presumably because CMTimebaseCreateWithSourceClock is an inline function. We have verified if we remove the one call to CMTimebaseCreateWithSourceClock that the CMTimebaseCreateWithSourceClock symbol will no longer appear in our DerivedData. This is preventing us from submitting an app update. Please advise asap.
Posted
by
Post not yet marked as solved
0 Replies
629 Views
How does one get the list of controls which a CMIOObject has to offer? How do the objects in the CMIO hierarchy map to CMIOExtension objects? I expected the hierarchy to be something like this: the system has owned objects of type: 'aplg' `(kCMIOPlugInClassID)` has owned objects of type 'adev' `(kCMIODeviceClassID,` which may have owned objects of type 'actl' `(kCMIOControlClassID)` and has at least one owned object of type 'astr' `(kCMIOStreamClassID),` each of which may have owned objects of type 'actl' `(kCMIOControlClassID)` Instead, when I recursively traverse the object hierarchy, I find the devices and the plug-ins at the same level (under the system object). Only some of the device in my system have owned streams, although they all have a kCMIODevicePropertyStreams ('stm#') property. None of the devices or streams appear to have any controls, and none of the streams have any owned objects. I'm not using the qualifier when searching for owned objects, because the documentation implies that it may be nil if I'm not interested in narrowing my search. Should I expect to find any devices or streams with controls? And if so, how do I get a list of them? CMIOHardwareObject.h says that "Wildcards... are especially useful ...for querying an CMIOObject's list of CMIOControls. ", but there's no example of how to do this. My own device (from my camera extension) has no owned objects of type stream. I don't see any API call to convey ownership of the stream I create by the device it belongs to. How does the OS decide that a stream is 'owned' by a device? I've tried various scopes and elements - kCMIOObjectPropertyScopeGlobal, kCMIOObjectPropertyScopeWildcard, kCMIOControlPropertyScope, and kCMIOObjectPropertyElementMain, kCMIOObjectPropertyElementWildcard and kCMIOControlPropertyElement. I can't get a list of controls using any of these. Ultimately, I'm trying to find my provider, my devices and my streams using the CMIO interface, so that I can set and query properties on them. Is it reasonable to assume that the CMIOObject of type 'aplg' is the one corresponding to a CMIOExtensionProviderSource? This is on Ventura 13.4.1 on M1.
Posted
by
Post not yet marked as solved
9 Replies
951 Views
Hi! I'm trying to move from CoreMedio I/O DAL Plug-In to CoreMedia I/O camera extensions, announced in macOS 12.3. I created a test extension, placed it inside my app bundle into Contents/Library/SystemExtensions and signed with codesigning certificate. But when I try to install my extension from inside my app, using this code (Swift): func installDriver() { guard let extensionIdentifer = DriverInstaller.extensionBundle().bundleIdentifier else { return } let activationReq = OSSystemExtensionRequest.activationRequest(forExtensionWithIdentifier: extensionIdentifer, queue: .main) activationReq.delegate = self OSSystemExtensionManager.shared.submitRequest(activationReq) } I'm getting an error: OSSystemExtensionErrorDomain error 8: Code Signature Invalid which is rather generic. Can anybody tell me what I am doing wrong? Or at least propose some steps to find it out? I'm posting here entitlements and codesign output for my extension and containing application for further information. Executable=../Contents/Library/SystemExtensions/com..RoomDevice.Extension.systemextension/Contents/MacOS/com..RoomDevice.Extension [Dict] [Key] com.apple.security.app-sandbox [Value] [Bool] true [Key] com.apple.security.application-groups [Value] [Array] [String] 893K7MTL2H. com.. [Key] com.apple.security.device.camera [Value] [Bool] true Executable=**********/Contents/MacOS/***** [Dict] [Key] com.apple.application-identifier [Value] [String] 893K7MTL2H.com..RoomDevice [Key] com.apple.developer.system-extension.install [Value] [Bool] true [Key] com.apple.developer.team-identifier [Value] [String] 893K7MTL2H [Key] com.apple.security.application-groups [Value] [Array] [String] 893K7MTL2H. com..******** Executable=***/Contents/MacOS/**** Identifier=com..RoomDevice Format=app bundle with Mach-O universal (x86_64 arm64) CodeDirectory v=20500 size=1345 flags=0x10000(runtime) hashes=31+7 location=embedded Hash type=sha256 size=32 CandidateCDHash sha256=3584714367d59119b462d0f830247d27ff1fbace CandidateCDHashFull sha256=3584714367d59119b462d0f830247d27ff1fbace53419d69abaa658fbb7a4f12 Hash choices=sha256 CMSDigest=3584714367d59119b462d0f830247d27ff1fbace53419d69abaa658fbb7a4f12 CMSDigestType=2 Launch Constraints: None CDHash=3584714367d59119b462d0f830247d27ff1fbace Signature size=4688 Authority=Developer ID Application: ****************(893K7MTL2H) Authority=Developer ID Certification Authority Authority=Apple Root CA Signed Time=01-Sep-2023 at 12:00:09 PM Info.plist entries=22 TeamIdentifier=893K7MTL2H Runtime Version=13.3.0 Sealed Resources version=2 rules=13 files=6 Internal requirements count=1 size=216 Executable=/Contents/Library/SystemExtensions/com.*****.RoomDevice.Extension.systemextension/Contents/MacOS/com..RoomDevice.Extension Identifier=com.******.RoomDevice.Extension Format=bundle with Mach-O universal (x86_64 arm64) CodeDirectory v=20500 size=3627 flags=0x10000(runtime) hashes=102+7 location=embedded Hash type=sha256 size=32 CandidateCDHash sha256=70580825016b7e262fb15c280ba380ad4e871bc1 CandidateCDHashFull sha256=70580825016b7e262fb15c280ba380ad4e871bc108951adb8cd474d652567f4f Hash choices=sha256 CMSDigest=70580825016b7e262fb15c280ba380ad4e871bc108951adb8cd474d652567f4f CMSDigestType=2 Launch Constraints: None CDHash=70580825016b7e262fb15c280ba380ad4e871bc1 Signature size=4688 Authority=Developer ID Application: ************ Ltd. (893K7MTL2H) Authority=Developer ID Certification Authority Authority=Apple Root CA Signed Time=01-Sep-2023 at 12:00:05 PM Info.plist entries=22 TeamIdentifier=893K7MTL2H Runtime Version=13.3.0 Sealed Resources version=2 rules=13 files=0 Internal requirements count=1 size=224 Please anyone help. Thanks in advance!
Posted
by
Post not yet marked as solved
0 Replies
609 Views
In our application, we play video-on-demand (VOD) content and display subtitles in different languages. The format we prefer for subtitles is WebVTT. We are planning to enhance caption styling (text color, background color, font weight, etc.) in WebVTT files. In our current flow, subtitles and images are loaded in 6-second chunks. Below is an example of one of the subtitle parts we use: WEBVTT X-TIMESTAMP-MAP=MPEGTS:0,LOCAL:00:00:00.000
Posted
by
Post not yet marked as solved
2 Replies
508 Views
It seems AVAssetWriter is rejecting CVPixelBuffers with error -12743 when appending NSData for kCVImageBufferAmbientViewingEnvironmentKey for HDR videos. Here is my code: var ambientViewingEnvironment:CMFormatDescription.Extensions.Value? var ambientViewingEnvironmentData:NSData? ambientViewingEnvironment = sampleBuffer.formatDescription?.extensions[.ambientViewingEnvironment] let plist = ambientViewingEnvironment?.propertyListRepresentation ambientViewingEnvironmentData = plist as? NSData And then attaching this data, CVBufferSetAttachment(dstPixelBuffer, kCVImageBufferAmbientViewingEnvironmentKey, ambientViewingEnvironmentData! as CFData, .shouldPropagate) No matter what I do, including copying the attachment from sourcePixelBuffer to destinationPixelBuffer as it is, the error remains! var attachmentMode:CVAttachmentMode = .shouldPropagate let attachment = CVBufferCopyAttachment(sourcePixelBuffer!, kCVImageBufferAmbientViewingEnvironmentKey, &attachmentMode) NSLog("Attachment \(attachment!), mode \(attachmentMode)") CVBufferSetAttachment(dstPixelBuffer, kCVImageBufferAmbientViewingEnvironmentKey, attachment!, attachmentMode) I need to know if there is anything wrong in the way metadata is copied.
Posted
by
Post not yet marked as solved
5 Replies
2.5k Views
Recently I've been trying to play some AV1-encoded streams on my iPhone 15 Pro Max. First, I check for hardware support: VTIsHardwareDecodeSupported(kCMVideoCodecType_AV1); // YES Then I need to create a CMFormatDescription in order to pass it into a VTDecompressionSession. I've tried the following: { mediaType:'vide' mediaSubType:'av01' mediaSpecific: { codecType: 'av01' dimensions: 394 x 852 } extensions: {{ CVFieldCount = 1; CVImageBufferChromaLocationBottomField = Left; CVImageBufferChromaLocationTopField = Left; CVPixelAspectRatio = { HorizontalSpacing = 1; VerticalSpacing = 1; }; FullRangeVideo = 0; }} } but VTDecompressionSessionCreate gives me error -8971 (codecExtensionNotFoundErr, I assume). So it has something to do with the extensions dictionary? I can't find anywhere which set of extensions is necessary for it to work 😿. VideoToolbox has convenient functions for creating descriptions of AVC and HEVC streams (CMVideoFormatDescriptionCreateFromH264ParameterSets and CMVideoFormatDescriptionCreateFromHEVCParameterSets), but not for AV1. As of today I am using XCode 15.0 with iOS 17.0.0 SDK.
Posted
by
Post not yet marked as solved
0 Replies
379 Views
I'm trying to show likeCommand and dislikeCommand on the Lock Screen of a music player without success. Are they still supported ? Is there any special configuration on the player or the track for them to show? My current code which works for playCommand looks like this MPRemoteCommandCenter.shared().likeCommand.addTarget { [unowned self] _ in if isPlaying { return .success } return .commandFailed }
Posted
by
Post not yet marked as solved
0 Replies
421 Views
I have a use case to rotate a CMSampleBuffer from landscape to portrait. I have written a rough code for it. But still am facing many issues with appending of the sampleBuffer to input frame. Here's the code: `guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return nil } // Get the dimensions of the image buffer let width = CVPixelBufferGetWidth(imageBuffer) let height = CVPixelBufferGetHeight(imageBuffer) // Determine if the image needs to be rotated let shouldRotate = width > height // Create a CIImage from the buffer var image = CIImage(cvImageBuffer: imageBuffer) // Rotate the CIImage if necessary if shouldRotate { image = image.oriented(forExifOrientation: 6) // Rotate 90 degrees clockwise } let originalPixelFormatType = CVPixelBufferGetPixelFormatType(imageBuffer) // Create a new pixel buffer var newPixelBuffer: CVPixelBuffer? let status = CVPixelBufferCreate(kCFAllocatorDefault, height, width, originalPixelFormatType, nil, &newPixelBuffer) guard status == kCVReturnSuccess, let pixelBuffer = newPixelBuffer else { return nil } CVBufferPropagateAttachments(imageBuffer, newPixelBuffer!) // Render the rotated image onto the new pixel buffer let context = CIContext() context.render(image, to: pixelBuffer) CVPixelBufferUnlockBaseAddress(pixelBuffer,CVPixelBufferLockFlags(rawValue: 0)) var videoInfo: CMVideoFormatDescription? CMVideoFormatDescriptionCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: newPixelBuffer!, formatDescriptionOut: &videoInfo) var sampleTimingInfo = CMSampleTimingInfo(duration: CMSampleBufferGetDuration(sampleBuffer), presentationTimeStamp: CMSampleBufferGetPresentationTimeStamp(sampleBuffer), decodeTimeStamp: CMSampleBufferGetDecodeTimeStamp(sampleBuffer)) var newSampleBuffer: CMSampleBuffer? CMSampleBufferCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: newPixelBuffer!, dataReady: true, makeDataReadyCallback: nil, refcon: nil, formatDescription: videoInfo!, sampleTiming: &sampleTimingInfo, sampleBufferOut: &newSampleBuffer) let attachments: CFArray! = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, createIfNecessary: true) let dictionary = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self) if let attachmentsArray = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, createIfNecessary: true) as? [CFDictionary] { for attachment in attachmentsArray { for (key, value) in attachment as! Dictionary<CFString, Any> { if let value = value as? CFTypeRef { CMSetAttachment(newSampleBuffer!, key: key, value: value, attachmentMode: kCMAttachmentMode_ShouldPropagate) } } } } return newSampleBuffer! The error that I am getting while appending the frame is: Error occured, isVideo = false, status = 3, Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12780), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x282b87390 {Error Domain=NSOSStatusErrorDomain Code=-12780 "(null)"}} I read online that this error might be due to different PixelFormatType. But how can that be because I am obtaining the PixelFormatType from the buffer itself. If you want to see the difference between original and rotated sample buffer. https://www.diffchecker.com/V0a55kCB/ Thanks in advance!
Posted
by
Post marked as solved
2 Replies
365 Views
Hey folks, I have an application that ships a CoreMedia I/O system extension to create a virtual camera. We separately ship an "uninstaller" app, which is a notarised AppKit app. This uninstaller removes the app, containers, and the system extension via the following API: let request = OSSystemExtensionRequest.deactivationRequest(forExtensionWithIdentifier: pluginIdentifier, queue: .main) request.delegate = self OSSystemExtensionManager.shared.submitRequest(request) The OSSystemExtensionRequest API does correctly deliver metadata via propertiesRequest(forExtensionWithIdentifier: …), but when we attempt to remove the extension using the above snippet, we get a failure OSSystemExtensionError.extensionNotFound. The uninstaller app is signed with the same entitlements and certificate as the host app. It also embeds a copy of the system extension as required by the API. I think the crux of the issue is: Should this be expected to work? We're all code-signed correctly etc, and the only difference is that the removal request is coming from an app with a different bundle identifier to the one that installed it start with. Thanks!
Posted
by
Post not yet marked as solved
2 Replies
546 Views
I made s target of "Camera Extension" on Xcode macOS Swift app. I got Swift code with CMIOExtensionDeviceSource. I add NSLog() and String.write() to file under FileManager.default.temporaryDirectory. My camera extension installaion was success and running with FaceTime. But I cannot see NSLog output or debug output temp file on Xcode or Console. How can I see debug output from my Camera Extension?
Posted
by
Post not yet marked as solved
2 Replies
462 Views
I got code of CMIO CameraExtension by Xcode target and it is running with FaceTime. I guess this kind of Extension has lots of security limitation. I like to run command like "netstat" in Extension. Is that possible to call Process.run()? I got keep getting error like "The file zsh doesn’t exist". Same code with Process.run() worked in macOS app. I like to run DistributedNotificationCenter and send text from App to CameraExtension. Is that possible? I do not receive any message on CameraExtension. If there is any other IPC method between macOS app and CameraExtension, please let me know.
Posted
by