Post not yet marked as solved
was curious if hardware accelerated encoding was only h264 supported?
Post not yet marked as solved
0 VideoToolbox ___vtDecompressionSessionRemote_DecodeFrameCommon_block_invoke_2()
1 libdispatch.dylib __dispatch_client_callout()
2 libdispatch.dylib __dispatch_client_callout()
3 libdispatch.dylib __dispatch_lane_barrier_sync_invoke_and_complete()
4 VideoToolbox _vtDecompressionSessionRemote_DecodeFrameCommon()
After the release of iOS 17 & iOS 16.7, thousands of crashes were added as videotoolbox each day. Has anyone encountered similar problems?
Post not yet marked as solved
Hello,
I'm trying to play an H.265 video stream, containing no I-frames but using GDR.
No video is shown using VTDecompressionSessionDecodeFrame, other H.265 videos work fine.
In the device log I can see the following errors:
AppleAVD: VADecodeFrame(): isRandomAccessSkipPicture fail
followed by
An unknown error occurred (-17694)
Is it possible to play such a video stream?
Do I need to configure CMVideoFormatDescription in some way to enable it?
Thanks for your help.
I use VTCompressionSession and set the average bit rate by kVTCompressionPropertyKey_AverageBitRate and kVTCompressionPropertyKey_DataRateLimits. The code like this:
VTCompressionSessionRef vtSession = session;
if (vtSession == NULL) {
vtSession = _encoderSession;
}
if (vtSession == nil) {
return;
}
int tmp = bitrate;
int bytesTmp = tmp * 0.15;
int durationTmp = 1;
CFNumberRef bitrateRef = CFNumberCreate(NULL, kCFNumberSInt32Type, &tmp);
CFNumberRef bytes = CFNumberCreate(NULL, kCFNumberSInt32Type, &bytesTmp);
CFNumberRef duration = CFNumberCreate(NULL, kCFNumberSInt32Type, &durationTmp);
if ([self isSupportPropertyWithSession:vtSession key:kVTCompressionPropertyKey_AverageBitRate]) {
[self setSessionPropertyWithSession:vtSession key:kVTCompressionPropertyKey_AverageBitRate value:bitrateRef];
}else {
NSLog(@"Video Encoder: set average bitRate error");
}
NSLog(@"Video Encoder: set bitrate bytes = %d, _bitrate = %d",bytesTmp, bitrate);
CFMutableArrayRef limit = CFArrayCreateMutable(NULL, 2, &kCFTypeArrayCallBacks);
CFArrayAppendValue(limit, bytes);
CFArrayAppendValue(limit, duration);
if([self isSupportPropertyWithSession:vtSession key:kVTCompressionPropertyKey_DataRateLimits]) {
OSStatus ret = VTSessionSetProperty(vtSession, kVTCompressionPropertyKey_DataRateLimits, limit);
if(ret != noErr){
NSError *error = [NSError errorWithDomain:NSOSStatusErrorDomain code:ret userInfo:nil];
NSLog(@"Video Encoder: set DataRateLimits failed with %s",error.description.UTF8String);
}
}else {
NSLog(@"Video Encoder: set data rate limits error");
}
CFRelease(bitrateRef);
CFRelease(limit);
CFRelease(bytes);
CFRelease(duration);
}
This work fine on iOS16 and below. But on iOS17 the bitrate of the generate video file is much lower than the value I set.
For exmaple, I set biterate 600k but on iOS17 the encoded video bitrate is lower than 150k. What went wrong?
Post not yet marked as solved
There is the crash on iOS17 beta7/beta8/RC/public.
Crash Stack
Post not yet marked as solved
Use iOS17 VTCompressionSessionEncodeFrame beta5 version of video coding, if App switch back to the foreground from the background, Can appear kVTInvalidSessionErr reset according to the error invoked when VTCompressionSessionInvalidate cause gets stuck, if anyone else encountered this kind of situation, how should solve?
Post not yet marked as solved
When I doing hardware videotoolbox encoding, it always failed in some mac machines. But it works in some other mac machines. I can not find the different of the machines.
In problem machine, when I tried use ffmpeg executable do h265/h264 hardware encoding directly, it has issue. tried use ffmpeg executable h265/h264 software encoding, it has no issue.
The error is like this:
./ffmpeg -f rawvideo -s 352x288 -pix_fmt yuv420p -i akiyo_cif.yuv -c:v hevc_videotoolbox test.mp4
[rawvideo @ 0x7f80b9704780] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from 'akiyo_cif.yuv':
Duration: 00:00:12.00, start: 0.000000, bitrate: 30412 kb/s
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 352x288, 30412 kb/s, 25 tbr, 25 tbn
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (hevc_videotoolbox))
Press [q] to stop, [?] for help
[hevc_videotoolbox @ 0x7f80ba905640] Error encoding frame: -12905
[hevc_videotoolbox @ 0x7f80ba905640] popping: -542398533
[vost#0:0/hevc_videotoolbox @ 0x7f80ba9052c0] Error initializing output stream: Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
Did more tests in problem machine and Check the processes in mac.
The first launched VTEncoderXPCService process always has something wrong. The later launched VTEncoderXPCService can work as expected
Hi Apple Expert, Do you know what's the reason is? Thanks.
Post not yet marked as solved
To my surprise, there is nothing information about VTPixelRotationSession on the Internet. I tried Baidu and Bing, and I got nothing about it.
Here is my code:
var rotationSession: VTPixelRotationSession!
override func viewDidLoad() {
let state = VTPixelRotationSessionCreate(kCFAllocatorDefault, &rotationSession)
if state != 0 {
return
}
VTSessionSetProperty(rotationSession, key: kVTPixelRotationPropertyKey_Rotation, value: kVTRotation_CW90)
}
func rotate(sourceBuffer: CVImageBuffer) {
let pixelFormat = CVPixelBufferGetPixelFormatType(sourceBuffer)
var rotatedImage: CVPixelBuffer?
CVPixelBufferCreate(kCFAllocatorDefault, 640, 480, pixelFormat, nil, &rotatedImage)
let state = VTPixelRotationSessionRotateImage(rotationSession, sourceBuffer, rotatedImage)
if state == kCVReturnSuccess {
......
}
}
The rotatedImageBuffer does not get any pixels of the sourceImageBuffer after calling VTPixelRotationSessionRotateImage.
Anyone can explain this issue?
Apple official does not give any examples about it.
Post not yet marked as solved
Hi,
I'm using VideoToolBox for h264 or HEVC video encoding.
Thanks proper algorithm I used to change dynamically the video bitrate (by adjusting AverageBitrate and DataRateLimits)
When running application on iOS 16.4 and further, video bitrate is no more respected by video encoder. Obtained bitrate is generally 70% percent lower than expected value.
Is there a new setting to retrieve previous video encoder behaviour?
Best regards
Hello,
I am attempting to simultaneously stream video to a remote client and run inference on a neural network (on the same video frame) locally. I have done this in other platforms, using Gstreamer on linux and on android using libstreaming for compression and packetization. I've attempted this now on iPhone using ffmpeg to stream and a capture session to feed the neural network but I run into the problem of multiple camera access.
Most of the posts I see are concerned with receiving RTP streams in iOS but I need to do the opposite. As I am new to iOS and Swift I was hoping someone could provide method for RTP packetization? Any library recommendations or example code for something similar?
Best,
sandesh64.