Access to the iOS' video decoder?

user_168225 0
Viewed 14285

The iPad/iOS has video streaming support for e.g. H.264 using MPMoviePlayerController etc., but i receive H.264 data through a custom, proprietary, stream and need to decode it in a soft real-time scenario.
Can the iPads/iOS' video decoder be accessed in any way to decode this data?

Update: Apparently the iOS 4.0 Core Media Framework supports decoding frames and knows of H.264, but there is no sample code nor can i see what i actually am supposed to call for the actual decoding.

Update (ten years later!)

For anyone googling here, you do this in iOS these days with "VideoToolbox".

video-streaming ipad iphone
5 Answers

2019

There are two solutions

  1. Do it "by hand" which means using AVFoundation and in particular VideoToolbox.

To get going with that you basically start with https://developer.apple.com/videos/play/wwdc2014/513/ Enjoy!

I have to say, that is really the "correct and better" solution.

  1. If you can get ffmpeg-api working inside your iOS app, you can use ffmpeg, FFmpeg will do hardware decoding after some fiddling.

There are a number of ways to get started with that. (One absolutely amazing new thing is the SWIFT ffmpeg made by sunlubo: https://github.com/sunlubo/SwiftFFmpeg )

Be aware with the "ffmpeg" approach that there are, in short, a number of legal/license issues with ffmpeg / iOS. One can search and read about those problems.

However on the technical side, these days indeed it is possible to compile ffmpeg right in to iOS, and use it raw in your iOS code. (Using a C library may be easiest.)

We just did an enormous project doing just this, as well as other approaches. (I never want to see FFmpeg again!)

You can in fact achieve actual hardware decoding, in iOS, using FFmpeg.

We found it to be incredibly fiddly. And a couple of bugs need to be patched in FFmpeg. (I hope I never see videotoolbox.c again :/ )

So once again your two options for hardware decoding in iOS are

  1. Do it "by hand" AVFoundation/VideoToolbox.

  2. Use FFmpeg.

Item 2 is incredibly fiddly and uses a lot of time. Item 1 uses a huge amount of time. Tough choice :/

user_2948840
Related