The libavformat and libavcodec libraries that come with
ffmpeg
are a great way of accessing a large variety of video file formats.
Unfortunately, there is no real documentation on using these libraries in your
own programs (at least I couldn't find any), and the example programs aren't
really very helpful either.
This situation meant that, when I used libavformat/libavcodec on a recent
project, it took quite a lot of experimentation to find out how to use them.
Here's what I learned - hopefully I'll be able to save others from having to
go through the same trial-and-error process. There's also a small
demo program
that you can download. The code
I'll present works with libavformat/libavcodec as included in version 0.4.8 of
ffmpeg (the most recent version as I'm writing this). If you find that later
versions break the code, please let me know.
In this document, I'll only cover how to read video streams from a file;
audio streams work pretty much the same way, but I haven't actually used them,
so I can't present any example code.
In case you're wondering why there are two libraries, libavformat and
libavcodec: Many video file formats (AVI being a prime example) don't actually
specify which codec(s) should be used to encode audio and video data; they
merely define how an audio and a video stream (or, potentially, several
audio/video streams) should be combined into a single file. This is why
sometimes, when you open an AVI file, you get only sound, but no picture -
because the right video codec isn't installed on your system. Thus,
libavformat deals with parsing video files and separating the streams
contained in them, and libavcodec deals with decoding raw audio and video
streams.
Opening a Video File
First things first - let's look at how to open a video file and get at the
streams contained in it. The first thing we need to do is to initialize
libavformat/libavcodec:
av_register_all();
This registers all available file formats and codecs with the library so they
will be used automatically when a file with the corresponding format/codec is
opened. Note that you only need to call av_register_all()
once, so
it's probably best to do this somewhere in your startup code. If you like,
it's possible to register only certain individual file formats and codecs, but
there's usually no reason why you would have to do that.
Next off, opening the file:
AVFormatContext *pFormatCtx;
const char *filename="myvideo.mpg";
// Open video file
if(av_open_input_file(&pFormatCtx, filename, NULL, 0, NULL)!=0)
handle_error(); // Couldn't open file
The last three parameters specify the file format, buffer size and format
parameters; by simply specifying NULL or 0 we ask libavformat to auto-detect
the format and use a default buffer size. Replace handle_error()
with
appropriate error handling code for your application.
Next, we need to retrieve information about the streams contained in the file:
// Retrieve stream information
if(av_find_stream_info(pFormatCtx)<0)
handle_error(); // Couldn't find stream information
This fills the streams
field of the AVFormatContext
with
valid information. As a debugging aid, we'll dump this information onto
standard error, but of course you don't have to do this in a production
application:
dump_format(pFormatCtx, 0, filename, false);
As mentioned in the introduction, we'll handle only video streams, not audio
streams. To make things nice and easy, we simply use the first video stream we
find:
int i, videoStream;
AVCodecContext *pCodecCtx;
// Find the first video stream
videoStream=-1;
for(i=0; i<pFormatCtx->nb_streams; i++)
if(pFormatCtx->streams[i]->codec.codec_type==CODEC_TYPE_VIDEO)
{
videoStream=i;
break;
}
if(videoStream==-1)
handle_error(); // Didn't find a video stream
// Get a pointer to the codec context for the video stream
pCodecCtx=&pFormatCtx->streams[videoStream]->codec;
OK, so now we've got a pointer to the so-called codec context for our video
stream, but we still have to find the actual codec and open it:
AVCodec *pCodec;
// Find the decoder for the video stream
pCodec=avcodec_find_decoder(pCodecCtx->codec_id);
if(pCodec==NULL)
handle_error(); // Codec not found
// Inform the codec that we can handle truncated bitstreams -- i.e.,
// bitstreams where frame boundaries can fall in the middle of packets
if(pCodec->capabilities & CODEC_CAP_TRUNCATED)
pCodecCtx->flags|=CODEC_FLAG_TRUNCATED;
// Open codec
if(avcodec_open(pCodecCtx, pCodec)<0)
handle_error(); // Could not open codec
(So what's up with those "truncated bitstreams"? Well, as we'll see in a
moment, the data in a video stream is split up into packets. Since the amount
of data per video frame can vary, the boundary between two video frames need
not coincide with a packet boundary. Here, we're telling the codec that we can
handle this situation.)
One important piece of information that is stored in the
AVCodecContext
structure is the frame rate of the video. To allow for
non-integer frame rates (like NTSC's 29.97 fps), the rate is stored as a
fraction, with the numerator in pCodecCtx->frame_rate
and the
denominator in pCodecCtx->frame_rate_base
. While testing the library
with different video files, I noticed that some codecs (notably ASF) seem to
fill these fields incorrectly (frame_rate_base
contains 1 instead of
1000). The following hack fixes this:
// Hack to correct wrong frame rates that seem to be generated by some
// codecs
if(pCodecCtx->frame_rate>1000 && pCodecCtx->frame_rate_base==1)
pCodecCtx->frame_rate_base=1000;
Note that it shouldn't be a problem to leave this fix in place even if the bug
is corrected some day - it's unlikely that a video would have a frame rate of
more than 1000 fps.
One more thing left to do: Allocate a video frame to store the decoded images
in:
AVFrame *pFrame;
pFrame=avcodec_alloc_frame();
That's it! Now let's start decoding some video.
Decoding Video Frames
As I've already mentioned, a video file can contain several audio and video
streams, and each of those streams is split up into packets of a particular
size. Our job is to read these packets one by one using libavformat, filter
out all those that aren't part of the video stream we're interested in, and
hand them on to libavcodec for decoding. In doing this, we'll have to take
care of the fact that the boundary between two frames can occur in the middle
of a packet.
Sound complicated? Lucikly, we can encapsulate this whole process in a routine
that simply returns the next video frame:
bool GetNextFrame(AVFormatContext *pFormatCtx, AVCodecContext *pCodecCtx,
int videoStream, AVFrame *pFrame)
{
static AVPacket packet;
static int bytesRemaining=0;
static uint8_t *rawData;
static bool fFirstTime=true;
int bytesDecoded;
int frameFinished;
// First time we're called, set packet.data to NULL to indicate it
// doesn't have to be freed
if(fFirstTime)
{
fFirstTime=false;
packet.data=NULL;
}
// Decode packets until we have decoded a complete frame
while(true)
{
// Work on the current packet until we have decoded all of it
while(bytesRemaining > 0)
{
// Decode the next chunk of data
bytesDecoded=avcodec_decode_video(pCodecCtx, pFrame,
&frameFinished, rawData, bytesRemaining);
// Was there an error?
if(bytesDecoded < 0)
{
fprintf(stderr, "Error while decoding frame/n");
return false;
}
bytesRemaining-=bytesDecoded;
rawData+=bytesDecoded;
// Did we finish the current frame? Then we can return
if(frameFinished)
return true;
}
// Read the next packet, skipping all packets that aren't for this
// stream
do
{
// Free old packet
if(packet.data!=NULL)
av_free_packet(&packet);
// Read new packet
if(av_read_packet(pFormatCtx, &packet)<0)
goto loop_exit;
} while(packet.stream_index!=videoStream);
bytesRemaining=packet.size;
rawData=packet.data;
}
loop_exit:
// Decode the rest of the last frame
bytesDecoded=avcodec_decode_video(pCodecCtx, pFrame, &frameFinished,
rawData, bytesRemaining);
// Free last packet
if(packet.data!=NULL)
av_free_packet(&packet);
return frameFinished!=0;
}
Now, all we have to do is sit in a loop, calling GetNextFrame()
until
it returns false. Just one more thing to take care of: Most codecs return
images in YUV 420 format (one luminance and two chrominance channels, with the
chrominance channels samples at half the spatial resolution of the luminance
channel). Depending on what you want to do with the video data, you may want
to convert this to RGB. (Note, though, that this is not necessary if all you
want to do is display the video data; take a look at the X11 Xvideo extension,
which does YUV-to-RGB and scaling in hardware.) Fortunately, libavcodec
provides a conversion routine called img_convert
, which does
conversion between YUV and RGB as well as a variety of other image formats.
The loop that decodes the video thus becomes:
while(GetNextFrame(pFormatCtx, pCodecCtx, videoStream, pFrame))
{
img_convert((AVPicture *)pFrameRGB, PIX_FMT_RGB24, (AVPicture*)pFrame,
pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height);
// Process the video frame (save to disk etc.)
DoSomethingWithTheImage(pFrameRGB);
}
The RGB image pFrameRGB
(of type AVFrame *
) is allocated
like this:
AVFrame *pFrameRGB;
int numBytes;
uint8_t *buffer;
// Allocate an AVFrame structure
pFrameRGB=avcodec_alloc_frame();
if(pFrameRGB==NULL)
handle_error();
// Determine required buffer size and allocate buffer
numBytes=avpicture_get_size(PIX_FMT_RGB24, pCodecCtx->width,
pCodecCtx->height);
buffer=new uint8_t[numBytes];
// Assign appropriate parts of buffer to image planes in pFrameRGB
avpicture_fill((AVPicture *)pFrameRGB, buffer, PIX_FMT_RGB24,
pCodecCtx->width, pCodecCtx->height);
Cleaning up
OK, we've read and processed our video, now all that's left for us to do is
clean up after ourselves:
// Free the RGB image
delete [] buffer;
av_free(pFrameRGB);
// Free the YUV frame
av_free(pFrame);
// Close the codec
avcodec_close(pCodecCtx);
// Close the video file
av_close_input_file(pFormatCtx);
Done!
Sample Code
A sample app that wraps all of this code up in compilable form is
here
. If you have any additional comments,
please contact me at boehme@inb.uni-luebeckREMOVETHIS.de
. Standard
disclaimer: I assume no liability for the correct functioning of the code
and techniques presented in this article.
分享到:
相关推荐
Using FFmpeg and libavcodec you can develop applications that can parse and display video files and be sure they will work on most platform, including of course Linux, Macintosh, and Windows boxes....
libavformat和libavcodec的简单应用
最新的libavformat以及libavcodec库
LAV Filters are a set of DirectShow filters based on the libavformat and libavcodec libraries from the ffmpeg project, which will allow you to play virtually any format in a DirectShow player. The ...
ffmpeg开发指南(经典),使用 libavformat 和 libavcodec
ffmpeg开发指南(使用 libavformat 和 libavcodec) Ffmpeg 中的Libavformat 和 libavcodec库是访问大多数视频文件格式的一个很好的方法。不幸的是,在开发您自己的程序时,这套库基本上没有提供什么实际的文档...
FFmpeg开发指南,使用 libavcodec 和 libavformat。libavcodec和libavformat是访问大多数视频文件格式的一个很好方法。
事情就是初始化libavformat/libavcodec: av_register_all(); 这一步注册库中含有的所有可用的文件格式和编码器,这样当打开一个文件时,它们才能 够自动选择相应的文件格式和编码器。要注意你只需调用一次 av_...
lav filters 是一组基于 ffmpeg 项目中的 libavformat/libavcodec 库的 directshow 分离器和音视频解码器,几乎允许您在 directshow 播放器中播放任何格式的媒体文件!
LAV Filters中文版是一款非常不错的视频...LAV Filters中文版是一组基于 ffmpeg 项目中的 libavformat/libavcodec 库的 directshow 分离器和音视频解码器,几乎允许您在 directshow 播放器中播放任何格式的媒体文件!
It contains libavcodec, libavutil, libavformat, libavdevice, libswscale and libswresample which can be used by applications. As well as ffmpeg, ffserver, ffplay and ffprobe which can be used by end ...
Ffmpeg 中的Libavformat 和 libavcodec库是访问大多数视频文件格式的一个很好的方法。不幸的是,在开发您自己的程序时,这套库基本上没有提供什么实际的文档可以用来作为参考(至少我没有找到任何文档),并且它的...
软件基于 ffmpeg 项目中的 libavformat/libavcodec 库,旨在最终取代播放链中的绝大部分滤镜,能播放所有的格式和任何现代媒体。软件除了通用的 DXVA2 外,也特别支持 NVIDIA CUDA 和 Intel QuickSync。软件简单全面...
ffmpeg libavformat.a静态库(包含i386,armv7,armv7s,arm64)
ffmpeg 的vs2008 项目及源码, 包含 libavcodec, libavformat,libavutil,libpostproc,libswscale
本书是杨书良对ffmpeg源码的分析,其对代码的分析程序不亚于《深入浅出》 包括libavutil libavformat libavcodec ffplay代码剖析
vgtmpeg是ffmpeg的替代产品,它为ffmpeg和libavformat / libavcodec库增加了许多附加功能: 通过使用添加新的dvdurl协议来DVD读取功能 通过使用libbluray获得蓝光读取功能 DVD / Bluray信息在转码后的流中具有丰富...
DirectShow分离器和音视解码器(LAV Filters)汉化版是一组基于 ffmpeg 项目中的 libavformat/libavcodec 库的 DirectShow 分离器和音视解码器,几乎允许您在 DirectShow 播放器中播放任何格式。 LAV Filters 包含三...