I'm trying to decode H.264 stream, which is sent over Socket from an Android application to a computer. And I also want to show the decoded stream using JavaFX. I searched for a long time, and decided to use JavaCV / FFmpeg. However I got error from FFmpeg. (I was inspired by this code.)
Questions:
- Why does FFmpeg make error?
- Is it a correct way to convert
AVFrametojavafx.scene.image.Image?
I'm using:
- javacv-platform 1.4.4
- ffmpeg-platform 4.1-1.4.4
Code:
This is a part of import and class fields, and method which runs once at the first time. (Actually the content of initialize() is wrapped by try~catch.)
import javafx.scene.image.Image;
private avcodec.AVCodec avCodec;
private avcodec.AVCodecContext avCodecContext;
private avutil.AVDictionary avDictionary;
private avutil.AVFrame avFrame;
public void initialize() {
avCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
if (avCodec == null) {
throw new RuntimeException("Can't find decoder");
}
avCodecContext = avcodec_alloc_context3(avCodec);
if (avCodecContext == null) {
throw new RuntimeException("Can't allocate decoder context");
}
int result = avcodec_open2(avCodecContext, avCodec, (AVDictionary) null);
if (result < 0) {
throw new RuntimeException("Can't open decoder");
}
avFrame = av_frame_alloc();
if (avFrame == null) {
throw new RuntimeException("Can't allocate frame");
}
}
And this is a method which is called every time when I receive a packet from Android. byte[] data is the packet data starting with 0x00, 0x00, 0x00, 0x01.
The place where I get error is number_of_written_bytes. It always gets <0.
private void decode(byte[] data) {
AVPacket avPacket = new AVPacket();
av_init_packet(avPacket);
avPacket.pts(AV_NOPTS_VALUE);
avPacket.dts(AV_NOPTS_VALUE);
BytePointer bytePointer = new BytePointer(data);
bytePointer.capacity(data.length);
avPacket.data(bytePointer);
avPacket.size(data.length);
avPacket.pos(-1);
avcodec_send_packet(avCodecContext, avPacket);
int result = avcodec_receive_frame(avCodecContext, avFrame);
if (result >= 0) {
int bufferOutputSize = av_image_get_buffer_size(avFrame.format(), avFrame.width(), avFrame.height(), 16);
Pointer pointer = av_malloc(bufferOutputSize);
BytePointer outputPointer = new BytePointer(pointer);
int number_of_written_bytes = av_image_copy_to_buffer(outputPointer, bufferOutputSize, avFrame.data(), avFrame.linesize(), avFrame.chroma_location(), avFrame.width(), avFrame.height(), 1);
if (number_of_written_bytes < 0) {
//The process always come here.
throw new RuntimeException("Can't copy image to buffer");
}
System.out.println("decode success");
Image image = new Image(new ByteArrayInputStream(outputPointer.asBuffer().array()));
} else {
System.out.println("decode failed");
}
}
Anything is helpful for me. Thanks.