2

We're trying to interate ffmpeg4 with Pyglet.

Its all going very well but we have one issue:

If a VP9 (or VP8) file is created using ffmpeg4, from png files with alphas, we get a video file (webm) with alpha internally. ffmpeg -i image_seq%03d.png -qmin 0 -qmax 50 -crf 5 -b:v 1M output.webm

Sample webm videos with alpha can be found on this page: https://simpl.info/videoalpha/

However on playing back we do not get the alpha unless we override the codec to be libvpx. (same behaviour can be demostrated in CLI version of ffplay). We are wrapping avutil, avcodec, etc.

Question: What is the correct way to identify (in code) that a VP8 or VP9 stream has an alpha component ? (So we can load it and access it via AVFrame() for use as an RGBA texture.)

More specifically:

We get the FormatContext from the file using AVFormatContext and the stream AVStream from that FormatContext.

Then from the AVStream we get the 'codecpar' param.

From the AVCodecParameters we can examine the codec_id (167 is VP9) and other useful params like bits_per_coded_sample.

Interestingly these values seem correct for streams like H264 (codec_id=27) at 24 bits but are marked as 0 for the VP9 codec. Leading me to think this is not the correct place to find the proper value.

Neon22
  • 121
  • 2

1 Answers1

0

OK the native codec is just not as good as the libvpx codec which is supplied by the Webm peeps. So if you want this alpha info then you should overide the codec on load with the libvpx codec.

How you do that - see here: https://stackoverflow.com/questions/35340437/how-can-i-use-avformat-open-input-function-ffmpeg

Basically the third argument to avformat_open_input() needs to be a av_find_input_format("libvpx") which is type AVInputFormat

If you're not sure if its a VP8,9 codec in the file context then you need to probe the file first, discover the codec, and then overide it in an actual load. See here for how to do that: https://stackoverflow.com/questions/14134589/what-does-the-avformat-open-input-do

and lastly - if you want to use the native instead of webm codec unless its got an alpha (but why would you) then you can check to see if its alpha flag is set by checking the AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL side data.

Check AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL if the side_data starts with (uint64_t) 1 there is a transparency layer.

But the general consensus sems to be to always use the libvpx decoder.

Edit: more details The id of the google VP8 codec is 139. and 167 is the Google VP9 codec. To replace with the libvpx versions you need to find "libvpx-vp8" and "libvpx-vp9" using e.g. avcodec_find_decoder_by_name("libvpx-vp9")

Neon22
  • 121
  • 2