69 lines
3 KiB
Text
69 lines
3 KiB
Text
Notes on the hevc_rpi decoder & associated support code
|
|
-------------------------------------------------------
|
|
|
|
There are 3 main parts to the existing code:
|
|
|
|
1) The decoder - this is all in libavcodec as rpi_hevc*.
|
|
|
|
2) A few filters to deal with Sand frames and a small patch to
|
|
automatically select the sand->i420 converter when required.
|
|
|
|
3) A kludge in ffmpeg.c to display the decoded video. This could & should
|
|
be converted into a proper ffmpeg display module.
|
|
|
|
|
|
Decoder
|
|
-------
|
|
|
|
The decoder is a modified version of the existing ffmpeg hevc decoder.
|
|
Generally it is ~100% faster than the existing ffmpeg hevc s/w decoder.
|
|
More complex bitstreams can be up to ~200% faster but particularly easy
|
|
streams can cut its advantage down to ~50%. This means that a Pi3+ can
|
|
display nearly all 8-bit 1080p30 streams and with some overclocking it can
|
|
display most lower bitrate 10-bit 1080p30 streams - this latter case is
|
|
not helped by the requirement to downsample to 8-bit before display on a
|
|
Pi.
|
|
|
|
It has had co-processor offload added for inter-pred and large block
|
|
residual transform. Various parts have had optimized ARM NEON assembler
|
|
added and the existing ARM asm sections have been profiled and
|
|
re-optimized for A53. The main C code has been substantially reworked at
|
|
its lower levels in an attempt to optimize it and minimize memory
|
|
bandwidth. To some extent code paths that deal with frame types that it
|
|
doesn't support have been pruned.
|
|
|
|
It outputs frames in Broadcom Sand format. This is a somewhat annoying
|
|
layout that doesn't fit into ffmpegs standard frame descriptions. It has
|
|
vertical stripes of 128 horizontal pixels (64 in 10 bit forms) with Y for
|
|
the stripe followed by interleaved U & V, that is then followed by the Y
|
|
for the next stripe, etc. The final stripe is always padded to
|
|
stripe-width. This is used in an attempt to help with cache locality and
|
|
cut down on the number of dram bank switches. It is annoying to use for
|
|
inter-pred with conventional processing but the way the Pi QPU (which is
|
|
used for inter-pred) works means that it has negligible downsides here and
|
|
the improved memory performance exceeds the overhead of the increased
|
|
complexity in the rest of the code.
|
|
|
|
Frames must be allocated out of GPU memory (as otherwise they can't be
|
|
accessed by the co-processors). Utility functions (in rpi_zc.c) have been
|
|
written to make this easier. As the frames are already in GPU memory they
|
|
can be displayed by the Pi h/w without any further copying.
|
|
|
|
|
|
Known non-features
|
|
------------------
|
|
|
|
Frame allocation should probably be done in some other way in order to fit
|
|
into the standard framework better.
|
|
|
|
Sand frames are currently declared as software frames, there is an
|
|
argument that they should be hardware frames but they aren't really.
|
|
|
|
There must be a better way of auto-selecting the hevc_rpi decoder over the
|
|
normal s/w hevc decoder, but I became confused by the existing h/w
|
|
acceleration framework and what I wanted to do didn't seem to fit in
|
|
neatly.
|
|
|
|
Display should be a proper device rather than a kludge in ffmpeg.c
|
|
|
|
|