blob: 2bff4bc3e7beda8d5c0a061376ac64212f88ccf5 [file] [log] [blame]
Video Acceleration
------------------
Status:
DRAFT. Outdated for 1.0, we want to use video/x-raw in order to
interoperate with all elements that handle video. Optimized handling of
the VA surfaces can be done with the metadata.
Purpose:
Provide an standardized generic way to introduce Video Acceleration APIs in
already available elements instead of duplicating those into specialized ones.
Provide a mechanism for a light GstBuffer subclassing in order to be able
exchange VA related context and surfaces between elements.
Provide some basic primitives to be used in the elements keeping
the implementation of those in the backends of the helper library.
The motivation of this proposal is:
- to avoid multiple duplicty of code
- to avoid the use of GstBuffer subclassing and be more close to GstBuffer2
- to avoid the overhead that's introduced with subclassing/GType checks
- to permit multiple elements interact having a common standard
Proposal:
video/x-raw-va
Light subclassing embedding an structure in the data field of a standard
GstBuffer.
struct {
context;
surface;
flags;
subpicture, x, y, w, h;
parent *
...
} VASurface
GstVA helper library
--------------------
Common API that it's implemented by a backend for each VA API.
+-------+ +---------------+
| |----> | VDPAU backend |
| GstVA | +---------------+
| | +---------------+
| |----> | VAAPI backend |
+-------+ +---------------+
gst_va_init () : Iterate in the backends and try to initialize those, stops
when a backend is able to be created.
bool gst_va_available() : permit to query if there's VA available in runtime.
GstBuffer * gst_va_buffer_new (vacontext, vasurface, vaflags, parent*,
free_func*):
Create a new GstBuffer with the proper light subclass and store on it
the provided context, surface, flags...
- parent* and free_func* to implement a mechanism to return surfaces in
the pool driven by parent.
gst_va_buffer_get_context (GstBuffer *)
gst_va_buffer_get_surface (GstBuffer *)
gst_va_buffer_get_flags (GstBuffer *)
....
Some public/private? accessors to be able recover fields from a VA GstBuffer
internal structure.
Primitives:
gst_va_render_display (GstBuffer * va, display, x, y, w, h, ...)
Put a surface in the screen at the specified position.
gst_va_render_pixmap (GstBuffer * va, pixmap *, w, h, ...)
Put a surface in a pixmap, to be used paired with GLX_texture_from_pixmap to
upload into a OpenGL texture.
gst_va_render_memory (GstBuffer * va, planes *, strides *, w, h, ...)
To pull back into main memory a VA surface.
gst_va_buffer_associate_subpicture (GstBuffer * va, GstBuffer * subpicture,
x, y, w, h,...)
Take a reference of subpicture and keep it in va internal structure.
Use cases:
Instead of create a new specialized element for each API just change
a generic element once.
To add support for a new API just have to create a new backend.
xvimagesink
-----------
- In _setcaps negotiate accept VA kind of stream when gst_va_init and
gst_va_available tell us that we have hardware capable of it.
- In _render when the buffers have the VA flag render display it with
gst_va_render_display()
videomixer, dvdspu, textoverlay, ...
------------------------------------
- In _setcaps negotiate accept VA kind of stream when gst_va_init and
gst_va_available tell us that we have hardware capable of it.
- Use gst_va_buffer_associate_subpicture () where necessary.