| Video Acceleration |
| ------------------ |
| |
| Status: |
| |
| DRAFT. Outdated for 1.0, we want to use video/x-raw in order to |
| interoperate with all elements that handle video. Optimized handling of |
| the VA surfaces can be done with the metadata. |
| |
| Purpose: |
| |
| Provide an standardized generic way to introduce Video Acceleration APIs in |
| already available elements instead of duplicating those into specialized ones. |
| |
| Provide a mechanism for a light GstBuffer subclassing in order to be able |
| exchange VA related context and surfaces between elements. |
| |
| Provide some basic primitives to be used in the elements keeping |
| the implementation of those in the backends of the helper library. |
| |
| The motivation of this proposal is: |
| - to avoid multiple duplicty of code |
| - to avoid the use of GstBuffer subclassing and be more close to GstBuffer2 |
| - to avoid the overhead that's introduced with subclassing/GType checks |
| - to permit multiple elements interact having a common standard |
| |
| Proposal: |
| |
| video/x-raw-va |
| |
| Light subclassing embedding an structure in the data field of a standard |
| GstBuffer. |
| |
| struct { |
| context; |
| surface; |
| flags; |
| subpicture, x, y, w, h; |
| parent * |
| ... |
| } VASurface |
| |
| GstVA helper library |
| -------------------- |
| |
| Common API that it's implemented by a backend for each VA API. |
| |
| +-------+ +---------------+ |
| | |----> | VDPAU backend | |
| | GstVA | +---------------+ |
| | | +---------------+ |
| | |----> | VAAPI backend | |
| +-------+ +---------------+ |
| |
| gst_va_init () : Iterate in the backends and try to initialize those, stops |
| when a backend is able to be created. |
| |
| bool gst_va_available() : permit to query if there's VA available in runtime. |
| |
| GstBuffer * gst_va_buffer_new (vacontext, vasurface, vaflags, parent*, |
| free_func*): |
| Create a new GstBuffer with the proper light subclass and store on it |
| the provided context, surface, flags... |
| - parent* and free_func* to implement a mechanism to return surfaces in |
| the pool driven by parent. |
| |
| gst_va_buffer_get_context (GstBuffer *) |
| gst_va_buffer_get_surface (GstBuffer *) |
| gst_va_buffer_get_flags (GstBuffer *) |
| .... |
| Some public/private? accessors to be able recover fields from a VA GstBuffer |
| internal structure. |
| |
| Primitives: |
| |
| gst_va_render_display (GstBuffer * va, display, x, y, w, h, ...) |
| Put a surface in the screen at the specified position. |
| |
| gst_va_render_pixmap (GstBuffer * va, pixmap *, w, h, ...) |
| Put a surface in a pixmap, to be used paired with GLX_texture_from_pixmap to |
| upload into a OpenGL texture. |
| |
| gst_va_render_memory (GstBuffer * va, planes *, strides *, w, h, ...) |
| To pull back into main memory a VA surface. |
| |
| gst_va_buffer_associate_subpicture (GstBuffer * va, GstBuffer * subpicture, |
| x, y, w, h,...) |
| Take a reference of subpicture and keep it in va internal structure. |
| |
| Use cases: |
| |
| Instead of create a new specialized element for each API just change |
| a generic element once. |
| |
| To add support for a new API just have to create a new backend. |
| |
| xvimagesink |
| ----------- |
| |
| - In _setcaps negotiate accept VA kind of stream when gst_va_init and |
| gst_va_available tell us that we have hardware capable of it. |
| |
| - In _render when the buffers have the VA flag render display it with |
| gst_va_render_display() |
| |
| videomixer, dvdspu, textoverlay, ... |
| ------------------------------------ |
| |
| - In _setcaps negotiate accept VA kind of stream when gst_va_init and |
| gst_va_available tell us that we have hardware capable of it. |
| |
| - Use gst_va_buffer_associate_subpicture () where necessary. |
| |