| Forcing keyframes |
| ----------------- |
| |
| Consider the following use case: |
| |
| We have a pipeline that performs video and audio capture from a live source, |
| compresses and muxes the streams and writes the resulting data into a file. |
| |
| Inside the uncompressed video data we have a specific pattern inserted at |
| specific moments that should trigger a switch to a new file, meaning, we close |
| the existing file we are writing to and start writing to a new file. |
| |
| We want the new file to start with a keyframe so that one can start decoding |
| the file immediately. |
| |
| Components: |
| |
| 1) We need an element that is able to detect the pattern in the video stream. |
| |
| 2) We need to inform the video encoder that it should start encoding a keyframe |
| starting from exactly the frame with the pattern. |
| |
| 3) We need to inform the demuxer that it should flush out any pending data and |
| start creating the start of a new file with the keyframe as a first video |
| frame. |
| |
| 4) We need to inform the sink element that it should start writing to the next |
| file. This requires application interaction to instruct the sink of the new |
| filename. The application should also be free to ignore the boundary and |
| continue to write to the existing file. The application will typically use |
| an event pad probe to detect the custom event. |
| |
| Implementation: |
| |
| The implementation would consist of generating a GST_EVENT_CUSTOM_DOWNSTREAM |
| event that marks the keyframe boundary. This event is inserted into the |
| pipeline by the application upon a certain trigger. In the above use case this |
| trigger would be given by the element that detects the pattern, in the form of |
| an element message. |
| |
| The custom event would travel further downstream to instruct encoder, muxer and |
| sink about the possible switch. |
| |
| The information passed in the event consists of: |
| |
| name: GstForceKeyUnit |
| (G_TYPE_UINT64)"timestamp" : the timestamp of the buffer that |
| triggered the event. |
| (G_TYPE_UINT64)"stream-time" : the stream position that triggered the |
| event. |
| (G_TYPE_UINT64)"running-time" : the running time of the stream when the |
| event was triggered. |
| (G_TYPE_BOOLEAN)"all-headers" : Send all headers, including those in |
| the caps or those sent at the start of |
| the stream. |
| |
| .... : optional other data fields. |
| |
| Note that this event is purely informational, no element is required to |
| perform an action but it should forward the event downstream, just like any |
| other event it does not handle. |
| |
| Elements understanding the event should behave as follows: |
| |
| 1) The video encoder receives the event before the next frame. Upon reception |
| of the event it schedules to encode the next frame as a keyframe. |
| Before pushing out the encoded keyframe it must push the GstForceKeyUnit |
| event downstream. |
| |
| 2) The muxer receives the GstForceKeyUnit event and flushes out its current state, |
| preparing to produce data that can be used as a keyunit. Before pushing out |
| the new data it pushes the GstForceKeyUnit event downstream. |
| |
| 3) The application receives the GstForceKeyUnit on a sink padprobe of the sink |
| and reconfigures the sink to make it perform new actions after receiving |
| the next buffer. |
| |
| |
| Upstream |
| -------- |
| |
| When using RTP packets can get lost or receivers can be added at any time, |
| they may request a new key frame. |
| |
| An downstream element sends an upstream "GstForceKeyUnit" event up the |
| pipeline. |
| |
| When an element produces some kind of key unit in output, but has |
| no such concept in its input (like an encoder that takes raw frames), |
| it consumes the event (doesn't pass it upstream), and instead sends |
| a downstream GstForceKeyUnit event and a new keyframe. |
