| <chapter id="chapter-advanced-events"> |
| <title>Events: Seeking, Navigation and More</title> |
| <para> |
| There are many different event types but only two ways they can travel in |
| the pipeline: downstream or upstream. It is very important to understand |
| how both of these methods work because if one element in the pipeline is not |
| handling them correctly the whole event system of the pipeline is broken. |
| We will try to explain here how these methods work and how elements are |
| supposed to implement them. |
| </para> |
| <sect1 id="section-events-downstream" xreflabel="Downstream events"> |
| <title>Downstream events</title> |
| <para> |
| Downstream events are received through the sink pad's event handler, |
| as set using <function>gst_pad_set_event_function ()</function> when |
| the pad was created. |
| </para> |
| <para> |
| Downstream events can travel in two ways: they can be in-band (serialised |
| with the buffer flow) or out-of-band (travelling through the pipeline |
| instantly, possibly not in the same thread as the streaming thread that |
| is processing the buffers, skipping ahead of buffers being processed |
| or queued in the pipeline). The most common downstream events |
| (SEGMENT, CAPS, TAG, EOS) are all serialised with the buffer flow. |
| </para> |
| <para> |
| Here is a typical event function: |
| </para> |
| <programlisting> |
| static gboolean |
| gst_my_filter_sink_event (GstPad *pad, GstObject * parent, GstEvent * event) |
| { |
| GstMyFilter *filter; |
| gboolean ret; |
| |
| filter = GST_MY_FILTER (parent); |
| ... |
| |
| switch (GST_EVENT_TYPE (event)) { |
| case GST_EVENT_SEGMENT: |
| /* maybe save and/or update the current segment (e.g. for output |
| * clipping) or convert the event into one in a different format |
| * (e.g. BYTES to TIME) or drop it and set a flag to send a segment |
| * event in a different format later */ |
| ret = gst_pad_push_event (filter->src_pad, event); |
| break; |
| case GST_EVENT_EOS: |
| /* end-of-stream, we should close down all stream leftovers here */ |
| gst_my_filter_stop_processing (filter); |
| ret = gst_pad_push_event (filter->src_pad, event); |
| break; |
| case GST_EVENT_FLUSH_STOP: |
| gst_my_filter_clear_temporary_buffers (filter); |
| ret = gst_pad_push_event (filter->src_pad, event); |
| break; |
| default: |
| ret = gst_pad_event_default (pad, parent, event); |
| break; |
| } |
| |
| ... |
| return ret; |
| } |
| </programlisting> |
| <para> |
| If your element is chain-based, you will almost always have to implement |
| a sink event function, since that is how you are notified about |
| segments, caps and the end of the stream. |
| </para> |
| <para> |
| If your element is exclusively loop-based, you may or may not want a |
| sink event function (since the element is driving the pipeline it will |
| know the length of the stream in advance or be notified by the flow |
| return value of <function>gst_pad_pull_range()</function>. In some cases |
| even loop-based element may receive events from upstream though (for |
| example audio decoders with an id3demux or apedemux element in front of |
| them, or demuxers that are being fed input from sources that send |
| additional information about the stream in custom events, as DVD sources |
| do). |
| </para> |
| </sect1> |
| <sect1 id="section-events-upstream" xreflabel="Upstream events"> |
| <title>Upstream events</title> |
| <para> |
| Upstream events are generated by an element somewhere downstream in |
| the pipeline (example: a video sink may generate navigation |
| events that informs upstream elements about the current position of |
| the mouse pointer). This may also happen indirectly on request of the |
| application, for example when the application executes a seek on a |
| pipeline this seek request will be passed on to a sink element which |
| will then in turn generate an upstream seek event. |
| </para> |
| <para> |
| The most common upstream events are seek events, Quality-of-Service |
| (QoS) and reconfigure events. |
| </para> |
| <para> |
| An upstream event can be sent using the |
| <function>gst_pad_send_event</function> function. This |
| function simply call the default event handler of that pad. The default |
| event handler of pads is <function>gst_pad_event_default</function>, and |
| it basically sends the event to the peer of the internally linked pad. |
| So upstream events always arrive on the src pad of your element and are |
| handled by the default event handler except if you override that handler |
| to handle it yourself. There are some specific cases where you have to |
| do that : |
| </para> |
| <itemizedlist mark="opencircle"> |
| <listitem> |
| <para> |
| If you have multiple sink pads in your element. In that case you will |
| have to decide which one of the sink pads you will send the event to |
| (if not all of them). |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| If you need to handle that event locally. For example a navigation |
| event that you will want to convert before sending it upstream, or |
| a QoS event that you want to handle. |
| </para> |
| </listitem> |
| </itemizedlist> |
| <para> |
| The processing you will do in that event handler does not really matter |
| but there are important rules you have to absolutely respect because |
| one broken element event handler is breaking the whole pipeline event |
| handling. Here they are : |
| </para> |
| <itemizedlist mark="opencircle"> |
| <listitem> |
| <para> |
| Always handle events you won't handle using the default |
| <function>gst_pad_event_default</function> method. This method will |
| depending on the event, forward the event or drop it. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| If you are generating some new event based on the one you received |
| don't forget to gst_event_unref the event you received. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Event handler function are supposed to return TRUE or FALSE indicating |
| if the event has been handled or not. Never simply return TRUE/FALSE |
| in that handler except if you really know that you have handled that |
| event. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Remember that the event handler might be called from a different |
| thread than the streaming thread, so make sure you use |
| appropriate locking everywhere. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </sect1> |
| |
| <sect1 id="section-events-definitions" xreflabel="All Events Together"> |
| <title>All Events Together</title> |
| <para> |
| In this chapter follows a list of all defined events that are currently |
| being used, plus how they should be used/interpreted. You can check the |
| what type a certain event is using the GST_EVENT_TYPE macro (or if you |
| need a string for debugging purposes you can use GST_EVENT_TYPE_NAME). |
| </para> |
| <para> |
| In this chapter, we will discuss the following events: |
| </para> |
| <itemizedlist> |
| <listitem><para><xref linkend="section-events-stream-start"/></para></listitem> |
| <listitem><para><xref linkend="section-events-caps"/></para></listitem> |
| <listitem><para><xref linkend="section-events-segment"/></para></listitem> |
| <listitem><para><xref linkend="section-events-tag"/></para></listitem> |
| <listitem><para><xref linkend="section-events-eos"/></para></listitem> |
| <listitem><para><xref linkend="section-events-toc"/></para></listitem> |
| <listitem><para><xref linkend="section-events-gap"/></para></listitem> |
| <listitem><para><xref linkend="section-events-flush-start"/></para></listitem> |
| <listitem><para><xref linkend="section-events-flush-stop"/></para></listitem> |
| <listitem><para><xref linkend="section-events-qos"/></para></listitem> |
| <listitem><para><xref linkend="section-events-seek"/></para></listitem> |
| <listitem><para><xref linkend="section-events-nav"/></para></listitem> |
| </itemizedlist> |
| <para> |
| For more comprehensive information about events and how they should be |
| used correctly in various circumstances please consult the GStreamer |
| design documentation. This section only gives a general overview. |
| </para> |
| |
| <sect2 id="section-events-stream-start" xreflabel="Stream Start"> |
| <title>Stream Start</title> |
| <para> |
| WRITEME |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-caps" xreflabel="Caps"> |
| <title>Caps</title> |
| <para> |
| The CAPS event contains the format description of the following |
| buffers. See <xref linkend="chapter-negotiation"/> for more |
| information about negotiation. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-segment" xreflabel="Segment"> |
| <title>Segment</title> |
| <para> |
| A segment event is sent downstream to announce the range of valid |
| timestamps in the stream and how they should be transformed into |
| running-time and stream-time. A segment event must always be sent |
| before the first buffer of data and after a flush (see above). |
| </para> |
| <para> |
| The first segment event is created by the element driving the |
| pipeline, like a source operating in push-mode or a demuxer/decoder |
| operating pull-based. This segment event then travels down the |
| pipeline and may be transformed on the way (a decoder, for example, |
| might receive a segment event in BYTES format and might transform |
| this into a segment event in TIMES format based on the average |
| bitrate). |
| </para> |
| <para> |
| Depending on the element type, the event can simply be forwarded using |
| <function>gst_pad_event_default ()</function>, or it should be parsed |
| and a modified event should be sent on. The last is true for demuxers, |
| which generally have a byte-to-time conversion concept. Their input |
| is usually byte-based, so the incoming event will have an offset in |
| byte units (<symbol>GST_FORMAT_BYTES</symbol>), too. Elements |
| downstream, however, expect segment events in time units, so that |
| it can be used to synchronize against the pipeline clock. Therefore, |
| demuxers and similar elements should not forward the event, but parse |
| it, free it and send a segment event (in time units, |
| <symbol>GST_FORMAT_TIME</symbol>) further downstream. |
| </para> |
| <para> |
| The segment event is created using the function |
| <function>gst_event_new_segment ()</function>. See the API |
| reference and design document for details about its parameters. |
| </para> |
| <para> |
| Elements parsing this event can use gst_event_parse_segment() |
| to extract the event details. Elements may find the GstSegment |
| API useful to keep track of the current segment (if they want to use |
| it for output clipping, for example). |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-tag" xreflabel="Tag (metadata)"> |
| <title>Tag (metadata)</title> |
| <para> |
| Tagging events are being sent downstream to indicate the tags as parsed |
| from the stream data. This is currently used to preserve tags during |
| stream transcoding from one format to the other. Tags are discussed |
| extensively in <xref linkend="chapter-advanced-tagging"/>. Most |
| elements will simply forward the event by calling |
| <function>gst_pad_event_default ()</function>. |
| </para> |
| <para> |
| The tag event is created using the function |
| <function>gst_event_new_tag ()</function>, but more often elements will |
| send a tag event downstream that will be converted into a message |
| on the bus by sink elements. |
| All of these functions require a filled-in taglist as |
| argument, which they will take ownership of. |
| </para> |
| <para> |
| Elements parsing this event can use the function |
| <function>gst_event_parse_tag ()</function> to acquire the |
| taglist that the event contains. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-eos" xreflabel="End of Stream (EOS)"> |
| <title>End of Stream (EOS)</title> |
| <para> |
| End-of-stream events are sent if the stream that an element sends out |
| is finished. An element receiving this event (from upstream, so it |
| receives it on its sinkpad) will generally just process any buffered |
| data (if there is any) and then forward the event further downstream. |
| The <function>gst_pad_event_default ()</function> takes care of all |
| this, so most elements do not need to support this event. Exceptions are |
| elements that explicitly need to close a resource down on EOS, and |
| N-to-1 elements. Note that the stream itself is <emphasis>not</emphasis> |
| a resource that should be closed down on EOS! Applications might seek |
| back to a point before EOS and continue playing again. |
| </para> |
| <para> |
| The EOS event has no properties, which makes it one of the simplest |
| events in &GStreamer;. It is created using the |
| <function>gst_event_new_eos()</function> function. |
| </para> |
| <para> |
| It is important to note that <emphasis>only elements driving the |
| pipeline should ever send an EOS event</emphasis>. If your element |
| is chain-based, it is not driving the pipeline. Chain-based elements |
| should just return GST_FLOW_EOS from their chain function at |
| the end of the stream (or the configured segment), the upstream |
| element that is driving the pipeline will then take care of |
| sending the EOS event (or alternatively post a SEGMENT_DONE message |
| on the bus depending on the mode of operation). If you are implementing |
| your own source element, you also do not need to ever manually send |
| an EOS event, you should also just return GST_FLOW_EOS in |
| your create or fill function (assuming your element derives from |
| GstBaseSrc or GstPushSrc). |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-toc" xreflabel="Table Of Contents"> |
| <title>Table Of Contents</title> |
| <para> |
| WRITEME |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-gap" xreflabel="Gap"> |
| <title>Gap</title> |
| <para> |
| WRITEME |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-flush-start" xreflabel="Flush Start"> |
| <title>Flush Start</title> |
| <para> |
| The flush start event is sent downstream (in push mode) or upstream |
| (in pull mode) if all buffers and caches in the pipeline should be |
| emptied. <quote>Queue</quote> elements will |
| empty their internal list of buffers when they receive this event, for |
| example. File sink elements (e.g. <quote>filesink</quote>) will flush |
| the kernel-to-disk cache (<function>fdatasync ()</function> or |
| <function>fflush ()</function>) when they receive this event. Normally, |
| elements receiving this event will simply just forward it, since most |
| filter or filter-like elements don't have an internal cache of data. |
| <function>gst_pad_event_default ()</function> does just that, so for |
| most elements, it is enough to forward the event using the default |
| event handler. |
| </para> |
| <para> |
| As a side-effect of flushing all data from the pipeline, this event |
| unblocks the streaming thread by making all pads reject data until |
| they receive a <xref linkend="section-events-flush-stop"/> signal |
| (elements trying to push data will get a FLUSHING flow return |
| and stop processing data). |
| </para> |
| <para> |
| The flush-start event is created with the |
| <function>gst_event_new_flush_start ()</function>. |
| Like the EOS event, it has no properties. This event is usually |
| only created by elements driving the pipeline, like source elements |
| operating in push-mode or pull-range based demuxers/decoders. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-flush-stop" xreflabel="Flush Stop"> |
| <title>Flush Stop</title> |
| <para> |
| The flush-stop event is sent by an element driving the pipeline |
| after a flush-start and tells pads and elements downstream that |
| they should accept events and buffers again (there will be at |
| least a SEGMENT event before any buffers first though). |
| </para> |
| <para> |
| If your element keeps temporary caches of stream data, it should |
| clear them when it receives a FLUSH-STOP event (and also whenever |
| its chain function receives a buffer with the DISCONT flag set). |
| </para> |
| <para> |
| The flush-stop event is created with |
| <function>gst_event_new_flush_stop ()</function>. It has one |
| parameter that controls if the running-time of the pipeline should |
| be reset to 0 or not. Normally after a flushing seek, the |
| running_time is set back to 0. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-qos" xreflabel="Quality Of Service (QOS)"> |
| <title>Quality Of Service (QOS)</title> |
| <para> |
| The QOS event contains a report about the current real-time |
| performance of the stream. See more info in |
| <xref linkend="chapter-advanced-qos"/>. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-seek" xreflabel="Seek Request"> |
| <title>Seek Request</title> |
| <para> |
| Seek events are meant to request a new stream position to elements. |
| This new position can be set in several formats (time, bytes or |
| <quote>default units</quote> [a term indicating frames for video, |
| channel-independent samples for audio, etc.]). Seeking can be done with |
| respect to the end-of-file or start-of-file, and |
| usually happens in upstream direction (downstream seeking is done by |
| sending a SEGMENT event with the appropriate offsets for elements |
| that support that, like filesink). |
| </para> |
| <para> |
| Elements receiving seek events should, depending on the element type, |
| either just forward it upstream (filters, decoders), change the |
| format in which the event is given and then forward it (demuxers), |
| or handle the event by changing the file pointer in their internal |
| stream resource (file sources, demuxers/decoders driving the pipeline |
| in pull-mode) or something else. |
| </para> |
| <para> |
| Seek events are built up using positions in specified formats (time, |
| bytes, units). They are created using the function |
| <function>gst_event_new_seek ()</function>. Note that many plugins do |
| not support seeking from the end of the stream. |
| An element not driving the pipeline and forwarding a seek |
| request should not assume that the seek succeeded or actually happened, |
| it should operate based on the SEGMENT events it receives. |
| </para> |
| <para> |
| Elements parsing this event can do this using |
| <function>gst_event_parse_seek()</function>. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-events-nav" xreflabel="Navigation"> |
| <title>Navigation</title> |
| <para> |
| Navigation events are sent upstream by video sinks to inform upstream |
| elements of where the mouse pointer is, if and where mouse pointer |
| clicks have happened, or if keys have been pressed or released. |
| </para> |
| <para> |
| All this information is contained in the event structure which can |
| be obtained with <function>gst_event_get_structure ()</function>. |
| </para> |
| <para> |
| Check out the navigationtest element in gst-plugins-good for an idea |
| how to extract navigation information from this event. |
| </para> |
| </sect2> |
| |
| </sect1> |
| </chapter> |