| <chapter id="chapter-scheduling" xreflabel="Different scheduling modes"> |
| <title>Different scheduling modes</title> |
| <para> |
| The scheduling mode of a pad defines how data is retrieved from (source) |
| or given to (sink) pads. &GStreamer; can operate in two scheduling |
| mode, called push- and pull-mode. &GStreamer; supports elements with pads |
| in any of the scheduling modes where not all pads need to be operating |
| in the same mode. |
| </para> |
| <para> |
| So far, we have only discussed <function>_chain ()</function>-operating |
| elements, i.e. elements that have a chain-function set on their sink pad |
| and push buffers on their source pad(s). We call this the push-mode |
| because a peer element will use <function>gst_pad_push ()</function> on |
| a srcpad, which will cause our <function>_chain ()</function>-function |
| to be called, which in turn causes our element to push out a buffer on |
| the source pad. The initiative to start the dataflow happens somewhere |
| upstream when it pushes out a buffer and all downstream elements get |
| scheduled when their <function>_chain ()</function>-functions are |
| called in turn. |
| </para> |
| <para> |
| Before we explain pull-mode scheduling, let's first understand how the |
| different scheduling modes are selected and activated on a pad. |
| </para> |
| |
| <sect1 id="section-scheduling-activation" |
| xreflabel="The pad activation stage"> |
| <title>The pad activation stage</title> |
| <para> |
| During the element state change of READY->PAUSED, the pads of an |
| element will be activated. This happens first on the source pads and |
| then on the sink pads of the element. &GStreamer; calls the |
| <function>_activate ()</function> of a pad. By default this function |
| will activate the pad in push-mode by calling |
| <function>gst_pad_activate_mode ()</function> with the GST_PAD_MODE_PUSH |
| scheduling mode. |
| It is possible to override the <function>_activate ()</function> of a pad |
| and decide on a different scheduling mode. You can know in what |
| scheduling mode a pad is activated by overriding the |
| <function>_activate_mode ()</function>-function. |
| </para> |
| <para> |
| &GStreamer; allows the different pads of an element to operate in |
| different scheduling modes. This allows for many different possible |
| use-cases. What follows is an overview of some typical use-cases. |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| If all pads of an element are activated in push-mode scheduling, |
| the element as a whole is operating in push-mode. |
| For source elements this means that they will have to start a |
| task that pushes out buffers on the source pad to the downstream |
| elements. |
| Downstream elements will have data pushed to them by upstream elements |
| using the sinkpads <function>_chain ()</function>-function which will |
| push out buffers on the source pads. |
| Prerequisites for this scheduling mode are that a chain-function was |
| set for each sinkpad using <function>gst_pad_set_chain_function ()</function> |
| and that all downstream elements operate in the same mode. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Alternatively, sinkpads can be the driving force behind a pipeline |
| by operating in pull-mode, while the sourcepads |
| of the element still operate in push-mode. In order to be the |
| driving force, those pads start a <classname>GstTask</classname> |
| when they are activated. This task is a thread, which |
| will call a function specified by the element. When called, this |
| function will have random data access (through |
| <function>gst_pad_pull_range ()</function>) over all sinkpads, and |
| can push data over the sourcepads, which effectively means that |
| this element controls data flow in the pipeline. Prerequisites for |
| this mode are that all downstream elements can act in push |
| mode, and that all upstream elements operate in pull-mode (see below). |
| </para> |
| <para> |
| Source pads can be activated in PULL mode by a downstream element |
| when they return GST_PAD_MODE_PULL from the GST_QUERY_SCHEDULING |
| query. Prerequisites for this scheduling mode are that a |
| getrange-function was set for the source pad using |
| <function>gst_pad_set_getrange_function ()</function>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Lastly, all pads in an element can be activated in PULL-mode. |
| However, contrary to the above, this does not mean that they |
| start a task on their own. Rather, it means that they are pull |
| slave for the downstream element, and have to provide random data |
| access to it from their <function>_get_range ()</function>-function. |
| Requirements are that the a <function>_get_range |
| ()</function>-function was set on this pad using the function |
| <function>gst_pad_set_getrange_function ()</function>. Also, if |
| the element has any sinkpads, all those pads (and thereby their |
| peers) need to operate in PULL access mode, too. |
| </para> |
| <para> |
| When a sink element is activated in PULL mode, it should start a |
| task that calls <function>gst_pad_pull_range ()</function> on its |
| sinkpad. It can only do this when the upstream SCHEDULING query |
| returns support for the GST_PAD_MODE_PULL scheduling mode. |
| </para> |
| </listitem> |
| </itemizedlist> |
| <para> |
| In the next two sections, we will go closer into pull-mode scheduling |
| (elements/pads driving the pipeline, and elements/pads providing random |
| access), and some specific use cases will be given. |
| </para> |
| </sect1> |
| |
| <sect1 id="section-scheduling-loop" xreflabel="Pads driving the pipeline"> |
| <title>Pads driving the pipeline</title> |
| <para> |
| Sinkpads operating in pull-mode, with the sourcepads operating in |
| push-mode (or it has no sourcepads when it is a sink), can start a task |
| that will drive the pipeline data flow. |
| Within this task function, you have random access over all of the sinkpads, |
| and push data over the sourcepads. |
| This can come in useful for several different kinds of elements: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| Demuxers, parsers and certain kinds of decoders where data comes |
| in unparsed (such as MPEG-audio or video streams), since those will |
| prefer byte-exact (random) access from their input. If possible, |
| however, such elements should be prepared to operate in push-mode |
| mode, too. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Certain kind of audio outputs, which require control over their |
| input data flow, such as the Jack sound server. |
| </para> |
| </listitem> |
| </itemizedlist> |
| <para> |
| First you need to perform a SCHEDULING query to check if the upstream |
| element(s) support pull-mode scheduling. If that is possible, you |
| can activate the sinkpad in pull-mode. Inside the activate_mode |
| function you can then start the task. |
| </para> |
| <programlisting><!-- example-begin task.c a --> |
| #include "filter.h" |
| #include <string.h> |
| |
| static gboolean gst_my_filter_activate (GstPad * pad, |
| GstObject * parent); |
| static gboolean gst_my_filter_activate_mode (GstPad * pad, |
| GstObject * parent, |
| GstPadMode mode, |
| gboolean active); |
| static void gst_my_filter_loop (GstMyFilter * filter); |
| |
| G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT); |
| <!-- example-end task.c a --> |
| <!-- example-begin task.c c --> |
| static void |
| gst_my_filter_init (GstMyFilter * filter) |
| { |
| <!-- example-end task.c c --> |
| [..]<!-- example-begin task.c d --><!-- |
| --><!-- example-end task.c d --> |
| <!-- example-begin task.c e --> |
| gst_pad_set_activate_function (filter->sinkpad, gst_my_filter_activate); |
| gst_pad_set_activatemode_function (filter->sinkpad, |
| gst_my_filter_activate_mode); |
| <!-- example-end task.c e --> |
| <!-- example-begin task.c f --><!-- |
| gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad); |
| |
| gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad); |
| --><!-- example-end task.c f --> |
| [..]<!-- example-begin task.c g --> |
| } |
| <!-- example-end task.c g --> |
| [..]<!-- example-begin task.c h --><!-- |
| #include "caps.func" |
| --><!-- example-end task.c h --> |
| <!-- example-begin task.c i --> |
| static gboolean |
| gst_my_filter_activate (GstPad * pad, GstObject * parent) |
| { |
| GstQuery *query; |
| gboolean pull_mode; |
| |
| /* first check what upstream scheduling is supported */ |
| query = gst_query_new_scheduling (); |
| |
| if (!gst_pad_peer_query (pad, query)) { |
| gst_query_unref (query); |
| goto activate_push; |
| } |
| |
| /* see if pull-mode is supported */ |
| pull_mode = gst_query_has_scheduling_mode_with_flags (query, |
| GST_PAD_MODE_PULL, GST_SCHEDULING_FLAG_SEEKABLE); |
| gst_query_unref (query); |
| |
| if (!pull_mode) |
| goto activate_push; |
| |
| /* now we can activate in pull-mode. GStreamer will also |
| * activate the upstream peer in pull-mode */ |
| return gst_pad_activate_mode (pad, GST_PAD_MODE_PULL, TRUE); |
| |
| activate_push: |
| { |
| /* something not right, we fallback to push-mode */ |
| return gst_pad_activate_mode (pad, GST_PAD_MODE_PUSH, TRUE); |
| } |
| } |
| |
| static gboolean |
| gst_my_filter_activate_pull (GstPad * pad, |
| GstObject * parent, |
| GstPadMode mode, |
| gboolean active) |
| { |
| gboolean res; |
| GstMyFilter *filter = GST_MY_FILTER (parent); |
| |
| switch (mode) { |
| case GST_PAD_MODE_PUSH: |
| res = TRUE; |
| break; |
| case GST_PAD_MODE_PULL: |
| if (active) { |
| filter->offset = 0; |
| res = gst_pad_start_task (pad, |
| (GstTaskFunction) gst_my_filter_loop, filter, NULL); |
| } else { |
| res = gst_pad_stop_task (pad); |
| } |
| break; |
| default: |
| /* unknown scheduling mode */ |
| res = FALSE; |
| break; |
| } |
| return res; |
| } |
| <!-- example-end task.c i --></programlisting> |
| <para> |
| Once started, your task has full control over input and output. The |
| most simple case of a task function is one that reads input and pushes |
| that over its source pad. It's not all that useful, but provides some |
| more flexibility than the old push-mode case that we've been looking |
| at so far. |
| </para> |
| <programlisting><!-- example-begin task.c j --> |
| #define BLOCKSIZE 2048 |
| |
| static void |
| gst_my_filter_loop (GstMyFilter * filter) |
| { |
| GstFlowReturn ret; |
| guint64 len; |
| GstFormat fmt = GST_FORMAT_BYTES; |
| GstBuffer *buf = NULL; |
| |
| if (!gst_pad_query_duration (filter->sinkpad, fmt, &len)) { |
| GST_DEBUG_OBJECT (filter, "failed to query duration, pausing"); |
| goto stop; |
| } |
| |
| if (filter->offset >= len) { |
| GST_DEBUG_OBJECT (filter, "at end of input, sending EOS, pausing"); |
| gst_pad_push_event (filter->srcpad, gst_event_new_eos ()); |
| goto stop; |
| } |
| |
| /* now, read BLOCKSIZE bytes from byte offset filter->offset */ |
| ret = gst_pad_pull_range (filter->sinkpad, filter->offset, |
| BLOCKSIZE, &buf); |
| |
| if (ret != GST_FLOW_OK) { |
| GST_DEBUG_OBJECT (filter, "pull_range failed: %s", gst_flow_get_name (ret)); |
| goto stop; |
| } |
| |
| /* now push buffer downstream */ |
| ret = gst_pad_push (filter->srcpad, buf); |
| |
| buf = NULL; /* gst_pad_push() took ownership of buffer */ |
| |
| if (ret != GST_FLOW_OK) { |
| GST_DEBUG_OBJECT (filter, "pad_push failed: %s", gst_flow_get_name (ret)); |
| goto stop; |
| } |
| |
| /* everything is fine, increase offset and wait for us to be called again */ |
| filter->offset += BLOCKSIZE; |
| return; |
| |
| stop: |
| GST_DEBUG_OBJECT (filter, "pausing task"); |
| gst_pad_pause_task (filter->sinkpad); |
| } |
| <!-- example-end task.c j --> |
| <!-- example-begin task.c k --><!-- |
| #include "register.func" |
| --><!-- example-end task.c k --></programlisting> |
| </sect1> |
| |
| <sect1 id="section-scheduling-randomxs" xreflabel="Providing random access"> |
| <title>Providing random access</title> |
| <para> |
| In the previous section, we have talked about how elements (or pads) |
| that are activated to drive the pipeline using their own task, must use |
| pull-mode scheduling on their sinkpads. This means that all pads linked |
| to those pads need to be activated in pull-mode. |
| Source pads activated in pull-mode must implement a |
| <function>_get_range ()</function>-function set using |
| <function>gst_pad_set_getrange_function ()</function>, and |
| that function will be called when the peer pad requests some data with |
| <function>gst_pad_pull_range ()</function>. |
| The element is then responsible for seeking to the right offset and |
| providing the requested data. Several elements can implement random |
| access: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| Data sources, such as a file source, that can provide data from any |
| offset with reasonable low latency. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Filters that would like to provide a pull-mode scheduling |
| over the whole pipeline. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Parsers who can easily provide this by skipping a small part of |
| their input and are thus essentially "forwarding" getrange |
| requests literally without any own processing involved. Examples |
| include tag readers (e.g. ID3) or single output parsers, such as |
| a WAVE parser. |
| </para> |
| </listitem> |
| </itemizedlist> |
| <para> |
| The following example will show how a <function>_get_range |
| ()</function>-function can be implemented in a source element: |
| </para> |
| <programlisting><!-- example-begin range.c a --> |
| #include "filter.h" |
| static GstFlowReturn |
| gst_my_filter_get_range (GstPad * pad, |
| GstObject * parent, |
| guint64 offset, |
| guint length, |
| GstBuffer ** buf); |
| |
| G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT); |
| <!-- example-end range.c a --> |
| <!-- example-begin range.c b --><!-- |
| static void |
| gst_my_filter_class_init (gpointer klass) |
| { |
| GstElementClass *element_class = GST_ELEMENT_CLASS (klass); |
| static GstElementDetails my_filter_details = { |
| "An example plugin", |
| "Example/FirstExample", |
| "Shows the basic structure of a plugin", |
| "your name <your.name@your.isp>" |
| }; |
| static GstStaticPadTemplate src_factory = |
| GST_STATIC_PAD_TEMPLATE ( |
| "src", |
| GST_PAD_SRC, |
| GST_PAD_ALWAYS, |
| GST_STATIC_CAPS ("ANY") |
| ); |
| |
| gst_element_class_set_details (element_class, &my_filter_details); |
| gst_element_class_add_pad_template (element_class, |
| gst_static_pad_template_get (&src_factory)); |
| } |
| |
| static void |
| gst_my_filter_class_init (GstMyFilterClass * klass) |
| { |
| } |
| --><!-- example-begin range.c b --> |
| <!-- example-begin range.c c --> |
| static void |
| gst_my_filter_init (GstMyFilter * filter) |
| { |
| <!-- example-end task.c c --><!-- |
| GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter); |
| |
| filter->srcpad = gst_pad_new_from_template ( |
| gst_element_class_get_pad_template (klass, "src"), "src"); |
| --> |
| [..]<!-- example-begin task.c d --><!-- |
| --><!-- example-end task.c d --> |
| <!-- example-begin task.c e --> |
| gst_pad_set_getrange_function (filter->srcpad, |
| gst_my_filter_get_range); |
| <!-- example-end range.c c --><!-- |
| gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad); |
| --> |
| [..]<!-- example-begin range.c d --> |
| } |
| |
| static GstFlowReturn |
| gst_my_filter_get_range (GstPad * pad, |
| GstObject * parent, |
| guint64 offset, |
| guint length, |
| GstBuffer ** buf) |
| { |
| <!-- example-end range.c d --> |
| GstMyFilter *filter = GST_MY_FILTER (parent); |
| |
| [.. here, you would fill *buf ..] |
| <!-- example-begin range.c e --> |
| return GST_FLOW_OK; |
| } |
| <!-- example-end range.c e --> |
| <!-- example-begin range.c f --><!-- |
| #include "register.func" |
| --><!-- example-end range.c f --></programlisting> |
| <para> |
| In practice, many elements that could theoretically do random access, |
| may in practice often be activated in push-mode scheduling anyway, |
| since there is no downstream element able to start its own task. |
| Therefore, in practice, those elements should implement both a |
| <function>_get_range ()</function>-function and a <function>_chain |
| ()</function>-function (for filters and parsers) or a <function>_get_range |
| ()</function>-function and be prepared to start their own task by |
| providing <function>_activate_* ()</function>-functions (for |
| source elements). |
| </para> |
| </sect1> |
| </chapter> |