| <chapter id="chapter-elements" xreflabel="Elements"> |
| <title>Elements</title> |
| <para> |
| The most important object in &GStreamer; for the application programmer |
| is the <ulink type="http" |
| url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink> |
| object. An element is the basic building block for a media pipeline. All |
| the different high-level components you will use are derived from |
| <classname>GstElement</classname>. Every decoder, encoder, demuxer, video |
| or audio output is in fact a <classname>GstElement</classname> |
| </para> |
| |
| <sect1 id="section-elements-design" xreflabel="What are elements?"> |
| <title>What are elements?</title> |
| <para> |
| For the application programmer, elements are best visualized as black |
| boxes. On the one end, you might put something in, the element does |
| something with it and something else comes out at the other side. For |
| a decoder element, for example, you'd put in encoded data, and the |
| element would output decoded data. In the next chapter (see <xref |
| linkend="chapter-pads"/>), you will learn more about data input and |
| output in elements, and how you can set that up in your application. |
| </para> |
| |
| <sect2 id="section-elements-src"> |
| <title>Source elements</title> |
| <para> |
| Source elements generate data for use by a pipeline, for example |
| reading from disk or from a sound card. <xref |
| linkend="section-element-srcimg"/> shows how we will visualise |
| a source element. We always draw a source pad to the right of |
| the element. |
| </para> |
| <figure float="1" id="section-element-srcimg"> |
| <title>Visualisation of a source element</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata scale="75" fileref="images/src-element.ℑ" |
| format="&IMAGE;"/> |
| </imageobject> |
| </mediaobject> |
| </figure> |
| <para> |
| Source elements do not accept data, they only generate data. You can |
| see this in the figure because it only has a source pad (on the |
| right). A source pad can only generate data. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-elements-filter"> |
| <title>Filters, convertors, demuxers, muxers and codecs</title> |
| <para> |
| Filters and filter-like elements have both input and outputs pads. |
| They operate on data that they receive on their input (sink) pads, |
| and will provide data on their output (source) pads. Examples of |
| such elements are a volume element (filter), a video scaler |
| (convertor), an Ogg demuxer or a Vorbis decoder. |
| </para> |
| <para> |
| Filter-like elements can have any number of source or sink pads. A |
| video demuxer, for example, would have one sink pad and several |
| (1-N) source pads, one for each elementary stream contained in the |
| container format. Decoders, on the other hand, will only have one |
| source and sink pads. |
| </para> |
| <figure float="1" id="section-element-filterimg"> |
| <title>Visualisation of a filter element</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata scale="75" fileref="images/filter-element.ℑ" |
| format="&IMAGE;"/> |
| </imageobject> |
| </mediaobject> |
| </figure> |
| <para> |
| <xref linkend="section-element-filterimg"/> shows how we will |
| visualise a filter-like element. This specific element has one source |
| and one sink element. Sink pads, receiving input data, are depicted |
| at the left of the element; source pads are still on the right. |
| </para> |
| <figure float="1" id="section-element-multifilterimg"> |
| <title>Visualisation of a filter element with |
| more than one output pad</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata scale="75" fileref="images/filter-element-multi.ℑ" |
| format="&IMAGE;" /> |
| </imageobject> |
| </mediaobject> |
| </figure> |
| <para> |
| <xref linkend="section-element-multifilterimg"/> shows another |
| filter-like element, this one having more than one output (source) |
| pad. An example of one such element could, for example, be an Ogg |
| demuxer for an Ogg stream containing both audio and video. One |
| source pad will contain the elementary video stream, another will |
| contain the elementary audio stream. Demuxers will generally fire |
| signals when a new pad is created. The application programmer can |
| then handle the new elementary stream in the signal handler. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-elements-sink"> |
| <title>Sink elements</title> |
| <para> |
| Sink elements are end points in a media pipeline. They accept |
| data but do not produce anything. Disk writing, soundcard playback, |
| and video output would all be implemented by sink elements. |
| <xref linkend="section-element-sinkimg"/> shows a sink element. |
| </para> |
| <figure float="1" id="section-element-sinkimg"> |
| <title>Visualisation of a sink element</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata scale="75" fileref="images/sink-element.ℑ" |
| format="&IMAGE;" /> |
| </imageobject> |
| </mediaobject> |
| </figure> |
| </sect2> |
| </sect1> |
| |
| <sect1 id="section-elements-create"> |
| <title>Creating a <classname>GstElement</classname></title> |
| <para> |
| The simplest way to create an element is to use <ulink type="http" |
| url="&URLAPI;GstElementFactory.html#gst-element-factory-make"><function>gst_element_factory_make |
| ()</function></ulink>. This function takes a factory name and an |
| element name for the newly created element. The name of the element |
| is something you can use later on to look up the element in a bin, |
| for example. The name will also be used in debug output. You can |
| pass <symbol>NULL</symbol> as the name argument to get a unique, |
| default name. |
| </para> |
| <para> |
| When you don't need the element anymore, you need to unref it using |
| <ulink type="http" |
| url="&URLAPI;GstObject.html#gst-object-unref"><function>gst_object_unref |
| ()</function></ulink>. This decreases the reference count for the |
| element by 1. An element has a refcount of 1 when it gets created. |
| An element gets destroyed completely when the refcount is decreased |
| to 0. |
| </para> |
| <para> |
| The following example &EXAFOOT; shows how to create an element named |
| <emphasis>source</emphasis> from the element factory named |
| <emphasis>fakesrc</emphasis>. It checks if the creation succeeded. |
| After checking, it unrefs the element. |
| </para> |
| <programlisting><!-- example-begin elementmake.c --><![CDATA[ |
| #include <gst/gst.h> |
| |
| int |
| main (int argc, |
| char *argv[]) |
| { |
| GstElement *element; |
| |
| /* init GStreamer */ |
| gst_init (&argc, &argv); |
| |
| /* create element */ |
| element = gst_element_factory_make ("fakesrc", "source"); |
| if (!element) { |
| g_print ("Failed to create element of type 'fakesrc'\n"); |
| return -1; |
| } |
| |
| gst_object_unref (GST_OBJECT (element)); |
| |
| return 0; |
| } |
| ]]><!-- example-end elementmake.c --></programlisting> |
| <para> |
| <function>gst_element_factory_make</function> is actually a shorthand |
| for a combination of two functions. A <ulink type="http" |
| url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink> |
| object is created from a factory. To create the element, you have to |
| get access to a <ulink type="http" |
| url="&URLAPI;GstElementFactory.html"><classname>GstElementFactory</classname></ulink> |
| object using a unique factory name. This is done with <ulink type="http" |
| url="&URLAPI;GstElementFactory.html#gst-element-factory-find"><function>gst_element_factory_find |
| ()</function></ulink>. |
| </para> |
| <para> |
| The following code fragment is used to get a factory that can be used |
| to create the <emphasis>fakesrc</emphasis> element, a fake data source. |
| The function <ulink type="http" |
| url="&URLAPI;GstElementFactory.html#gst-element-factory-create"><function>gst_element_factory_create |
| ()</function></ulink> will use the element factory to create an |
| element with the given name. |
| </para> |
| <programlisting><!-- example-begin elementcreate.c --><![CDATA[ |
| #include <gst/gst.h> |
| |
| int |
| main (int argc, |
| char *argv[]) |
| { |
| GstElementFactory *factory; |
| GstElement * element; |
| |
| /* init GStreamer */ |
| gst_init (&argc, &argv); |
| |
| /* create element, method #2 */ |
| factory = gst_element_factory_find ("fakesrc"); |
| if (!factory) { |
| g_print ("Failed to find factory of type 'fakesrc'\n"); |
| return -1; |
| } |
| element = gst_element_factory_create (factory, "source"); |
| if (!element) { |
| g_print ("Failed to create element, even though its factory exists!\n"); |
| return -1; |
| } |
| |
| gst_object_unref (GST_OBJECT (element)); |
| |
| return 0; |
| } |
| ]]><!-- example-end elementcreate.c --></programlisting> |
| </sect1> |
| |
| <sect1 id="section-elements-properties"> |
| <title>Using an element as a <classname>GObject</classname></title> |
| <para> |
| A <ulink type="http" |
| url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink> |
| can have several properties which are implemented using standard |
| <classname>GObject</classname> properties. The usual |
| <classname>GObject</classname> methods to query, set and get |
| property values and <classname>GParamSpecs</classname> are |
| therefore supported. |
| </para> |
| <para> |
| Every <classname>GstElement</classname> inherits at least one |
| property from its parent <classname>GstObject</classname>: the |
| "name" property. This is the name you provide to the functions |
| <function>gst_element_factory_make ()</function> or |
| <function>gst_element_factory_create ()</function>. You can get |
| and set this property using the functions |
| <function>gst_object_set_name</function> and |
| <function>gst_object_get_name</function> or use the |
| <classname>GObject</classname> property mechanism as shown below. |
| </para> |
| <programlisting><!-- example-begin elementget.c --><![CDATA[ |
| #include <gst/gst.h> |
| |
| int |
| main (int argc, |
| char *argv[]) |
| { |
| GstElement *element; |
| gchar *name; |
| |
| /* init GStreamer */ |
| gst_init (&argc, &argv); |
| |
| /* create element */ |
| element = gst_element_factory_make ("fakesrc", "source"); |
| |
| /* get name */ |
| g_object_get (G_OBJECT (element), "name", &name, NULL); |
| g_print ("The name of the element is '%s'.\n", name); |
| g_free (name); |
| |
| gst_object_unref (GST_OBJECT (element)); |
| |
| return 0; |
| } |
| ]]><!-- example-end elementget.c --></programlisting> |
| <para> |
| Most plugins provide additional properties to provide more information |
| about their configuration or to configure the element. |
| <command>gst-inspect</command> is a useful tool to query the properties |
| of a particular element, it will also use property introspection to give |
| a short explanation about the function of the property and about the |
| parameter types and ranges it supports. See |
| <xref linkend="section-applications-inspect"/> |
| in the appendix for details about <command>gst-inspect</command>. |
| </para> |
| <para> |
| For more information about <classname>GObject</classname> |
| properties we recommend you read the <ulink |
| url="http://developer.gnome.org/gobject/stable/rn01.html" |
| type="http">GObject manual</ulink> and an introduction to <ulink |
| url="http://developer.gnome.org/gobject/stable/pt01.html" type="http"> |
| The Glib Object system</ulink>. |
| </para> |
| <para> |
| A <ulink type="http" url="&URLAPI;GstElement.html"> |
| <classname>GstElement</classname></ulink> also provides various |
| <classname>GObject</classname> signals that can be used as a flexible |
| callback mechanism. Here, too, you can use <command>gst-inspect</command> |
| to see which signals a specific element supports. Together, signals |
| and properties are the most basic way in which elements and |
| applications interact. |
| </para> |
| </sect1> |
| |
| <sect1 id="section-elements-factories"> |
| <title>More about element factories</title> |
| <para> |
| In the previous section, we briefly introduced the <ulink type="http" |
| url="&URLAPI;GstElementFactory.html"><classname>GstElementFactory</classname></ulink> |
| object already as a way to create instances of an element. Element |
| factories, however, are much more than just that. Element factories |
| are the basic types retrieved from the &GStreamer; registry, they |
| describe all plugins and elements that &GStreamer; can create. This |
| means that element factories are useful for automated element |
| instancing, such as what autopluggers do, and for creating lists |
| of available elements. |
| </para> |
| |
| <sect2 id="section-elements-factories-details"> |
| <title>Getting information about an element using a factory</title> |
| <para> |
| Tools like <command>gst-inspect</command> will provide some generic |
| information about an element, such as the person that wrote the |
| plugin, a descriptive name (and a shortname), a rank and a category. |
| The category can be used to get the type of the element that can |
| be created using this element factory. Examples of categories include |
| <classname>Codec/Decoder/Video</classname> (video decoder), |
| <classname>Codec/Encoder/Video</classname> (video encoder), |
| <classname>Source/Video</classname> (a video generator), |
| <classname>Sink/Video</classname> (a video output), and all these |
| exist for audio as well, of course. Then, there's also |
| <classname>Codec/Demuxer</classname> and |
| <classname>Codec/Muxer</classname> and a whole lot more. |
| <command>gst-inspect</command> will give a list of all factories, and |
| <command>gst-inspect <factory-name></command> will list all |
| of the above information, and a lot more. |
| </para> |
| <programlisting><!-- example-begin elementfactory.c --><![CDATA[ |
| #include <gst/gst.h> |
| |
| int |
| main (int argc, |
| char *argv[]) |
| { |
| GstElementFactory *factory; |
| |
| /* init GStreamer */ |
| gst_init (&argc, &argv); |
| |
| /* get factory */ |
| factory = gst_element_factory_find ("fakesrc"); |
| if (!factory) { |
| g_print ("You don't have the 'fakesrc' element installed!\n"); |
| return -1; |
| } |
| |
| /* display information */ |
| g_print ("The '%s' element is a member of the category %s.\n" |
| "Description: %s\n", |
| gst_plugin_feature_get_name (GST_PLUGIN_FEATURE (factory)), |
| gst_element_factory_get_metadata (factory, GST_ELEMENT_METADATA_KLASS), |
| gst_element_factory_get_metadata (factory, GST_ELEMENT_METADATA_DESCRIPTION)); |
| |
| return 0; |
| } |
| ]]><!-- example-end elementfactory.c --></programlisting> |
| <para> |
| You can use <function>gst_registry_pool_feature_list (GST_TYPE_ELEMENT_FACTORY)</function> |
| to get a list of all the element factories that &GStreamer; knows |
| about. |
| </para> |
| </sect2> |
| |
| <sect2 id="section-elements-factories-padtemplates"> |
| <title>Finding out what pads an element can contain</title> |
| <para> |
| Perhaps the most powerful feature of element factories is that |
| they contain a full description of the pads that the element |
| can generate, and the capabilities of those pads (in layman words: |
| what types of media can stream over those pads), without actually |
| having to load those plugins into memory. This can be used |
| to provide a codec selection list for encoders, or it can be used |
| for autoplugging purposes for media players. All current |
| &GStreamer;-based media players and autopluggers work this way. |
| We'll look closer at these features as we learn about |
| <classname>GstPad</classname> and <classname>GstCaps</classname> |
| in the next chapter: <xref linkend="chapter-pads"/> |
| </para> |
| </sect2> |
| </sect1> |
| |
| <sect1 id="section-elements-link" xreflabel="Linking elements"> |
| <title>Linking elements</title> |
| <para> |
| By linking a source element with zero or more filter-like |
| elements and finally a sink element, you set up a media |
| pipeline. Data will flow through the elements. This is the |
| basic concept of media handling in &GStreamer;. |
| </para> |
| <figure float="1" id="section-link"> |
| <title>Visualisation of three linked elements</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata scale="75" fileref="images/linked-elements.ℑ" |
| format="&IMAGE;"/> |
| </imageobject> |
| </mediaobject> |
| </figure> |
| <para> |
| By linking these three elements, we have created a very simple |
| chain of elements. The effect of this will be that the output of |
| the source element (<quote>element1</quote>) will be used as input |
| for the filter-like element (<quote>element2</quote>). The |
| filter-like element will do something with the data and send the |
| result to the final sink element (<quote>element3</quote>). |
| </para> |
| <para> |
| Imagine the above graph as a simple Ogg/Vorbis audio decoder. The |
| source is a disk source which reads the file from disc. The second |
| element is a Ogg/Vorbis audio decoder. The sink element is your |
| soundcard, playing back the decoded audio data. We will use this |
| simple graph to construct an Ogg/Vorbis player later in this manual. |
| </para> |
| <para> |
| In code, the above graph is written like this: |
| </para> |
| <programlisting><!-- example-begin elementlink.c a --> |
| #include <gst/gst.h> |
| |
| int |
| main (int argc, |
| char *argv[]) |
| { |
| GstElement *pipeline; |
| GstElement *source, *filter, *sink; |
| |
| /* init */ |
| gst_init (&argc, &argv); |
| |
| /* create pipeline */ |
| pipeline = gst_pipeline_new ("my-pipeline"); |
| |
| /* create elements */ |
| source = gst_element_factory_make ("fakesrc", "source"); |
| filter = gst_element_factory_make ("identity", "filter"); |
| sink = gst_element_factory_make ("fakesink", "sink"); |
| |
| /* must add elements to pipeline before linking them */ |
| gst_bin_add_many (GST_BIN (pipeline), source, filter, sink, NULL); |
| |
| /* link */ |
| if (!gst_element_link_many (source, filter, sink, NULL)) { |
| g_warning ("Failed to link elements!"); |
| } |
| <!-- example-end elementlink.c a --> |
| [..]<!-- example-begin elementlink.c b --><!-- |
| return 0; |
| --><!-- example-end elementlink.c b --> |
| <!-- example-begin elementlink.c c --> |
| } |
| <!-- example-end elementlink.c c --></programlisting> |
| <para> |
| For more specific behaviour, there are also the functions |
| <function>gst_element_link ()</function> and |
| <function>gst_element_link_pads ()</function>. You can also obtain |
| references to individual pads and link those using various |
| <function>gst_pad_link_* ()</function> functions. See the API |
| references for more details. |
| </para> |
| <para> |
| Important: you must add elements to a bin or pipeline |
| <emphasis>before</emphasis> linking them, since adding an element to |
| a bin will disconnect any already existing links. Also, you cannot |
| directly link elements that are not in the same bin or pipeline; if |
| you want to link elements or pads at different hierarchy levels, you |
| will need to use ghost pads (more about ghost pads later, |
| see <xref linkend="section-pads-ghost"/>). |
| </para> |
| </sect1> |
| |
| <sect1 id="section-elements-states"> |
| <title>Element States</title> |
| <para> |
| After being created, an element will not actually perform any actions |
| yet. You need to change elements state to make it do something. |
| &GStreamer; knows four element states, each with a very specific |
| meaning. Those four states are: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| <classname>GST_STATE_NULL</classname>: this is the default state. |
| No resources are allocated in this state, so, transitioning to it |
| will free all resources. The element must be in this state when |
| its refcount reaches 0 and it is freed. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| <classname>GST_STATE_READY</classname>: in the ready state, an |
| element has allocated all of its global resources, that is, |
| resources that can be kept within streams. You can think about |
| opening devices, allocating buffers and so on. However, the |
| stream is not opened in this state, so the stream positions is |
| automatically zero. If a stream was previously opened, it should |
| be closed in this state, and position, properties and such should |
| be reset. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| <classname>GST_STATE_PAUSED</classname>: in this state, an |
| element has opened the stream, but is not actively processing |
| it. An element is allowed to modify a stream's position, read |
| and process data and such to prepare for playback as soon as |
| state is changed to PLAYING, but it is <emphasis>not</emphasis> |
| allowed to play the data which would make the clock run. |
| In summary, PAUSED is the same as PLAYING but without a running |
| clock. |
| </para> |
| <para> |
| Elements going into the PAUSED state should prepare themselves |
| for moving over to the PLAYING state as soon as possible. Video |
| or audio outputs would, for example, wait for data to arrive and |
| queue it so they can play it right after the state change. Also, |
| video sinks can already play the first frame (since this does |
| not affect the clock yet). Autopluggers could use this same |
| state transition to already plug together a pipeline. Most other |
| elements, such as codecs or filters, do not need to explicitly |
| do anything in this state, however. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| <classname>GST_STATE_PLAYING</classname>: in the PLAYING state, |
| an element does exactly the same as in the PAUSED state, except |
| that the clock now runs. |
| </para> |
| </listitem> |
| </itemizedlist> |
| <para> |
| You can change the state of an element using the function |
| <function>gst_element_set_state ()</function>. If you set an element |
| to another state, &GStreamer; will internally traverse all intermediate |
| states. So if you set an element from NULL to PLAYING, &GStreamer; |
| will internally set the element to READY and PAUSED in between. |
| </para> |
| <para> |
| When moved to <classname>GST_STATE_PLAYING</classname>, pipelines |
| will process data automatically. They do not need to be iterated in |
| any form. Internally, &GStreamer; will start threads that take this |
| task on to them. &GStreamer; will also take care of switching |
| messages from the pipeline's thread into the application's own |
| thread, by using a <ulink type="http" |
| url="&URLAPI;GstBus.html"><classname>GstBus</classname></ulink>. See |
| <xref linkend="chapter-bus"/> for details. |
| </para> |
| <para> |
| When you set a bin or pipeline to a certain target state, it will usually |
| propagate the state change to all elements within the bin or pipeline |
| automatically, so it's usually only necessary to set the state of the |
| top-level pipeline to start up the pipeline or shut it down. However, |
| when adding elements dynamically to an already-running pipeline, e.g. |
| from within a "pad-added" signal callback, you |
| need to set it to the desired target state yourself using |
| <function>gst_element_set_state ()</function> or |
| <function>gst_element_sync_state_with_parent ()</function>. |
| </para> |
| </sect1> |
| </chapter> |
