| <chapter id="chapter-pads" xreflabel="Pads and capabilities"> |
| <title>Pads and capabilities</title> |
| <para> |
| As we have seen in <xref linkend="chapter-elements"/>, the pads are |
| the element's interface to the outside world. Data streams from one |
| element's source pad to another element's sink pad. The specific |
| type of media that the element can handle will be exposed by the |
| pad's capabilities. We will talk more on capabilities later in this |
| chapter (see <xref linkend="section-caps"/>). |
| </para> |
| |
| <sect1 id="section-pads"> |
| <title>Pads</title> |
| <para> |
| A pad type is defined by two properties: its direction and its |
| availability. As we've mentioned before, &GStreamer; defines two |
| pad directions: source pads and sink pads. This terminology is |
| defined from the view of within the element: elements receive data |
| on their sink pads and generate data on their source pads. |
| Schematically, sink pads are drawn on the left side of an element, |
| whereas source pads are drawn on the right side of an element. In |
| such graphs, data flows from left to right. |
| <footnote> |
| <para> |
| In reality, there is no objection to data flowing from a |
| source pad to the sink pad of an element upstream (to the |
| left of this element in drawings). Data will, however, always |
| flow from a source pad of one element to the sink pad of |
| another. |
| </para> |
| </footnote> |
| </para> |
| |
| <para> |
| Pad directions are very simple compared to pad availability. A pad |
| can have any of three availabilities: always, sometimes and on |
| request. The meaning of those three types is exactly as it says: |
| always pads always exist, sometimes pad exist only in certain |
| cases (and can disappear randomly), and on-request pads appear |
| only if explicitely requested by applications. |
| </para> |
| |
| <sect2 id="section-pads-dynamic"> |
| <title>Dynamic (or sometimes) pads</title> |
| <para> |
| Some elements might not have all of their pads when the element is |
| created. This can happen, for example, with an Ogg demuxer element. |
| The element will read the Ogg stream and create dynamic pads for |
| each contained elementary stream (vorbis, theora) when it detects |
| such a stream in the Ogg stream. Likewise, it will delete the pad |
| when the stream ends. This principle is very useful for demuxer |
| elements, for example. |
| </para> |
| <para> |
| Running <application>gst-inspect oggdemux</application> will show |
| that the element has only one pad: a sink pad called 'sink'. The |
| other pads are <quote>dormant</quote>. You can see this in the pad |
| template because there is an <quote>Exists: Sometimes</quote> |
| property. Depending on the type of Ogg file you play, the pads will |
| be created. We will see that this is very important when you are |
| going to create dynamic pipelines. You can attach a signal handler |
| to an element to inform you when the element has created a new pad |
| from one of its <quote>sometimes</quote> pad templates. The |
| following piece of code is an example of how to do this: |
| </para> |
| <programlisting><!-- example-begin pad.c a --> |
| #include <gst/gst.h> |
| |
| static void |
| cb_new_pad (GstElement *element, |
| GstPad *pad, |
| gpointer data) |
| { |
| gchar *name; |
| |
| name = gst_pad_get_name (pad); |
| g_print ("A new pad %s was created\n", name); |
| g_free (name); |
| |
| /* here, you would setup a new pad link for the newly created pad */ |
| <!-- example-end pad.c a -->[..] |
| <!-- example-begin pad.c b --> |
| } |
| |
| int |
| main (int argc, |
| char *argv[]) |
| { |
| GstElement *pipeline, *source, *demux; |
| GMainLoop *loop; |
| |
| /* init */ |
| gst_init (&argc, &argv); |
| |
| /* create elements */ |
| pipeline = gst_pipeline_new ("my_pipeline"); |
| source = gst_element_factory_make ("filesrc", "source"); |
| g_object_set (source, "location", argv[1], NULL); |
| demux = gst_element_factory_make ("oggdemux", "demuxer"); |
| |
| /* you would normally check that the elements were created properly */ |
| |
| /* put together a pipeline */ |
| gst_bin_add_many (GST_BIN (pipeline), source, demux, NULL); |
| gst_element_link_pads (source, "src", demux, "sink"); |
| |
| /* listen for newly created pads */ |
| g_signal_connect (demux, "pad-added", G_CALLBACK (cb_new_pad), NULL); |
| |
| /* start the pipeline */ |
| gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING); |
| loop = g_main_loop_new (NULL, FALSE); |
| g_main_loop_run (loop); |
| <!--example-end pad.c b --> |
| [..]<!-- example-begin pad.c c --><!-- |
| return 0; |
| --><!-- example-end pad.c c --> |
| <!-- example-begin pad.c d --> |
| } |
| <!-- example-end pad.c d --></programlisting> |
| </sect2> |
| |
| <sect2 id="section-pads-request"> |
| <title>Request pads</title> |
| <para> |
| An element can also have request pads. These pads are not created |
| automatically but are only created on demand. This is very useful |
| for multiplexers, aggregators and tee elements. Aggregators are |
| elements that merge the content of several input streams together |
| into one output stream. Tee elements are the reverse: they are |
| elements that have one input stream and copy this stream to each |
| of their output pads, which are created on request. Whenever an |
| application needs another copy of the stream, it can simply request |
| a new output pad from the tee element. |
| </para> |
| <para> |
| The following piece of code shows how you can request a new output |
| pad from a <quote>tee</quote> element: |
| </para> |
| <programlisting> |
| static void |
| some_function (GstElement *tee) |
| { |
| GstPad * pad; |
| gchar *name; |
| |
| pad = gst_element_get_request_pad (tee, "src%d"); |
| name = gst_pad_get_name (pad); |
| g_print ("A new pad %s was created\n", name); |
| g_free (name); |
| |
| /* here, you would link the pad */ |
| [..] |
| |
| /* and, after doing that, free our reference */ |
| gst_object_unref (GST_OBJECT (pad)); |
| } |
| </programlisting> |
| <para> |
| The <function>gst_element_get_request_pad ()</function> method |
| can be used to get a pad from the element based on the name of |
| the pad template. It is also possible to request a pad that is |
| compatible with another pad template. This is very useful if |
| you want to link an element to a multiplexer element and you |
| need to request a pad that is compatible. The method |
| <function>gst_element_get_compatible_pad ()</function> can be |
| used to request a compatible pad, as shown in the next example. |
| It will request a compatible pad from an Ogg multiplexer from |
| any input. |
| </para> |
| <programlisting> |
| static void |
| link_to_multiplexer (GstPad *tolink_pad, |
| GstElement *mux) |
| { |
| GstPad *pad; |
| gchar *srcname, *sinkname; |
| |
| srcname = gst_pad_get_name (tolink_pad); |
| pad = gst_element_get_compatible_pad (mux, tolink_pad); |
| gst_pad_link (tolinkpad, pad); |
| sinkname = gst_pad_get_name (pad); |
| gst_object_unref (GST_OBJECT (pad)); |
| |
| g_print ("A new pad %s was created and linked to %s\n", srcname, sinkname); |
| g_free (sinkname); |
| g_free (srcname); |
| } |
| </programlisting> |
| </sect2> |
| </sect1> |
| |
| <sect1 id="section-caps"> |
| <title>Capabilities of a pad</title> |
| <para> |
| Since the pads play a very important role in how the element is |
| viewed by the outside world, a mechanism is implemented to describe |
| the data that can flow or currently flows through the pad by using |
| capabilities. Here, we will briefly describe what capabilities are |
| and how to use them, enough to get an understanding of the concept. |
| For an in-depth look into capabilities and a list of all capabilities |
| defined in &GStreamer;, see the <ulink type="http" |
| url="http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/index.html">Plugin |
| Writers Guide</ulink>. |
| </para> |
| <para> |
| Capabilities are attached to pad templates and to pads. For pad |
| templates, it will describe the types of media that may stream |
| over a pad created from this template. For pads, it can either |
| be a list of possible caps (usually a copy of the pad template's |
| capabilities), in which case the pad is not yet negotiated, or it |
| is the type of media that currently streams over this pad, in |
| which case the pad has been negotiated already. |
| </para> |
| |
| <sect2 id="section-caps-structure"> |
| <title>Dissecting capabilities</title> |
| <para> |
| A pads capabilities are described in a <classname>GstCaps</classname> |
| object. Internally, a <ulink type="http" |
| url="../../gstreamer/html/gstreamer-GstCaps.html"><classname>GstCaps</classname></ulink> |
| will contain one or more <ulink type="http" |
| url="../../gstreamer/html/gstreamer-GstStructure.html"><classname>GstStructure</classname></ulink> |
| that will describe one media type. A negotiated pad will have |
| capabilities set that contain exactly <emphasis>one</emphasis> |
| structure. Also, this structure will contain only |
| <emphasis>fixed</emphasis> values. These constraints are not |
| true for unnegotiated pads or pad templates. |
| </para> |
| <para> |
| As an example, below is a dump of the capabilities of the |
| <quote>vorbisdec</quote> element, which you will get by running |
| <command>gst-inspect vorbisdec</command>. You will see two pads: |
| a source and a sink pad. Both of these pads are always available, |
| and both have capabilities attached to them. The sink pad will |
| accept vorbis-encoded audio data, with the mime-type |
| <quote>audio/x-vorbis</quote>. The source pad will be used |
| to send raw (decoded) audio samples to the next element, with |
| a raw audio mime-type (in this case, |
| <quote>audio/x-raw-int</quote>) The source pad will also |
| contain properties for the audio samplerate and the amount of |
| channels, plus some more that you don't need to worry about |
| for now. |
| </para> |
| <programlisting> |
| Pad Templates: |
| SRC template: 'src' |
| Availability: Always |
| Capabilities: |
| audio/x-raw-float |
| rate: [ 8000, 50000 ] |
| channels: [ 1, 2 ] |
| endianness: 1234 |
| width: 32 |
| buffer-frames: 0 |
| |
| SINK template: 'sink' |
| Availability: Always |
| Capabilities: |
| audio/x-vorbis |
| </programlisting> |
| </sect2> |
| |
| <sect2 id="section-caps-props"> |
| <title>Properties and values</title> |
| <para> |
| Properties are used to describe extra information for |
| capabilities. A property consists of a key (a string) and |
| a value. There are different possible value types that can be used: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| Basic types, this can be pretty much any |
| <classname>GType</classname> registered with Glib. Those |
| properties indicate a specific, non-dynamic value for this |
| property. Examples include: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| An integer value (<classname>G_TYPE_INT</classname>): |
| the property has this exact value. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| A boolean value (<classname>G_TYPE_BOOLEAN</classname>): |
| the property is either TRUE or FALSE. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| A float value (<classname>G_TYPE_FLOAT</classname>): |
| the property has this exact floating point value. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| A string value (<classname>G_TYPE_STRING</classname>): |
| the property contains a UTF-8 string. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| A fraction value (<classname>GST_TYPE_FRACTION</classname>): |
| contains a fraction expressed by an integer numerator and |
| denominator. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <para> |
| Range types are <classname>GType</classname>s registered by |
| &GStreamer; to indicate a range of possible values. They are |
| used for indicating allowed audio samplerate values or |
| supported video sizes. The two types defined in &GStreamer; |
| are: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| An integer range value |
| (<classname>GST_TYPE_INT_RANGE</classname>): the property |
| denotes a range of possible integers, with a lower and an |
| upper boundary. The <quote>vorbisdec</quote> element, for |
| example, has a rate property that can be between 8000 and |
| 50000. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| A float range value |
| (<classname>GST_TYPE_FLOAT_RANGE</classname>): the property |
| denotes a range of possible floating point values, with a |
| lower and an upper boundary. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| A fraction range value |
| (<classname>GST_TYPE_FRACTION_RANGE</classname>): the property |
| denotes a range of possible fraction values, with a |
| lower and an upper boundary. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </listitem> |
| <listitem> |
| <para> |
| A list value (<classname>GST_TYPE_LIST</classname>): the |
| property can take any value from a list of basic values |
| given in this list. |
| </para> |
| <para> |
| Example: caps that express that either |
| a sample rate of 44100 Hz and a sample rate of 48000 Hz |
| is supported would use a list of integer values, with |
| one value being 44100 and one value being 48000. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| An array value (<classname>GST_TYPE_ARRAY</classname>): the |
| property is an array of values. Each value in the array is a |
| full value on its own, too. All values in the array should be |
| of the same elementary type. This means that an array can |
| contain any combination of integers, lists of integers, integer |
| ranges together, and the same for floats or strings, but it can |
| not contain both floats and ints at the same time. |
| </para> |
| <para> |
| Example: for audio where there are more than two channels involved |
| the channel layout needs to be specified (for one and two channel |
| audio the channel layout is implicit unless stated otherwise in the |
| caps). So the channel layout would be an array of integer enum |
| values where each enum value represents a loudspeaker position. |
| Unlike a <classname>GST_TYPE_LIST</classname>, the values in an |
| array will be interpreted as a whole. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </sect2> |
| </sect1> |
| |
| <sect1 id="section-caps-api"> |
| <title>What capabilities are used for</title> |
| <para> |
| Capabilities (short: caps) describe the type of data that is streamed |
| between two pads, or that one pad (template) supports. This makes them |
| very useful for various purposes: |
| </para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| Autoplugging: automatically finding elements to link to a |
| pad based on its capabilities. All autopluggers use this |
| method. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Compatibility detection: when two pads are linked, &GStreamer; |
| can verify if the two pads are talking about the same media |
| type. The process of linking two pads and checking if they |
| are compatible is called <quote>caps negotiation</quote>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Metadata: by reading the capabilities from a pad, applications |
| can provide information about the type of media that is being |
| streamed over the pad, which is information about the stream |
| that is currently being played back. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Filtering: an application can use capabilities to limit the |
| possible media types that can stream between two pads to a |
| specific subset of their supported stream types. An application |
| can, for example, use <quote>filtered caps</quote> to set a |
| specific (fixed or non-fixed) video size that should stream |
| between two pads. You will see an example of filtered caps |
| later in this manual, in <xref linkend="section-data-spoof"/>. |
| You can do caps filtering by inserting a capsfilter element into |
| your pipeline and setting its <quote>caps</quote> property. Caps |
| filters are often placed after converter elements like audioconvert, |
| audioresample, ffmpegcolorspace or videoscale to force those |
| converters to convert data to a specific output format at a |
| certain point in a stream. |
| </para> |
| </listitem> |
| </itemizedlist> |
| |
| <sect2 id="section-caps-metadata"> |
| <title>Using capabilities for metadata</title> |
| <para> |
| A pad can have a set (i.e. one or more) of capabilities attached |
| to it. Capabilities (<classname>GstCaps</classname>) are represented |
| as an array of one or more <classname>GstStructure</classname>s, and |
| each <classname>GstStructure</classname> is an array of fields where |
| each field consists of a field name string (e.g. "width") and a |
| typed value (e.g. <classname>G_TYPE_INT</classname> or |
| <classname>GST_TYPE_INT_RANGE</classname>). |
| </para> |
| <para> |
| Note that there is a distinct difference between the |
| <emphasis>possible</emphasis> capabilities of a pad (ie. usually what |
| you find as caps of pad templates as they are shown in gst-inspect), |
| the <emphasis>allowed</emphasis> caps of a pad (can be the same as |
| the pad's template caps or a subset of them, depending on the possible |
| caps of the peer pad) and lastly <emphasis>negotiated</emphasis> caps |
| (these describe the exact format of a stream or buffer and contain |
| exactly one structure and have no variable bits like ranges or lists, |
| ie. they are fixed caps). |
| </para> |
| <para> |
| You can get values of properties in a set of capabilities |
| by querying individual properties of one structure. You can get |
| a structure from a caps using |
| <function>gst_caps_get_structure ()</function> and the number of |
| structures in a <classname>GstCaps</classname> using |
| <function>gst_caps_get_size ()</function>. |
| </para> |
| <para> |
| Caps are called <emphasis>simple caps</emphasis> when they contain |
| only one structure, and <emphasis>fixed caps</emphasis> when they |
| contain only one structure and have no variable field types (like |
| ranges or lists of possible values). Two other special types of caps |
| are <emphasis>ANY caps</emphasis> and <emphasis>empty caps</emphasis>. |
| </para> |
| <para> |
| Here is an example of how to extract the width and height from |
| a set of fixed video caps: |
| <programlisting> |
| static void |
| read_video_props (GstCaps *caps) |
| { |
| gint width, height; |
| const GstStructure *str; |
| |
| g_return_if_fail (gst_caps_is_fixed (caps)); |
| |
| str = gst_caps_get_structure (caps, 0); |
| if (!gst_structure_get_int (str, "width", &width) || |
| !gst_structure_get_int (str, "height", &height)) { |
| g_print ("No width/height available\n"); |
| return; |
| } |
| |
| g_print ("The video size of this set of capabilities is %dx%d\n", |
| width, height); |
| } |
| </programlisting> |
| </para> |
| </sect2> |
| |
| <sect2 id="section-caps-filter"> |
| <title>Creating capabilities for filtering</title> |
| <para> |
| While capabilities are mainly used inside a plugin to describe the |
| media type of the pads, the application programmer often also has |
| to have basic understanding of capabilities in order to interface |
| with the plugins, especially when using filtered caps. When you're |
| using filtered caps or fixation, you're limiting the allowed types of |
| media that can stream between two pads to a subset of their supported |
| media types. You do this using a <classname>capsfilter</classname> |
| element in your pipeline. In order to do this, you also need to |
| create your own <classname>GstCaps</classname>. The easiest way to |
| do this is by using the convenience function |
| <function>gst_caps_new_simple ()</function>: |
| </para> |
| <para> |
| <programlisting> |
| static gboolean |
| link_elements_with_filter (GstElement *element1, GstElement *element2) |
| { |
| gboolean link_ok; |
| GstCaps *caps; |
| |
| caps = gst_caps_new_simple ("video/x-raw-yuv", |
| "format", GST_TYPE_FOURCC, GST_MAKE_FOURCC ('I', '4', '2', '0'), |
| "width", G_TYPE_INT, 384, |
| "height", G_TYPE_INT, 288, |
| "framerate", GST_TYPE_FRACTION, 25, 1, |
| NULL); |
| |
| link_ok = gst_element_link_filtered (element1, element2, caps); |
| gst_caps_unref (caps); |
| |
| if (!link_ok) { |
| g_warning ("Failed to link element1 and element2!"); |
| } |
| |
| return link_ok; |
| } |
| </programlisting> |
| This will force the data flow between those two elements to a |
| a certain video format, width, height and framerate (or the linking |
| will fail if that cannot be achieved in the context of the elments |
| involved). Keep in mind that when you use <function> |
| gst_element_link_filtered ()</function> it will automatically create |
| a <classname>capsfilter</classname> element for you and insert it into |
| your bin or pipeline between the two elements you want to connect (this |
| is important if you ever want to disconnect those elements). |
| </para> |
| <para> |
| In some cases, you will want to create a more elaborate set of |
| capabilities to filter a link between two pads. Then, this function |
| is too simplistic and you'll want to use the method |
| <function>gst_caps_new_full ()</function>: |
| </para> |
| <programlisting> |
| static gboolean |
| link_elements_with_filter (GstElement *element1, GstElement *element2) |
| { |
| gboolean link_ok; |
| GstCaps *caps; |
| |
| caps = gst_caps_new_full ( |
| gst_structure_new ("video/x-raw-yuv", |
| "width", G_TYPE_INT, 384, |
| "height", G_TYPE_INT, 288, |
| "framerate", GST_TYPE_FRACTION, 25, 1, |
| NULL), |
| gst_structure_new ("video/x-raw-rgb", |
| "width", G_TYPE_INT, 384, |
| "height", G_TYPE_INT, 288, |
| "framerate", GST_TYPE_FRACTION, 25, 1, |
| NULL), |
| NULL); |
| |
| link_ok = gst_element_link_filtered (element1, element2, caps); |
| gst_caps_unref (caps); |
| |
| if (!link_ok) { |
| g_warning ("Failed to link element1 and element2!"); |
| } |
| |
| return link_ok; |
| } |
| </programlisting> |
| <para> |
| See the API references for the full API of |
| <classname>GstStructure</classname> and |
| <classname>GstCaps</classname>. |
| </para> |
| </sect2> |
| </sect1> |
| |
| <sect1 id="section-pads-ghost"> |
| <title>Ghost pads</title> |
| <para> |
| You can see from <xref linkend="section-bin-noghost-img"/> how a bin |
| has no pads of its own. This is where "ghost pads" come into play. |
| </para> |
| <figure float="1" id="section-bin-noghost-img"> |
| <title>Visualisation of a <ulink type="http" |
| url="../../gstreamer/html/GstBin.html"><classname>GstBin</classname></ulink> |
| element without ghost pads</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="images/bin-element-noghost.ℑ" |
| format="&IMAGE;"/> |
| </imageobject> |
| </mediaobject> |
| </figure> |
| <para> |
| A ghost pad is a pad from some element in the bin that can be |
| accessed directly from the bin as well. Compare it to a symbolic |
| link in UNIX filesystems. Using ghost pads on bins, the bin also |
| has a pad and can transparently be used as an element in other |
| parts of your code. |
| </para> |
| |
| <figure float="1" id="section-bin-ghost-img"> |
| <title>Visualisation of a <ulink type="http" |
| url="../../gstreamer/html/GstBin.html"><classname>GstBin</classname></ulink> |
| element with a ghost pad</title> |
| <mediaobject> |
| <imageobject> |
| <imagedata fileref="images/bin-element-ghost.ℑ" |
| format="&IMAGE;"/> |
| </imageobject> |
| </mediaobject> |
| </figure> |
| <para> |
| <xref linkend="section-bin-ghost-img"/> is a representation of a |
| ghost pad. The sink pad of element one is now also a pad of the bin. |
| Because ghost pads look and work like any other pads, they can be added |
| to any type of elements, not just to a <classname>GstBin</classname>, |
| just like ordinary pads. |
| </para> |
| <para> |
| A ghostpad is created using the function |
| <function>gst_ghost_pad_new ()</function>: |
| </para> |
| <programlisting><!-- example-begin ghostpad.c a --> |
| #include <gst/gst.h> |
| |
| int |
| main (int argc, |
| char *argv[]) |
| { |
| GstElement *bin, *sink; |
| GstPad *pad; |
| |
| /* init */ |
| gst_init (&argc, &argv); |
| |
| /* create element, add to bin */ |
| sink = gst_element_factory_make ("fakesink", "sink"); |
| bin = gst_bin_new ("mybin"); |
| gst_bin_add (GST_BIN (bin), sink); |
| |
| /* add ghostpad */ |
| pad = gst_element_get_static_pad (sink, "sink"); |
| gst_element_add_pad (bin, gst_ghost_pad_new ("sink", pad)); |
| gst_object_unref (GST_OBJECT (pad)); |
| <!-- example-end ghostpad.c a --> |
| [..]<!-- example-begin ghostpad.c b --><!-- |
| return 0; |
| --><!-- example-end ghostpad.c b --> |
| <!-- example-begin ghostpad.c c --> |
| } |
| <!-- example-end ghostpad.c c --></programlisting> |
| <para> |
| In the above example, the bin now also has a pad: the pad called |
| <quote>sink</quote> of the given element. The bin can, from here |
| on, be used as a substitute for the sink element. You could, for |
| example, link another element to the bin. |
| </para> |
| </sect1> |
| </chapter> |
