| purpose | |
| ------- | |
| GStreamer has a very flexible mechanism to describe media types using | |
| mime types and key/value pairs (GstCaps). The definition of media types | |
| is entirely done by the plugins which can set the media type to one or | |
| more of the plugins GstPads. | |
| Pads with the same mime type and 'compatible' properties are allowed to | |
| connect. It is possible that a pad can accept or produce many different | |
| media types. | |
| The purpose of caps negotiation is to provide a framework for plugins so | |
| that they can agree on a common media type for their pads. | |
| Capabilities (GstCaps) | |
| ---------------------- | |
| The core component in the caps negotiation system are GstCaps. They consist | |
| of: | |
| - a name (ex. "my_audio_capabilities") | |
| - a mime-type (ex. audio/raw, audio/mp3, ...) | |
| - a list of key/value pairs (ex. channels=2, rate=44100, ...) | |
| The list of key/value pairs is maintained by the GstProps object. | |
| The GstProps object maintains a GList of GstPropsEntries. An entry has | |
| a key, which is always a string constant (internally converted to a GQuark) | |
| and a value, which can be one of the following: | |
| - an integer constant (ex. 5) | |
| - a float contant (ex. 1.0) | |
| - a string constant (ex. "int") | |
| - a boolean constant (ex. FALSE) | |
| - a fourcc constant (ex. I420) | |
| * fourcc codes are usually used to describe a video format | |
| In addition to these constant values, the following variable values are | |
| supported too: | |
| - an integer range (ex. 0-200) | |
| - a float range (ex. 1.0-3.0) | |
| - a list of values (ex. 1, 2, 5). | |
| * A List cannot contain another list and the | |
| entries in the list should all have the | |
| same type (int, float, string, fourcc). It is | |
| allowed to mix integers/floats and | |
| integer/float ranges in a list. | |
| A capability is usually described as follows: | |
| GST_CAPS_NEW ( | |
| capability name ---> "my_audio_capabilities", | |
| mime-type ---------> "audio/raw", | |
| ( "format", GST_PROPS_STRING ("int"), | |
| GstProps ---------> ( "channels", GST_PROPS_INT_RANGE (1, 2), | |
| (list of entries) ( "rate", GST_PROPS_INT (44100) | |
| ) | |
| (-----------) (--------------------------) | |
| entry key entry value | |
| Two capabilities can be chained together to form a larger capability: | |
| ( GST_CAPS_NEW ( | |
| ( "my_mp3_capability", | |
| ( "audio/mp3", | |
| one capability ----> ( NULL | |
| created by chaining ( ), | |
| two capabilities. ( GST_CAPS_NEW ( | |
| ( "my_vorbis_capability", | |
| ( "audio/vorbis", | |
| ( NULL | |
| ( ) | |
| Capabilities always work as constraints, this means that they constrain the | |
| media type to the given mime-type and properties. By this definition a NULL | |
| GstCaps or a NULL GstProps means: no constraints. | |
| Variable capabilities vs. fixed capabilities | |
| -------------------------------------------- | |
| Definition: | |
| A GstProps structure is said to be fixed if it doesn't contain lists or | |
| ranges, otherwise it is a variable GstProps. A variable GstProps, by definitin | |
| does not constrain the media type to a set of fixed properties. | |
| A GstCaps is said to be fixed if it is not chained and it doesn't contain | |
| a variable GstProps component. | |
| GstCaps compatibility | |
| --------------------- | |
| <write me> | |
| GstCaps intersection | |
| -------------------- | |
| <write me> | |
| GstCaps usage | |
| ------------- | |
| GstCaps are used in the following data structures: | |
| - padtemplates. | |
| * padtemplates are added to elementfactory to describe the possible | |
| pads that an element created from said factory can have. | |
| * padtemplates contain the name, direction and presence of the pads. | |
| * padtemplates also describe the media types that this element can | |
| accept/produce using GstCaps. | |
| * padtemplates can provide fixed or variable GstCaps for the pads. | |
| * padtemplates can be used by the element to create its pads and is | |
| highly recommended. | |
| * the padtemplate GstCaps are saved into the registry so that the | |
| media types an element can operate on, are known without having to | |
| bring the element into memory. | |
| - pad caps | |
| * pad caps are _fixed_ caps attached to a pad to describe the exact media | |
| type this pad is handling. A pad with caps is said to be a "tainted" pad. | |
| - connection filters | |
| * a connection filter is created when two pads are connected. It describes | |
| the media type(s) that _can_ flow through this connection. | |
| - application connection filters | |
| * When the application connects two pads, it can specify an application | |
| connection filter that will act as additional constraints on the media types | |
| that can flow through the connection. | |
| Connection filters and application filters are cleared when two connected pads | |
| are disconnected. Pad caps are not cleared. Padtemplates are immutable and | |
| never cleared. | |
| The GstPad get_caps function | |
| ---------------------------- | |
| the gst_pad_get_caps function returns the caps of a given pad. The pad caps are | |
| calculated as: | |
| - if the pad has pad caps, return those | |
| - else if the pad has a getcaps function, call the function and return the result | |
| - else if the pad has a padtemplate, return the padtemplate caps | |
| - else return NULL | |
| Purpose of caps negotiation | |
| --------------------------- | |
| The purpose of the caps negotiation procedure is to set "pad caps" on a pad | |
| so that it is compatible with the "connection filter". This has to be done | |
| _before_ any data passing happens through the connection. Data passing between two | |
| pads is _not_ allowed when the pad caps are not compatible with the connection | |
| filter or when the pad caps of two pads participating in the connection are not | |
| equal. | |
| Real caps negotiation starts as soon as an element is in the READY state or higher. | |
| This means that the connect functions of the pads are only called when the element | |
| is at least READY. The intersection between two pads is made at connect time, | |
| regardless of element state. | |
| GstPad connection | |
| ----------------- | |
| When two pads are connected get_caps is called on both pads to get their caps. | |
| Then the intersection between those caps is made, this will give us all the | |
| possible media types that can flow through this pad connection. Optionally the | |
| application can provide additional caps, the pad intersection is also made with | |
| the application caps. | |
| The intersection and the optional application caps are stored in the two pads. | |
| If the intersection is NULL, the two pads have no common types and the connection | |
| is refused. | |
| If the intersection is a fixed caps, this means there is only one possible media type | |
| that can be used for this connection. | |
| For all not NULL intersections the pad connect functions are called with the | |
| intersection. Depending on the result of the connect funtion the connection is | |
| allowed or refused. | |
| If the intersection is fixed and the pad connect functions agreed to the caps, | |
| the caps are set on the pads. | |
| Note that pad caps are never set for non fixed caps. | |
| !Example 1: | |
| ! | |
| ! 1. before connecting the pads, they both have a set of possible media types, | |
| ! on the pad, through the getcaps function or on the padtemplate (here | |
| ! represented with capital letters) | |
| ! | |
| ! srcpad sinkpad | |
| ! A B | |
| ! B F | |
| ! C A | |
| ! G | |
| ! | |
| ! 2. when performing the connection, the intersection between the two sets of caps | |
| ! is made. | |
| ! | |
| ! srcpad sinkpad | |
| ! A ) ( B | |
| ! B )--------> A <------( F | |
| ! C ) B ( A | |
| ! ( G | |
| ! | |
| ! 3. In this case the intersection is not a fixed caps (it's a chained caps). | |
| ! the connect function of the two pads are called (if any), the connect | |
| ! function can accept of refuse the caps. | |
| ! | |
| ! 4. if the caps are accepted, the intersection caps are set on both pads. | |
| ! | |
| ! 5. plugins will typically not configure themselves if they get variable caps. | |
| ! It is possible though for a plugin to select one of the caps, fixate | |
| ! some properties and refine the filter to fixed caps (see later) | |
| ! | |
| !Example 2: | |
| ! | |
| ! 1. we take two pads that intersect to fixed caps (B): | |
| ! | |
| ! srcpad sinkpad | |
| ! A ) ( B | |
| ! B )-------> B <-------( F | |
| ! | |
| ! 2. both pad connect functions are called. | |
| ! | |
| ! 3. assuming the connect functions did not refuse the connection, the caps | |
| ! are set on both pads (because they are fixed). | |
| ! | |
| ! srcpad (B)--------------(B) sinkpad | |
| ! A B | |
| ! B F | |
| ! | |
| !Example 3: | |
| ! | |
| ! 1. we take two pads, one with pad caps, another one with a padtemplate. | |
| ! | |
| ! srcpad (B) sinkpad | |
| ! ! ( B | |
| ! ------> B <-------( F | |
| ! | |
| ! 2. the pad get_caps function will return the pad caps of the srcpad and | |
| ! the padtemplate caps of the sinkpad. The intersection is made, yielding | |
| ! a fixed caps (B) which is sent to both connect functions (if any) | |
| ! | |
| ! 3. If the connect function(s) didn't refuse the connection the fixed caps | |
| ! are set on both pads. The intersection is also kept in the pad object. | |
| ! | |
| ! srcpad (B)--------------(B) sinkpad | |
| ! B | |
| ! F | |
| ! | |
| ! 4. when fixed caps are received on the sinkpad the plugin will typically | |
| ! configure itself to deal with the format. | |
| ! | |
| Pad connect functions | |
| --------------------- | |
| A Pad can be notified when another pad is connected or reconnected to it with | |
| fixed or variable caps. This notification will be done with the optional | |
| GstPadConnectFunction callback that an element can provide for a pad. | |
| Remember that this connection is _always_ called, not only whith fixed caps | |
| but also with variable caps (if any). | |
| We will explain some of the common situations with some examples. | |
| !Example 4: | |
| ! | |
| ! 1. We have a simple mpeg audio decoder connected to a simple audio sink. | |
| ! The mpeg decoder doesn't have a connect function on | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |
| ! | |