| purpose |
| ------- |
| |
| GStreamer has a very flexible mechanism to describe media types using |
| mime types and key/value pairs (GstCaps). The definition of media types |
| is entirely done by the plugins which can set the media type to one or |
| more of the plugins GstPads. |
| |
| Pads with the same mime type and 'compatible' properties are allowed to |
| connect. It is possible that a pad can accept or produce many different |
| media types. |
| |
| The purpose of caps negotiation is to provide a framework for plugins so |
| that they can agree on a common media type for their pads. |
| |
| |
| Capabilities (GstCaps) |
| ---------------------- |
| |
| The core component in the caps negotiation system are GstCaps. They consist |
| of: |
| |
| - a name (ex. "my_audio_capabilities") |
| - a mime-type (ex. audio/raw, audio/mp3, ...) |
| - a list of key/value pairs (ex. channels=2, rate=44100, ...) |
| |
| The list of key/value pairs is maintained by the GstProps object. |
| |
| The GstProps object maintains a GList of GstPropsEntries. An entry has |
| a key, which is always a string constant (internally converted to a GQuark) |
| and a value, which can be one of the following: |
| |
| - an integer constant (ex. 5) |
| - a float contant (ex. 1.0) |
| - a string constant (ex. "int") |
| - a boolean constant (ex. FALSE) |
| - a fourcc constant (ex. I420) |
| * fourcc codes are usually used to describe a video format |
| |
| In addition to these constant values, the following variable values are |
| supported too: |
| |
| - an integer range (ex. 0-200) |
| - a float range (ex. 1.0-3.0) |
| - a list of values (ex. 1, 2, 5). |
| * A List cannot contain another list and the |
| entries in the list should all have the |
| same type (int, float, string, fourcc). It is |
| allowed to mix integers/floats and |
| integer/float ranges in a list. |
| |
| A capability is usually described as follows: |
| |
| GST_CAPS_NEW ( |
| capability name ---> "my_audio_capabilities", |
| mime-type ---------> "audio/raw", |
| ( "format", GST_PROPS_STRING ("int"), |
| GstProps ---------> ( "channels", GST_PROPS_INT_RANGE (1, 2), |
| (list of entries) ( "rate", GST_PROPS_INT (44100) |
| ) |
| |
| (-----------) (--------------------------) |
| entry key entry value |
| |
| Two capabilities can be chained together to form a larger capability: |
| |
| ( GST_CAPS_NEW ( |
| ( "my_mp3_capability", |
| ( "audio/mp3", |
| one capability ----> ( NULL |
| created by chaining ( ), |
| two capabilities. ( GST_CAPS_NEW ( |
| ( "my_vorbis_capability", |
| ( "audio/vorbis", |
| ( NULL |
| ( ) |
| |
| Capabilities always work as constraints, this means that they constrain the |
| media type to the given mime-type and properties. By this definition a NULL |
| GstCaps or a NULL GstProps means: no constraints. |
| |
| |
| Variable capabilities vs. fixed capabilities |
| -------------------------------------------- |
| |
| Definition: |
| |
| A GstProps structure is said to be fixed if it doesn't contain lists or |
| ranges, otherwise it is a variable GstProps. A variable GstProps, by definitin |
| does not constrain the media type to a set of fixed properties. |
| |
| A GstCaps is said to be fixed if it is not chained and it doesn't contain |
| a variable GstProps component. |
| |
| |
| GstCaps compatibility |
| --------------------- |
| |
| <write me> |
| |
| GstCaps intersection |
| -------------------- |
| |
| <write me> |
| |
| |
| GstCaps usage |
| ------------- |
| |
| GstCaps are used in the following data structures: |
| |
| - padtemplates. |
| * padtemplates are added to elementfactory to describe the possible |
| pads that an element created from said factory can have. |
| * padtemplates contain the name, direction and presence of the pads. |
| * padtemplates also describe the media types that this element can |
| accept/produce using GstCaps. |
| * padtemplates can provide fixed or variable GstCaps for the pads. |
| * padtemplates can be used by the element to create its pads and is |
| highly recommended. |
| * the padtemplate GstCaps are saved into the registry so that the |
| media types an element can operate on, are known without having to |
| bring the element into memory. |
| |
| - pad caps |
| * pad caps are _fixed_ caps attached to a pad to describe the exact media |
| type this pad is handling. A pad with caps is said to be a "tainted" pad. |
| |
| - connection filters |
| * a connection filter is created when two pads are connected. It describes |
| the media type(s) that _can_ flow through this connection. |
| |
| - application connection filters |
| * When the application connects two pads, it can specify an application |
| connection filter that will act as additional constraints on the media types |
| that can flow through the connection. |
| |
| Connection filters and application filters are cleared when two connected pads |
| are disconnected. Pad caps are not cleared. Padtemplates are immutable and |
| never cleared. |
| |
| |
| The GstPad get_caps function |
| ---------------------------- |
| |
| the gst_pad_get_caps function returns the caps of a given pad. The pad caps are |
| calculated as: |
| |
| - if the pad has pad caps, return those |
| - else if the pad has a getcaps function, call the function and return the result |
| - else if the pad has a padtemplate, return the padtemplate caps |
| - else return NULL |
| |
| |
| Purpose of caps negotiation |
| --------------------------- |
| |
| The purpose of the caps negotiation procedure is to set "pad caps" on a pad |
| so that it is compatible with the "connection filter". This has to be done |
| _before_ any data passing happens through the connection. Data passing between two |
| pads is _not_ allowed when the pad caps are not compatible with the connection |
| filter or when the pad caps of two pads participating in the connection are not |
| equal. |
| |
| Real caps negotiation starts as soon as an element is in the READY state or higher. |
| This means that the connect functions of the pads are only called when the element |
| is at least READY. The intersection between two pads is made at connect time, |
| regardless of element state. |
| |
| |
| GstPad connection |
| ----------------- |
| |
| When two pads are connected get_caps is called on both pads to get their caps. |
| Then the intersection between those caps is made, this will give us all the |
| possible media types that can flow through this pad connection. Optionally the |
| application can provide additional caps, the pad intersection is also made with |
| the application caps. |
| |
| The intersection and the optional application caps are stored in the two pads. |
| |
| If the intersection is NULL, the two pads have no common types and the connection |
| is refused. |
| |
| If the intersection is a fixed caps, this means there is only one possible media type |
| that can be used for this connection. |
| |
| For all not NULL intersections the pad connect functions are called with the |
| intersection. Depending on the result of the connect funtion the connection is |
| allowed or refused. |
| |
| If the intersection is fixed and the pad connect functions agreed to the caps, |
| the caps are set on the pads. |
| |
| Note that pad caps are never set for non fixed caps. |
| |
| !Example 1: |
| ! |
| ! 1. before connecting the pads, they both have a set of possible media types, |
| ! on the pad, through the getcaps function or on the padtemplate (here |
| ! represented with capital letters) |
| ! |
| ! srcpad sinkpad |
| ! A B |
| ! B F |
| ! C A |
| ! G |
| ! |
| ! 2. when performing the connection, the intersection between the two sets of caps |
| ! is made. |
| ! |
| ! srcpad sinkpad |
| ! A ) ( B |
| ! B )--------> A <------( F |
| ! C ) B ( A |
| ! ( G |
| ! |
| ! 3. In this case the intersection is not a fixed caps (it's a chained caps). |
| ! the connect function of the two pads are called (if any), the connect |
| ! function can accept of refuse the caps. |
| ! |
| ! 4. if the caps are accepted, the intersection caps are set on both pads. |
| ! |
| ! 5. plugins will typically not configure themselves if they get variable caps. |
| ! It is possible though for a plugin to select one of the caps, fixate |
| ! some properties and refine the filter to fixed caps (see later) |
| ! |
| |
| |
| !Example 2: |
| ! |
| ! 1. we take two pads that intersect to fixed caps (B): |
| ! |
| ! srcpad sinkpad |
| ! A ) ( B |
| ! B )-------> B <-------( F |
| ! |
| ! 2. both pad connect functions are called. |
| ! |
| ! 3. assuming the connect functions did not refuse the connection, the caps |
| ! are set on both pads (because they are fixed). |
| ! |
| ! srcpad (B)--------------(B) sinkpad |
| ! A B |
| ! B F |
| ! |
| |
| !Example 3: |
| ! |
| ! 1. we take two pads, one with pad caps, another one with a padtemplate. |
| ! |
| ! srcpad (B) sinkpad |
| ! ! ( B |
| ! ------> B <-------( F |
| ! |
| ! 2. the pad get_caps function will return the pad caps of the srcpad and |
| ! the padtemplate caps of the sinkpad. The intersection is made, yielding |
| ! a fixed caps (B) which is sent to both connect functions (if any) |
| ! |
| ! 3. If the connect function(s) didn't refuse the connection the fixed caps |
| ! are set on both pads. The intersection is also kept in the pad object. |
| ! |
| ! srcpad (B)--------------(B) sinkpad |
| ! B |
| ! F |
| ! |
| ! 4. when fixed caps are received on the sinkpad the plugin will typically |
| ! configure itself to deal with the format. |
| ! |
| |
| |
| Pad connect functions |
| --------------------- |
| |
| A Pad can be notified when another pad is connected or reconnected to it with |
| fixed or variable caps. This notification will be done with the optional |
| GstPadConnectFunction callback that an element can provide for a pad. |
| Remember that this connection is _always_ called, not only whith fixed caps |
| but also with variable caps (if any). |
| |
| We will explain some of the common situations with some examples. |
| |
| !Example 4: |
| ! |
| ! 1. We have a simple mpeg audio decoder connected to a simple audio sink. |
| ! The mpeg decoder doesn't have a connect function on |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| ! |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
