| OUTDATED |
| -------- |
| |
| |
| caps negotiation |
| ================ |
| |
| 1) purpose |
| ---------- |
| |
| The pads expose the media types they can handle using a mime |
| type and a set of properties. Before the pad is created or |
| used to pass buffers, we only know the global 'range' of media |
| data this pad can accept. When the element has had a chance to |
| look at the media data, only then it knows the exact values of the |
| properties. |
| |
| example1: |
| ! |
| ! The mp3 decoder exposes the capabilities of its src pad |
| ! with the following caps: |
| ! |
| ! 'mpg123_src': |
| ! MIME type: 'audio/raw': |
| ! format: Integer: 16 |
| ! depth: Integer: 16 |
| ! rate: Integer range: 11025 - 48000 |
| ! channels: Integer range: 1 - 2 |
| |
| as you can see in example1, the padtemplate has both a range |
| (for the audio rate) and a list (for the number of channels) |
| for its properties. |
| |
| only when the mpg123 element has decoded the first mpeg audio |
| header, it knows the exact values of the rate and channels |
| properties. |
| |
| suppose that we want to connect this src pad to the sink pad |
| of an audiosink with the properties given in example2: |
| |
| example2: |
| ! |
| ! 'audiosink_sink': |
| ! MIME type: 'audio/raw': |
| ! format: Integer: 16 |
| ! depth: List: |
| ! Integer: 8 |
| ! Integer: 16 |
| ! rate: Integer range: 8000 - 44000 |
| ! channels: Integer range: 1 - 2 |
| |
| we can see that connecting the mpg123 src pad with the |
| audiosinks sink pad can cause a potential problem with the |
| rate property. |
| |
| When the mpg123 decoder decides to output raw audio with a |
| 48000Hz samplerate, the audiosink will not be able to handle |
| it. The conservative approach would be to disallow the connection |
| between the two incompatible pads. This rules out any potential |
| problems but severely limits the amount of possible connections |
| between the elements. |
| |
| Another approach would be to allow the connection (and mark it |
| as dangerous) and let the two elements figure out a suitable |
| media type at runtime. This procedure is called caps negotiation. |
| |
| |
| 2) a bit of history |
| ------------------- |
| |
| The typing of the data that was attached to a buffer used to be |
| done using GstMeta* (and it still is as of 11 feb 2001). With |
| the new GstCaps and GstProps system this typing is gradually moved |
| to the pads and to the padtemplates. This has several advantages: |
| |
| - the typing of the data tends to be static. The type of media |
| doesn't change for every buffer. |
| |
| - Moving the typing up to the pad(templates) allows us to save |
| them into the registry and allows us to figure out what pads |
| are compatible. |
| |
| - the current metadata implementation needs header files. this may |
| change when we also use properties for metadata. |
| |
| example3: |
| ! |
| ! This is the current GstMeta structure that travels with audio buffers |
| ! |
| ! struct _MetaAudioRaw { |
| ! GstMeta meta; |
| ! |
| ! /* formatting information */ |
| ! gint format; |
| ! gint channels; |
| ! gint frequency; |
| ! gint bps; |
| ! }; |
| |
| |
| The question still remains what purpose the metadata will serve |
| now that we expose the media type in the pads. Some possibilities: |
| |
| - interesting information, not describing the data itself but the |
| context in which the data was generated (suggested buffer size, |
| timestamps, etc...) |
| |
| - user app metadata. |
| |
| In this proposal we also assume that the current use of metadata using |
| GstMeta is deprecated and that we move this information to the properties |
| of the pads. |
| |
| |
| 3) the pad/padtemplates caps |
| ---------------------------- |
| |
| All elements have to provide a padtemplate for their pads. |
| |
| The padtemplates provide a range of possible media types this pad can |
| src/sink. the main purpose for the padtemplates is to allow a |
| rough guess at which pads are compatible before even a single buffer |
| has been processed by the element. |
| |
| pads are usually created from the templates. When the pad is created |
| it has no GstCaps* attached to it yet. The possible caps this pad |
| can have is exposed in the padtemplate. The caps are filled in by |
| the element when it knows the values for the caps. |
| |
| |
| 4) the connect function |
| ----------------------- |
| |
| when two pads are connected the following steps will take |
| place (not sure, FIXME): |
| |
| - if both pads have caps, the caps are checked. If the caps |
| are incompatible, the padtemplates are checked, if they |
| are compatible, caps negotiation is performed. |
| |
| - if one of the pads has caps, the caps is checked against |
| the padtemplate of the peer pad. If they are incompatible, |
| the padtemplates are compared, if they are incompatible, |
| caps negotiation is performed. |
| |
| - if none of the pads have caps, the padtemplates are checked, |
| if they are incompatible, a warning is issued. |
| |
| |
| 5) when the element knows the media type it is handling |
| ------------------------------------------------------- |
| |
| When the element has received its first buffer it will know |
| the media type it is handling by inspecting the buffer. |
| |
| before pushing the data out to its peer element(s), the element |
| will set its src pad with the appropriate caps and properties. |
| These caps must follow the following rules: |
| |
| - the caps must be compatible with the padtemplates of this |
| pad. |
| |
| - the caps cannot contain ranges or lists. |
| |
| when the element wants to change the caps of a pad, it has to |
| perform gst_pad_renegotiate (GstPad *pad). this will trigger |
| the caps negotiation procedure. |
| |
| this will trigger the class method of the pad and calls the pads |
| gst_pad_negotiate function: |
| |
| GstCaps *gst_pad_negotiate (GstPad *pad, GstCaps *caps, guint count); |
| |
| This function takes a GstCaps *structure as an argument (typically the |
| current caps of the pad) and a negotiation counter. this counter can be |
| used to keep track of the negotiation process. |
| |
| The pad then creates a new caps structure with the desired caps. |
| If the caps are accepted, it just returns the provided input caps. the |
| _renegotiate function will set the caps of both pads whenever the |
| input caps are the same (pointer wise) as the input caps. |
| |
| the caps structure is checked against the padtemplate of the peer pad, |
| if it is incompatible the gst_pad_negotiate function is called again |
| and the element is supposed to create another caps structure. |
| |
| the gst_pad_renegotiate function then calls the gst_pad_negotiate |
| function of the peer pad with the new caps as the argument. The peer |
| pad can adjust or create a new caps if it doesn't accept it. |
| |
| the caps structure keeps on bouncing between the two pads until one |
| of the pads negotiation functions returns the caps unmodified. |
| |
| The element can also return a NULL pointer if it has run out of |
| options for the caps structure. When this happens, both pads are set |
| the NULL caps again and the pad connnection is broken. |
| |
| The negotiation process is stopped after a fixed number of tries, |
| when the counter has reached some limit. This limit is typically |
| checked by the pads negotiate function. |
| |
| |
| 6) caps negotiation function |
| ---------------------------- |
| |
| the negotiate function of a pad is called whenever the pad or |
| peer pad has performed _renegotiate. |
| |
| example5: |
| ! |
| ! this is the caps negotiation function implemented by an element on |
| ! one of its sink pads. |
| ! |
| ! static GstCaps* |
| ! gst_pad_negotiate (GstPad *pad, GstCaps *caps, guint counter) |
| ! { |
| ! /* we don't accept anything else than audio/raw */ |
| ! if (strcmp (gst_caps_get_mime (caps), "audio/raw")) |
| ! return NULL; |
| ! |
| ! if (gst_caps_get_int_prop (caps, "format") != AFMT_S16_LE) |
| ! return NULL; |
| ! |
| ! /* we accept everything else */ |
| ! return caps; |
| ! } |
| |
| When the negotiate function returns NULL (it does not accept the |
| specified caps of the peer pad), the negotiation process is stopped. |
| |
| |
| |
| APPENDIX A: use cases |
| ===================== |
| |
| 1) mpg123 src!sink audiosink |
| ---------------------------- |
| |
| When the pads are connected the padtemplates are checked and it |
| turns out that the pads might be incompatible (mpg123 can do |
| 48000Hz while audiosink can do 44000Hz). Nothing happens at |
| connect time except for the user app that can mark this connection |
| as possibly dangerous and keep some spare elements ready for when |
| the pads turn out to be incompatible. |
| |
| both elements start out with no caps at all (NULL). mpg123 wants |
| to output a buffer with specific properties. It calls |
| gst_pad_renegotiate (mpg123->srcpad). |
| |
| The _renegotiate functions calls the negotiate function of the |
| mpg123->srcpad. the negotiate function would look like this: |
| |
| |
| /* |
| * The mpg123 element cannot convert the decoded type into something |
| * else so it has to force the caps of the src pad into the specific |
| * type as defined by the mp3. |
| */ |
| static GstCaps* |
| gst_mpeg123_src_negotiate (GstPad *pad, GstCaps *caps, guint counter) |
| { |
| GstMpg123 *mpg123; |
| |
| mpg123 = GST_MPG123 (gst_pad_get_parent (pad)); |
| |
| /* we got caps in, check them */ |
| if (caps != NULL) { |
| if (!strcmp (gst_caps_get_mime (caps), "audio/raw") && |
| (gst_caps_get_int_prop (caps, "format") == AFMT_S16_LE) && |
| (gst_caps_get_int_prop (caps, "depth") == 16) && |
| (gst_caps_get_int_prop (caps, "rate") == mpg123->rate) && |
| (gst_caps_get_int_prop (caps, "channels") == mpg123->channels)) { |
| return caps; |
| } |
| } |
| /* we didn't get caps, so we decide */ |
| else if (counter != 2) { |
| GstCaps *new; |
| |
| /* fill in our desired caps */ |
| new = gst_caps_new_with_props ( |
| "src_caps", /* name */ |
| "audio/raw", /* mime */ |
| gst_props_new ( |
| "format", GST_PROPS_INT (AFMT_S16_LE), |
| "depth", GST_PROPS_INT (16), |
| "rate", GST_PROPS_INT (mpg123->rate), |
| "channels", GST_PROPS_INT (mpg123->channels), |
| NULL |
| ) |
| ); |
| return caps; |
| } |
| /* too many attempts at nogotiation, bail out */ |
| return NULL; |
| } |
| |
| |
| The audiosink pad negotiate function would look like this: |
| |
| /* |
| * The audiosink has a wide range of possible parameters for |
| * its sink pad, based on the audio card capabilities and |
| * possibly the element configuration. |
| * we assume the audiosink element can be both the initiator of |
| * the negotiations and the negotiated one. |
| */ |
| static GstCaps* |
| gst_audiosink_sink_negotiate (GstPad *pad, GstCaps *caps, guint counter) |
| { |
| GstAudiosink *audiosink; |
| gboolean accepted = TRUE; |
| |
| audiosink = GST_AUDIOSINK (gst_pad_get_parent (pad)); |
| |
| /* we got caps in, we know they will match the padtemplate */ |
| if (caps != NULL) { |
| return caps; |
| } |
| /* we didn't get caps, so we decide */ |
| else if (counter != 2) { |
| GstCaps *new; |
| |
| /* fill in our desired caps */ |
| new = gst_caps_new_with_props ( |
| "sink_caps", /* name */ |
| "audio/raw", /* mime */ |
| gst_props_new ( |
| "format", GST_PROPS_INT (audiosink->format), |
| "depth", GST_PROPS_INT (audiosink->depth), |
| "rate", GST_PROPS_INT (audiosink->rate), |
| "channels", GST_PROPS_INT (audiosink->channels), |
| NULL |
| ) |
| ); |
| return caps; |
| } |
| /* too many attempts at nogotiation, bail out */ |
| return NULL; |
| } |
| |
| |
| |
| |
| |
| |
| |
| |
| |