| The elementfactory for a given element will contain some information about the capabilities of element's | |
| pads or potential pads. An indication will be provided as to whether the pad always exists, always | |
| exists once data is present, or *might* exist once data is present (the latter case is for things like | |
| the MPEG system parsers, where an audio stream might or might not exist). | |
| First, an entirely normal example: | |
| ---------------------------------- | |
| (-----------) (----------) (-------------) | |
| ! disksrc ! ! mpg123 ! ! audiosink ! | |
| ! src sink src sink ! | |
| ! ! ! ! ! ! | |
| (-----------) (----------) (-------------) | |
| We start with only the disksrc. The typefind filter is attached to the disksrc, and via typefind magic | |
| the properties of the disksrc are found to be: | |
| disksrc->src->caps = { | |
| "audio/mp3", | |
| "layer", GST_CAPS_INT (3), | |
| "bitrate", GST_CAPS_INT (128), | |
| NULL | |
| }; | |
| A look through the plugin registry shows that we have an element called mpg123 that has the following | |
| caps: | |
| static GstCapsFactory mpg123_sink_caps = { | |
| "audio/mp3", | |
| "layer", GST_CAPS_INT_RANGE (1, 3), | |
| "bitrate", GST_CAPS_INT_RANGE (8, 320), | |
| NULL | |
| }; | |
| The caps of the disksrc fit within those parameters, so we instantiate an mpg123 and attach it to the | |
| disksrc. The connection succeeds negotiation and as a result the mpg123 specifies its output caps as: | |
| mpg123->src->caps = { | |
| "audio/raw", | |
| "format", GST_CAPS_BITFIELD (S16), | |
| "depth", GST_CAPS_INT (16), | |
| "rate", GST_CAPS_INT (44100), | |
| "channels", GST_CAPS_INT (2), | |
| NULL | |
| }; | |
| Again from the plugin registry we find an element audiosink that has appropriate caps: | |
| static GstCapsFactory audiosink_src_caps = { | |
| "audio/raw", | |
| "format", GST_CAPS_BITFIELD (S16,....), | |
| "depth", GST_CAPS_INT (16), | |
| "rate", GST_CAPS_INT_RANGE (4000, 96000), | |
| "channels", GST_CAPS_INT_RANGE (1, 2), | |
| NULL | |
| }; | |
| A copy of the audiosink is instantiated and attached, negotiation goes smoothly, and we're done. No | |
| dataflow has occured, no failure found, etc. An ideal autoplug. | |
| Now, a slightly more convoluted example: | |
| ---------------------------------------- | |
| Start with the same graph: | |
| (-----------) (----------) (-------------) | |
| ! disksrc ! ! mpg123 ! ! audiosink ! | |
| ! src sink src sink ! | |
| ! ! ! ! ! ! | |
| (-----------) (----------) (-------------) | |
| Run typefind on the disksrc's output, get the same output caps: | |
| disksrc->src->caps = { | |
| "audio/mp3", | |
| "layer", GST_CAPS_INT (3), | |
| "bitrate", GST_CAPS_INT (128), | |
| NULL | |
| }; | |
| Find and attach mpg123, get the following output caps this time: | |
| mpg123->src->caps = { | |
| "audio/raw", | |
| "format", GST_CAPS_BITFIELD (S16), | |
| "depth", GST_CAPS_INT (16), | |
| "rate", GST_CAPS_INT (44100), | |
| "channels", GST_CAPS_INT (1), | |
| NULL | |
| }; | |
| Note that this time we have a mono output. A look into the audiosink caps shows that we have a match. | |
| So we instantiate a copy. Oops. We now find that the caps for the input pad on our audiosink have | |
| changed: | |
| mpg123->src->caps = { | |
| "audio/raw", | |
| "format", GST_CAPS_BITFIELD (S16,...), | |
| "depth", GST_CAPS_INT (16), | |
| "rate", GST_CAPS_INT (11025, 48000), | |
| "channels", GST_CAPS_INT (2), | |
| NULL | |
| }; | |
| Whoops. It seems that the sound card we've got in this machine (FIXME how on earth to deal with | |
| multiple sound cards???) doesn't support mono output *at all*. This is a problem. We now find that we | |
| hae no options as far as directly matching the mpg123 to the audiosink. | |
| A look through our (ficticious) plugin registry shows at least one element that at least has audio/raw | |
| on both input and ouput (since both mpg123 and audiosink have open pads with this mime type). A closerlook shows that its caps are: | |
| static GstCapsFactory mono2stereo_sink_caps = { | |
| "audio/raw", | |
| "channels", GST_CAPS_INT (1), | |
| NULL | |
| }; | |
| static GstCapsFactory mono2stereo_src_caps = { | |
| "audio/raw", | |
| "channels", GST_CAPS_INT (2), | |
| NULL | |
| }; | |
| Wow, that's a perfect match. Instantiate, attach to mpg123, no problems. Attach to audiosink, no | |
| problems. Done. When we start up the pipeline, we should get absolutely no callbacks from pads saying | |
| "help me, I've fallen and..., er, I don't like this buffer!". | |
| A really messy case: | |
| -------------------- | |
| Start with a disksrc, typefind it, get the following: | |
| disksrc->src->caps = { | |
| "audio/mp3", | |
| "layer", GST_CAPS_INT (3), | |
| "bitrate", GST_CAPS_INT (128), | |
| NULL | |
| }; | |
| Look through the plugin registry, find mpg123. Instantiate it, attach it. It spits out audio | |
| parameters as usual: | |
| mpg123->src->caps = { | |
| "audio/raw", | |
| "format", GST_CAPS_BITFIELD (S16), | |
| "depth", GST_CAPS_INT (16), | |
| "rate", GST_CAPS_INT (44100), | |
| "channels", GST_CAPS_INT (2), | |
| NULL | |
| }; | |
| Now we instantiate an audiosink plugin. This time, we're sunk: | |
| mpg123->src->caps = { | |
| "audio/raw", | |
| "format", GST_CAPS_BITFIELD (S8,U8), | |
| "depth", GST_CAPS_INT (8), | |
| "rate", GST_CAPS_INT_RANGE (11025, 22050), | |
| "channels", GST_CAPS_INT (1), | |
| NULL | |
| }; | |
| ACK! It's one of those Disney Sound Source things. We've got a problem here that isn't obviously | |
| solvable. However, there happens to be another mp3 decoder sitting around. It's got the same | |
| properties as mpg123, but a lower merit value. Let's instantiate one and attach it. We get the | |
| following output pad caps: | |
| mp3decoder->src->caps = { | |
| "audio/raw", | |
| "format", GST_CAPS_BITFIELD (S8,S16), | |
| "depth", GST_CAPS_INT_RANGE (8,16), | |
| "rate", GST_CAPS_INT_RANGE (8000, 44100), | |
| "channels", GST_CAPS_INT (1,2), | |
| NULL | |
| }; | |
| Well, that matches the audiosink. We try attaching it, and during negotiation the mp3decoder finds | |
| sufficient common ground with the castrated audiosink and sets its output pad to match the best of the | |
| options: S8 at 22050 KHz. | |
| Next to impossible scenario: DVD | |
| -------------------------------- | |
| Start with a dvdsrc. It's output pad caps are: | |
| static GstCapsFactory dvdsrc_src_caps = { | |
| "video/mpeg", | |
| "mpegversion", GST_CAPS_INT (2), | |
| "systemstream", GST_CAPS_BOOLEAN (TRUE), | |
| NULL | |
| }; | |
| The type would be classified as incomplete via some mechanism. This might cause the autoplug code to go | |
| and run the typefind function. It would flesh the type out to the following: | |
| dvdsrc->src->caps = { | |
| "video/mpeg", | |
| "mpegversion", GST_CAPS_INT (2), | |
| "systemstream", GST_CAPS_BOOLEAN (TRUE), | |
| "videostreams", GST_CAPS_INT (1), | |
| "audiostreams", GST_CAPS_INT (3), | |
| "bitrate", GST_CAPS_INT (40960), | |
| NULL, | |
| }; | |
| Wow, that helped a lot. A check through the plugin registry shows that the mpeg2parse will match those | |
| properties: | |
| static GstCapsFactory mpeg2parse_sink_caps = { | |
| "video/mpeg", | |
| "mpegversion", GST_CAPS_INT (2), | |
| "systemstream", GST_CAPS_BOOLEAN (TRUE), | |
| NULL | |
| }; | |
| (In retrospect, it may not be necessary to run typefind if there's match this good right away. Only run | |
| typefind when there's no exact match.) | |
| Since there are no output pads yet, we have to actually push data through the pipeline. The moment a | |
| buffer or two get to the mpeg2parse element, it promptly goes and creates an output pad, probably of the | |
| following caps: | |
| mpeg2parse_video_src_caps = { | |
| "video/mpeg", | |
| "mpegversion", GST_CAPS_RANGE (1,2), | |
| "systemstream", GST_CAPS_BOOLEAN (FALSE), | |
| NULL | |
| }; | |
| This seems to be a task for typefind again. But since data is flowing, we have to be careful with the | |
| buffers. (This is the case in any typefind maneuver, but moreso when one really can't rewind the | |
| source without consequences) The autoplug system attaches a special pseudo-element to mpeg2parse's new | |
| output pad, and attaches the typefind element to the end of that. The pseudo-element takes the buffer, | |
| stores it, and passes a copy off to the attached element, in this case typefind. This repeats until | |
| typefind has determined the type, at which point the typefind is removed, and the newly found element is | |
| attached instead. | |
| The pseudo-element is 'rewound' and the stored buffers flow out and into the newly attached element. | |
| When the cache of buffers is gone, a signal fires and the autoplug system removes the pseudo-element and | |
| reconnects the pipeline. | |
| In this case, the typefind function will find the following: | |
| mpeg2parse_video_src_caps = { | |
| "video/mpeg", | |
| "mpegversion", GST_CAPS_INT (2), | |
| "systemstream", GST_CAPS_BOOLEAN (FALSE), | |
| "bitrate", GST_CAPS_INT (36864), | |
| "width", GST_CAPS_INT (720), | |
| "height", GST_CAPS_INT (480), | |
| "framerate", GST_CAPS_FLOAT (29.97002997), | |
| "chromaformat", GST_CAPS_INT (1), [GST_CAPS_STRING ("4:2:0") ?] | |
| NULL | |
| }; | |
| Back to the plugin registry, we find our only choice is mpeg2dec, which has input caps of: | |
| static GstCapsFactory mpeg2dec_sink_caps = { | |
| "video/mpeg", | |
| "mpegversion", GST_CAPS_RANGE (1,2), | |
| "systemstream", GST_CAPS_BOOLEAN (FALSE), | |
| NULL | |
| }; | |
| Once again it just so happens that we really didn't need to do the typefind at all. But it can't hurt | |
| unless the typefind is slow and painful, which we can guess won't be the case since the choices are | |
| rather limited by the fact that there's already a MIME type attached, meaning we can drastically reduce | |
| the number of typefind functions we try (down to one, actually). | |
| However, since we *have* run the typefind, upon attachment of the input pad of mpeg2dec, the output pad | |
| looks like the following: | |
| mpeg2dec_src_caps = { | |
| "video/raw", | |
| "fourcc", GST_CAPS_LIST ( | |
| GST_CAPS_FOURCC ("YV12"), [identical...] | |
| GST_CAPS_FOURCC ("IYUV"), | |
| GST_CAPS_FOURCC ("I420"), | |
| ), | |
| "width", GST_CAPS_INT (720), | |
| "height", GST_CAPS_INT (480), | |
| "framerate", GST_CAPS_FLOAT (29.97002997), | |
| NULL | |
| }; | |
| Currently only videosink supports the output of video/raw. It claims a list of FOURCCs but nothing | |
| more: | |
| static GstCapsFactory videosink_sink_caps = { | |
| "video/raw", | |
| "fourcc", GST_CAP_LIST ( GST_CAPS_FOURCC ("YV12"), | |
| GST_CAPS_FOURCC ("IYUV"), GST_CAPS_FOURCC ("I420"), | |
| GST_CAPS_FOURCC ("YUY2"), GST_CAPS_FOURCC ("UYVY"), | |
| [ etc... ], | |
| ), | |
| NULL | |
| }; | |
| When instantiated, we potentially have the same problem as with the audiosink: we don't necessarily know | |
| which hardware output to use. Somehow we have to solve the problem of setting some element arguments | |
| before we can get useful information out of them as to the properties. In this case anyway, if the | |
| videosink were to find only one output possibility, it would trim the list of FOURCCs it can deal with | |
| to what the hardware can handle, as well as add further properties: | |
| videosink_sink_caps = { | |
| "video/raw", | |
| "fourcc", GST_CAPS_LIST (GST_CAPS_FOURCC ("YV12"), | |
| GST_CAPS_FOURCC ("YUY2"), | |
| ), | |
| "width", GST_CAPS_INT_RANGE (4,1020), | |
| "height", GST_CAPS_INT_RANGE (4,1020), | |
| NULL | |
| }; | |
| We can now connect the mpeg2dec output to the videosink, and we now have displaying video. | |
| . . . . |