| OUTDATED |
| -------- |
| |
| |
| 1. Introduction |
| --------------- |
| |
| The type system is used to attach meaning to the bytes in a GstBuffer. |
| A plugin can decide to add metadata to the GstBuffer, this metadata |
| will carry an associated typeid. |
| |
| Types are also used by the plugins to expose the type of their pads to |
| the type system. |
| |
| Types are essential for autoplugging. |
| |
| We will explain the inner workings of the type system in this document, as |
| well as the API used in the plugins. |
| |
| |
| 2. Type properties |
| ------------------ |
| |
| a) identification of the type |
| |
| Types are identified by one or more MIME types. |
| |
| b) detection of types |
| |
| The type of any given GstBuffer can be detected using |
| |
| - a typefind function: a function that will inspect the bytes |
| in the buffer and return a gboolean indicating the |
| buffer is of the given type. |
| |
| - a template for the bytes/bits in the data that must be |
| satisfied in order for the GstBuffer to be of the given |
| type. |
| |
| - other properties that act more like a hint like: |
| the extension of the source filename. |
| the URI of the source. |
| |
| |
| 3. Type registration |
| -------------------- |
| |
| a) the core libraries will create to types: |
| |
| video/raw image/raw |
| audio/raw |
| |
| b) all other types will be provided by the plugins |
| |
| before a type can become available to other plugins, the plugin |
| has to register the type. |
| |
| The system will keep a directed graph of the types and the plugins |
| that operate on them. |
| |
| example: |
| |
| video/mpeg |
| ! |
| mpeg1parse |
| / \ |
| video/mpeg1 audio/mp3 -----\ |
| ! ! \ ! |
| mpeg_play mpeg2dec mpg123 xing ... |
| \ / \ / |
| video/raw audio/raw-----\ |
| ! ! ! ! |
| videosink SDLsink audiosink alsasink ... |
| |
| |
| The system can find the needed plugins to convert video/mpeg to |
| audio/raw using this graph. |
| |
| |
| 4. type equivalence |
| ------------------- |
| |
| some types can have the same meaning for example: |
| |
| audio/mp3 and audio/mpeg |
| |
| or |
| |
| video/raw and image/raw |
| |
| |
| a plugin that exposes its output type as audio/mpeg and another plugins |
| with input type audio/mp3 can be connected. The system has to know about |
| the equivalence of both types, even it they have a different mime type. |
| |
| |
| 5. type hierarchy |
| ----------------- |
| |
| some plugins can ouput a specific subset of an already existing type. |
| |
| example: |
| |
| mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames |
| with mime type: audio/mp3-frame |
| |
| mpg123 only accepts audio/mp3-frame but not audio/mp3. |
| |
| another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also |
| audio/mp3-frame) |
| |
| it must be possible to connect both libmpg123 and mpg123 to the mp3parse |
| element. |
| it must not be possible to connect mpg123 to an element that outputs only |
| audio/mp3 but not audio/mp3-frame. |
| |
| |
| We say that audio/mp3-frame is a more specific subset of type audio/mp3. |
| |
| we can create a type hierarchy: |
| |
| audio/mp3 |
| / \ |
| audio/mp3-frame audio/mp3-layer12 |
| / \ |
| audio/mp3-layer1 audio/mp3-layer2 |
| |
| |
| 6. multi-type pads |
| ------------------ |
| |
| certain plugins might accept multiple non equivalent types in one of their |
| input pads. Consequently a plugin might output non equivalent types in |
| its output pad. |
| |
| It must therefore be possible to specify multiple types for a pad. |
| |
| example: |
| |
| mpegdemux may be able to demux both MPEG1 and MPEG2 system streams. |
| we show the type hierarchy of the video/mpeg as follows: |
| |
| video/mpeg |
| / \ |
| video/mpeg1 video/mpeg2 ---------------\ |
| / \ / \ ! |
| mpeg1-system* mpeg1-video mpeg2-ts mpeg2-ps* mpeg2-video |
| |
| |
| the mpegdemux element might specify the type of the input pad as |
| one of video/mpeg1-system and video/mpeg2-ts |
| |
| |
| |
| 7. definition of types |
| ---------------------- |
| |
| A plugin will provide the following information to the type system: |
| |
| - a mime type string describing the hierarchy and where the types |
| they provide are located in that hierarchy. |
| |
| - typefind functions for some of the types. |
| |
| - extensions for some of the types |
| |
| We will propose a syntax to define the type hierarchy |
| |
| a) equivalent types : |
| |
| separated with a | sign |
| |
| ( audio/mp3 | audio/mpeg ) |
| |
| b) type hierarchy : |
| |
| in braces: |
| |
| ( audio/mp3 ( audio/mp3-frame)) |
| |
| c) multi-type pads |
| |
| ( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) ) |
| |
| the pad will have type mpegdemux which is the parent for |
| type video/mpeg1-system or video/mpeg2-ps |
| |
| mpegdemux |
| / \ |
| video/mpeg1-system video/mpeg2-ps |
| |
| |
| |
| once the type hierarchy has been registered, the typeid of each |
| element can be obtained with: |
| |
| guint16 gst_type_find_by_mime (gchar *mime) |
| |
| extra typefind functions and/or extensions can be added to the |
| typeid afterwards. |
| |
| |
| 8. type matching |
| ---------------- |
| |
| The more specific typefind functions, the functions associated with |
| types in the leaf nodes, are tried first. |
| |
| when a specific type has been found ex. video/mpeg1-system elements |
| that can handle this type or one of its parents are selected: |
| |
| mpegdemux: mpeg1parse: supermpegdecoder: |
| |
| video/mpeg video/mpeg video/mpeg |
| ! ! |
| mpegdemux video/mpeg1-system |
| ! |
| video/mpeg1-system |
| |
| In the above case, also the super mpeg element is selected because it stated |
| that it can handle all sorts of video/mpeg data. |
| |
| |
| example 2: |
| |
| suppose the typefind functions finds an mp3 stream, fillowing elments |
| are selected: |
| |
| libmpg123 mp3parse: |
| |
| audio/mp3 audio/mp3 |
| |
| mpg123 is not selected because its pad type is too specific (audio/mp3-frame): |
| |
| mpg123 |
| |
| audio/mp3 |
| ! |
| audio/mp3-frame |
| |
| if the typefind would find a mp3-frame type, all three objects would be selected. |
| |
| |
| 8. consideration |
| ---------------- |
| |
| It is clear that clear indications have to be given to the type hierarchy, |
| especially for the top nodes. |
| |
| The more specific an element is in its mime type specification, the more flexible |
| and extendible the plugin system can be. |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
