OUTDATED
--------


1. Introduction
---------------

The type system is used to attach meaning to the bytes in a GstBuffer.
A plugin can decide to add metadata to the GstBuffer, this metadata
will carry an associated typeid.

Types are also used by the plugins to expose the type of their pads to
the type system.

Types are essential for autoplugging. 

We will explain the inner workings of the type system in this document, as
well as the API used in the plugins.


2. Type properties
------------------

a) identification of the type

  Types are identified by one or more MIME types. 

b) detection of types

  The type of any given GstBuffer can be detected using

    - a typefind function: a function that will inspect the bytes
           in the buffer and return a gboolean indicating the
	   buffer is of the given type.

    - a template for the bytes/bits in the data that must be
           satisfied in order for the GstBuffer to be of the given 
	   type.
  
    - other properties that act more like a hint like:
           the extension of the source filename.
	   the URI of the source.


3. Type registration
--------------------

a) the core libraries will create to types:

   video/raw image/raw
   audio/raw

b) all other types will be provided by the plugins

before a type can become available to other plugins, the plugin
has to register the type. 

The system will keep a directed graph of the types and the plugins
that operate on them.

example:

                     video/mpeg
                         ! 
		     mpeg1parse
		      /     \
	      video/mpeg1   audio/mp3 -----\
	      !        !           \        !
	 mpeg_play   mpeg2dec      mpg123   xing  ...
              \        /              \     /		        
              video/raw              audio/raw-----\
	      !      !                   !         !
        videosink   SDLsink           audiosink    alsasink ...
			

The system can find the needed plugins to convert video/mpeg to
audio/raw using this graph.


4. type equivalence
-------------------

some types can have the same meaning for example:

  audio/mp3 and audio/mpeg 

or

  video/raw and image/raw
  

a plugin that exposes its output type as audio/mpeg and another plugins
with input type audio/mp3 can be connected. The system has to know about
the equivalence of both types, even it they have a different mime type.

 
5. type hierarchy
-----------------

some plugins can ouput a specific subset of an already existing type.

example:

  mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames
  with mime type: audio/mp3-frame

  mpg123 only accepts audio/mp3-frame but not audio/mp3.

  another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also 
  audio/mp3-frame)

  it must be possible to connect both libmpg123 and mpg123 to the mp3parse
  element.
  it must not be possible to connect mpg123 to an element that outputs only
  audio/mp3 but not audio/mp3-frame.


We say that audio/mp3-frame is a more specific subset of type audio/mp3.

we can create a type hierarchy:

          audio/mp3
         /        \
  audio/mp3-frame  audio/mp3-layer12
                     /        \
	   audio/mp3-layer1    audio/mp3-layer2


6. multi-type pads
------------------

certain plugins might accept multiple non equivalent types in one of their
input pads. Consequently a plugin might output non equivalent types in
its output pad.

It must therefore be possible to specify multiple types for a pad.

example:

  mpegdemux may be able to demux both MPEG1 and MPEG2 system streams.
  we show the type hierarchy of the video/mpeg as follows:

                          video/mpeg
		         /        \
               video/mpeg1         video/mpeg2 ---------------\
	       /    \                    /    \               !
      mpeg1-system*  mpeg1-video    mpeg2-ts   mpeg2-ps*    mpeg2-video


  the mpegdemux element might specify the type of the input pad as 
  one of video/mpeg1-system and video/mpeg2-ts


7. definition of types
----------------------

A plugin will provide the following information to the type system:

 - a mime type string describing the hierarchy and where the types
   they provide are located in that hierarchy.

 - typefind functions for some of the types.

 - extensions for some of the types

We will propose a syntax to define the type hierarchy

a) equivalent types :

   separated with a | sign

   ( audio/mp3 | audio/mpeg )

b) type hierarchy :

   in braces:

   ( audio/mp3 ( audio/mp3-frame))

c) multi-type pads

  ( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) )

  the pad will have type mpegdemux which is the parent for
  type video/mpeg1-system or video/mpeg2-ps

                  mpegdemux
          	  /       \
 video/mpeg1-system      video/mpeg2-ps
   

once the type hierarchy has been registered, the typeid of each
element can be obtained with:

  guint16 gst_type_find_by_mime (gchar *mime)

extra typefind functions and/or extensions can be added to the 
typeid afterwards.


8. type matching
----------------

The more specific typefind functions, the functions associated with
types in the leaf nodes, are tried first.

when a specific type has been found ex. video/mpeg1-system elements
that can handle this type or one of its parents are selected:

  mpegdemux:                     mpeg1parse:                    supermpegdecoder:

    video/mpeg                       video/mpeg                   video/mpeg
     !                                  !
    mpegdemux                        video/mpeg1-system
     !                                 
    video/mpeg1-system         

In the above case, also the super mpeg element is selected because it stated
that it can handle all sorts of video/mpeg data. 


example 2:

   suppose the typefind functions finds an mp3 stream, fillowing elments
   are selected:

       libmpg123              mp3parse:

       audio/mp3              audio/mp3
 
   mpg123 is not selected because its pad type is too specific (audio/mp3-frame):

       mpg123

       audio/mp3
         !
       audio/mp3-frame

   if the typefind would find a mp3-frame type, all three objects would be selected.


8. consideration
----------------

It is clear that clear indications have to be given to the type hierarchy,
especially for the top nodes.

The more specific an element is in its mime type specification, the more flexible 
and extendible the plugin system can be.