| Introduction (1) |
| (sorry, no cool logo/graphic yet, ideas?) |
| GStreamer is a library and set of tools to build arbitrary, |
| reconfigurable filter graphs. It derives from the OGI Pipeline |
| and DirectShow (docs, no experience), and is in its second |
| generation (first was completed/abandoned *in* Atlanta on the |
| way to the Linux Expo). |
| 0.1.0 release is scheduled for Oct 31, 1999. |
| Will cover Background, Goals, Design, and Futures |
| Why do we need this? |
| launch reads the command line to create the graph, from .so's |
| Connections (queues) are made by launcher, lots of switchout code |
| Argument handling is messy, at start-time only |
| ...thus there is basically only one state: running |
| There is no real master process capable of seeing the whole |
| pipeline as a graph, so wiring A->B->C with some backchannel |
| (parameter, not data stream) from C to A is hard |
| Channels try to do IPC, file, and network I/O, excess abstraction |
| Goals (1) |
| Provide a clean way to both build graphs and write new elements |
| Make things easier by providing auto-connect, stock sub-graphs |
| Include tools sorely lacking in OGI pipeline, like editor, saves |
| Enable Linux to catch up with M$'s world, allowing commercial |
| plugins to a stable API (as of 1.0) so we don't end up with N |
| wheels from N-M different people (multiple projects) |
| Overview (1) |
| Object hierarchy capable of run-time discovery, based on GtkObject |
| Deeply nested parent-child relationships enable threads, blackboxes |
| Buffers can point to anything, are typed, and can carry metadata |
| Plugins can be loaded at any point, and registry reduces loads |
| Symbiotic editor lets you design/run/save graphs visually |
| What are filter graphs? (1) |
| Filters take data in and spit data out, doing something to it |
| Filters have N>=0 inputs and M>=0 outputs |
| Filter graphs are many filters connected together, like a circuit |
| The goal is typically to move data from 'left' to 'right', towards |
| some kind of user-visible conclusion, i.e. audio or video |
| Architecture (3?) |
| - Graphs of Elements |
| (here lies screen-grab from editor) |
| Element is core Object, Bins hold (and are) Elements |
| Pads are fundamental to an Element, are cross-wired with pointers |
| Since Bins hold Elements, and Bins are Elements, Bins hold Bins |
| 'Ghostpads' provide interfaces for Bins without native interfaces |
| # Threads are type of Bin that actually run in separate threads |
| |
| - States |
| (table of states, invariants, and descriptions) |
| COMPLETE Element has all needed information |
| RUNNING Element has acquired resources, ready to go |
| DISCOVERY ... (unimplemented) |
| PREROLL ... (unimplemented) |
| PLAYING Element is actively trading data |
| PAUSED Special state where things just don't run (?..) |
| States are used to keep elements in check |
| |
| - Buffers |
| Buffers designed to be versatile, with arbitrary typing/metadata |
| Has pointer to data, length, so can point to someone else's data |
| Type system (crude so far) ensures buffers don't go stray |
| Metadata can be attached, such as the audio parameters |
| Ref-counting and copy-on-write avoids most copies, not complete |
| Sub-buffers can be created from bigger buffer, limitting copies |
| Gtk+ Object System (2) |
| - Pros |
| C-language object system, well tested (Gtk+, Gnome...) |
| Arguments of any fundamental type, read/write, built-in hooks |
| Signals used for hooks into object events, overridable |
| Run-time discovery of args, signals (quarks) |
| - Cons |
| No multiple-inheritance, though I haven't *needed* it |
| There are some holes (can't attach quarks to *eveything*) |
| |
| - Design |
| Classes, instances are structs; 1st item is parent class |
| Type system allows clean casting, ^^^^^^ |
| Arguments are set/get by string, use functions to do the work, |
| thus setting an arg might trigger a redraw of the GUI |
| Signals are strings, use marshallers, various firing methods |
| Basic GStreamer objects (1) |
| - Elements |
| (show class,instance structs) |
| Very simple, just provides a means to handle pads, state |
| - Bins |
| (show class,instance structs) |
| Supports children, handles group state transitions |
| Pads (1) |
| Pad list type, direction, and chaining function ptr |
| When creating a sink pad (!src) you set the chaining function |
| gst_pad_connect() sets the peers, and copies chain function to src |
| Passing buffer to a src pad transparently calls the chain function |
| (graph goes here...) |
| Sources (1) |
| Source provides functions to push data |
| Regular push() function just takes next N bytes and sends them |
| Async push_region() grabs N bytes at offset O and sends them |
| EOF signal [will] reset the state from PLAYING down to !RUNNING |
| "location" argument is global by convention, for filenames...URIs |
| Connections (1) |
| Special type of Filter that |
| Threads (1) |
| Special case of Bin that actually creates a thread transparently |
| When RUNNING, thread exists, mutex/cond used to go [!]PLAYING |
| Automatically determines how to start sub-graph |
| Looks for both Sources and Elements wired to Connection |
| Will cooperate with Pipelines when threading is not available |
| Typing and Metadata (1) |
| - Types |
| Based on MIME types, set up as quarks, and dealt with as int |
| Usable entirely at run-time, since they're registerable by plugins |
| - Metadata |
| Also registered as an int, but must be compile time due to structs |
| Have refcounts and CoW semantics, since they travel with buffers |
| Plugins (1) |
| Plugin architecture designed around class system |
| Arguments and signals provide interface over standard base class |
| Each Element defined by ElementFactory, which is queried by name |
| At plugin load, any number of ElementFactories and Types registered |
| Element registers against Type as source or sink |
| Editor (2+?) |
| (show filter graph snapshot, a different one, more complex) |
| Built as a parallel object hierarchy on top of GNOME Canvas |
| Every object in filter graph has equivalent in editor, plus some |
| Canvas is designed with groups and signal-propagation, so... |
| Why not build the whole thing as subclasses of CanvasGroup? |
| |
| ...because updates get messy/recursive (the way I *was* doing it) |
| Solution is to modify objects so they own Group rather than being |
| Relatively trivial modification, but requires lots of repointering |
| Still a genealogical mess of parents and children... |
| XML |
| The goal is to use XML heavily, with an eye towards DOM |
| Used for both saving and providing pre-build components |
| Both graph and editor will have namespace, they'll interleave |
| A generic save function will exist for Elements, with hooks |
| Saving an EditorElement will also save Element |
| Also used for a plugin registry, to avoid loading all plugins |
| |
| |
| |
| |
| leaky bucket is trivial |
| applications - generic conferencing tool (repluggable codecs), mixing |
| environment (data flow graphs) |