docs/random/bbb/streamselection

Stream selection
=

1. Problem
URIs (that being either a media stream or a media stream plus subtitle) can
contain multiple streams of a type (audio, subtitle). A user has to be given
the option of selecting one of those streams (or none altogether).

2. Implementation ideas
Stream selection, in GStreamer, has to be integrated at the player plugging
level, which is (in the case of Totem) playbin. Playbin offers a feature to
'mute' a stream (which means that no processing is done on that stream
altogether, saving the decoding step). Currently, playbin will select the
first occurrence of a stream type and mute all others. A queue is connected
(for pre-roll) to the active stream. What is missing here is a way to change
the active stream.
Playbin interface - one possible interface could simply consist of a bunch of
GObject properties: 'textstream' and 'audiostream', both integer. The number
of available streams can be retrieved using the 'stream-info' GObject property.
Similar to the 'nstreams' property, we could add utility GObject properties
for getting the number of available audio/text streams ('naudiostreams' and
'ntextstreams'). Names of these streams (like language name or so) can be
added as an additional GObject property to streaminfo. Some container
formats contain such names internally. Alternatively, we could allow those
to be user-settable as well (for .sub files).
On a set of either of these properties, playbasebin would mute the old
selected stream (if any), unmute the newly selected stream (if any) and
replug the preroll queue. The queue itself is disabled as well if no new
stream was linked. Alternatively, a switch-like element is used, which
requires no replugging. Pad disabling/enabling is then enough. This also
makes relinking less painful. The switch-like element needs to proxy the
active pads' caps. However, since those caps are (in practice) always the
same across streams, caps setting will (inside the core) immediately
return success.
The switch-like element simply works like this:
=
static void
loop_func (GstElement * element)
{
  GList *inpads;
  GstPad *pad;
  GstData *data;

  for (inpads = ..; inpads != NULL; inpads = inpads->next) {
    pad = inpads->data;
    if (!GST_PAD_IS_USABLE (pad))
      continue;

    /* you'd also want some way to synchronize the inputs... */
    data = gst_pad_pull (pad);
    if (is_active_pad (pad))
      gst_pad_push (srcpad, data);
    else
      gst_data_unref (data);
  }
}
=
It'd require an active-stream property itself, which (when set) takes
care of doing renegotiation and so on. Using internal pad linkage is
extremely useful here, and requires little code in the switch-like
element itself. Note that there is a slight bit of duplication in the
playbin interface and the switch-like element interface, but that's "just
the way it is".
The implemention of the switch-like element could initially be local to
playbin, until it has been cleaned up and confirmed to be useful to a
wider audience. This allows a lot of experimenting with interfaces because
we won't be forced to maintain a stable interface.
The current 'switch' element (gst-plugins/gst/switch/) already does a few
of those operations, but stream synchronization, re-negotiation on stream
changes, internal pad linkage and some other things are completely missing.
If we're gonna use this element, it'll need a large overhaul. The choice of
writing a new element or using an existing element as basis, and also the
choice of whether or not to make this element local to playbin, should be
based on technical merits and cost/effect analysis and not on personal
pride.

Notes:
* seamless has the same switch-like element, but it's chain-based. Apart
from scheduler considerations, this is a good idea, but limits its use
(making either good docs and abuse-prevention [see multifilesrc] or
private-to-playbin a prerequisite).
* maybe text-* properties need to be renamed to subtitle-*.

3. Written by
Ronald S. Bultje <rbultje@ronald.bitfreak.net> - Jan. 2nd, 2005.