| Element types |
| ------------- |
| |
| SOURCES |
| ------- |
| |
| * never chain-based |
| * have no sinkpads |
| |
| 1) get based src |
| |
| (----------) |
| ! fakesrc ! |
| ! src- (get based) |
| (----------) |
| |
| * no sinkpads |
| * srcpad(s) that are get-based |
| |
| 2) loop based src |
| |
| (----------) |
| ! fakesrc ! |
| ! src- (loop based) |
| (----------) |
| |
| * no sinkpads |
| * element is loop-based |
| * data is pushed on the srcpad(s) |
| |
| FILTERS |
| ------- |
| |
| 3) chain based filter |
| |
| (----------) |
| ! identity ! |
| -sink src- |
| (----------) |
| |
| * sinkpad(s) have a chain function |
| * srcpad(s) push data |
| |
| 4) loop-based filter |
| |
| (----------) |
| ! identity ! |
| -sink src- |
| (----------) |
| |
| * element is loop-based |
| * data is pushed on the srcpads |
| * data is pulled from sinkpad(s) |
| |
| |
| SINKS |
| ----- |
| |
| 5) chain based sink |
| |
| (----------) |
| ! fakesink ! |
| -sink ! |
| (----------) |
| |
| * sinkpad(s) have a chain function |
| * no srcpads |
| |
| 6) loop-based sink |
| |
| (----------) |
| ! fakesink ! |
| -sink ! |
| (----------) |
| |
| * element is loop-based |
| * data is pulled from sinkpad(s) |
| |
| |
| DECOUPLED |
| --------- |
| |
| 7) decoupled element |
| |
| |
| (----------) |
| ! queue ! |
| -sink src- |
| (----------) |
| |
| * sinkpad(s) have chain function |
| * srcpad(s) have get function |
| * never loop-based |
| * always acts like a chain-based sink for upstream elements |
| * always acts like a get-based src for downstream elements |
| * are not added to a group, but marked as an entry point in |
| case it acts as a src element |
| |
| |
| Connection types |
| ---------------- |
| |
| 1) get based src |
| 2) loop based src |
| 3) chain based filter |
| 4) loop-based filter |
| 5) chain based sink |
| 6) loop-based sink |
| 7) decoupled element |
| |
| |
| ! 1 ! 2 ! 3 ! 4 ! 5 ! 6 ! 7 ! |
| ---+-----+-----+-----+-----+-----+-----+-----+ |
| 1 ! X ! X ! A ! C ! A ! C ! A ! |
| ! ! ! ! ! ! ! ! |
| 2 ! X ! X ! B ! F ! B ! F ! B ! |
| ! ! ! ! ! ! ! ! |
| 3 ! X ! X ! D ! E ! D ! E ! D ! |
| ! ! ! ! ! ! ! ! |
| 4 ! X ! X ! B ! F ! B ! F ! B ! |
| ! ! ! ! ! ! ! ! |
| 5 ! X ! X ! X ! X ! X ! X ! X ! |
| ! ! ! ! ! ! ! ! |
| 6 ! X ! X ! X ! X ! X ! X ! X ! |
| ! ! ! ! ! ! ! ! |
| 7 ! X ! X ! A ! C ! A ! C ! X ! |
| ! ! ! ! ! ! ! ! |
| |
| |
| A) |
| |
| src -> sink |
| src -> filter |
| src -> decoupled |
| decoupled -> sink |
| decoupled -> filter |
| |
| - get based source |
| - chain based sink |
| |
| * one group |
| * src at start of group and entry point |
| * _get from src, push to sink |
| |
| (-group1---------------) |
| ! ! |
| ! *fakesrc -> fakesink ! |
| (----------------------) |
| |
| |
| B) |
| |
| src -> sink |
| src -> filter |
| src -> decoupled |
| filter -> sink |
| filter -> filter |
| filter -> decoupled |
| |
| - loop based source/filter |
| - chain based sink/filter/decoupled |
| |
| * one group |
| * src/filter at start of group and entry point |
| * loop on src, chainhandler set to chain function |
| |
| (-group1----------------) |
| ! ! |
| ! %*fakesrc -> fakesink ! |
| (-----------------------) |
| |
| |
| C) |
| |
| src -> sink |
| src -> filter |
| decoupled -> sink |
| decoupled -> filter |
| |
| - get based source/decoupled |
| - loop based sink/filter |
| |
| * one group |
| * loop based element is entry point |
| * loop on sink/filter, gethandler set to getfunction |
| |
| (-group1----------------) |
| ! ! |
| ! fakesrc -> %*fakesink ! |
| (-----------------------) |
| |
| D) |
| |
| filter -> filter |
| filter -> sink |
| filter -> decoupled |
| |
| - chain based filter |
| - chain based filter/sink/decoupled |
| |
| * one group is created to hold the two elements |
| * no entry point |
| * chainhandler set to peer chainfunction |
| |
| (-group1----------------) |
| ! ! |
| ! identity -> identity ! |
| (-----------------------) |
| |
| E) |
| |
| filter -> filter |
| filter -> sink |
| |
| - chain based filter |
| - loop based filter/sink |
| |
| * two groups |
| * group is created for src element if needed |
| * chainhandler of loop based element set to loop wrapper, control is |
| handed to the peer group |
| * gethandler of loop based element set to get wrapper |
| |
| (-group1---) (-group2------) |
| ! ! ! ! |
| ! identity ---> %*identity ! |
| (----------) (-------------) |
| |
| |
| F) |
| |
| src -> filter |
| src -> sink |
| filter -> filter |
| filter -> sink |
| |
| - loop based filter/src |
| - loop based filter/sink |
| |
| * two groups |
| * two entry points |
| * chainhandler set to loop wrapper |
| * gethandler set to get wrapper |
| |
| (-group1-----) (-group2------) |
| ! ! ! ! |
| ! %*identity ---> %*identity ! |
| (------------) (-------------) |
| |
| |
| Grouping |
| -------- |
| |
| * a group has at most one loop based element |
| * elements in a group are sorted, src elements first (not mandatory) |
| * a group has one cothread |
| * a group is created immediatly for loop based elements, all other elements |
| are added to a group when a pad connection is made |
| * get-based plugins are put in the same group as a peer loop based element |
| * chain based elements are put in the same group as sink peer elements |
| * entry point in the group is: |
| - loopbased element |
| - first src element if no loopbased element exists in the group |
| |
| Result: you end up with a group of connected elements with either: |
| - a loop based plugin as the entry point |
| - a get based plugin as the entry point |
| |
| Scheduling the group is a matter of starting the cothread and calling |
| the loop function or doing a _get/_push on a srcpad. |
| |
| |
| other examples of groups: |
| ------------------------- |
| |
| % = loop based |
| * = entry point of group |
| |
| . |
| (-group1---------------) |
| ! ! |
| ! *fakesrc -> fakesink ! |
| (----------------------) |
| |
| . |
| (-group1---------------------------------------) |
| ! ! |
| ! *fakesrc -> identity -> identity -> fakesink ! |
| (----------------------------------------------) |
| |
| . |
| (-group1-----------------------------------------) |
| ! ! |
| ! fakesrc -> %*identity -> identity -> fakesink ! |
| (------------------------------------------------) |
| |
| . |
| (-group1---------------) (-group2-----------------) |
| ! ! ! ! |
| ! *fakesrc -> identity --> *%identity -> fakesink ! |
| (----------------------) (------------------------) |
| |
| . |
| (-group1------------------------------------) |
| ! ! |
| ! *fakesrc -> tee --> identity -> fakesink ! |
| ! --> identity -> fakesink ! |
| (-------------------------------------------) |
| |
| . |
| (-group1------------------------------------) |
| ! ! |
| ! *fakesrc -> tee --> identity -> fakesink ! |
| (--------------!----------------------------) |
| v |
| (-group2-----------------) |
| ! ! |
| ! *%identity -> fakesink ! |
| (------------------------) |
| |
| . |
| (-group1----------) (-group2-----------------) |
| ! ! ! ! |
| ! *fakesrc -> tee --> *%identity -> fakesink ! |
| (--------------!--) (------------------------) |
| v |
| (-group3-----------------) |
| ! ! |
| ! *%identity -> fakesink ! |
| (------------------------) |
| |
| . |
| (-group1-----------------------) (-group2----------------------) |
| ! ! ! ! |
| ! filesrc -> *%mpegdemux --> queue* -> mpeg2dec -> xvideosink ! |
| ! ! (-----------------------------) |
| ! ! (-group3----------------------) |
| ! ! ! ! |
| ! --> queue* -> mad -> osssink ! |
| (------------------------------) (-----------------------------) |
| |
| |
| Chaining |
| -------- |
| |
| * groups that are connected end up in the same chain |
| * a group always belongs to a chain |
| * updating the chain is only needed when two groups are |
| connected with a connection of type E/F. for other |
| connection types, the group itself is updated. |
| * a chain is scheduled by scheduling a random group in the chain. |
| |
| |
| Wrapper functions |
| ----------------- |
| |
| |
| |
| iterating without cothreads |
| --------------------------- |
| |
| A cothread for each group is the easiest way to schedule most of |
| the pipelines. Some pipelines are however schedulable without |
| any cothreads. |
| |
| Each group is schedulable without cothreads, one can call the |
| group schedule function and be done with it. Problems arise |
| one the group boundaries of connected elements, which are always |
| of type E and F (chain->loop, loop->loop) |
| |
| We always have a producer group and a provider group in this case. |
| |
| chain->loop |
| ----------- |
| |
| |
| |
| Scheduler algorithm |
| ------------------- |
| |
| 1: select (random?) group in chain |
| 2: schedule group |
| 3: on E/F connections, the get/chain wrapper is called |
| - get wrapper puts the peer element on the runqueue and |
| recursively invokes the scheduler. |
| - chain wrapper puts the buffer in the bufpen and puts |
| the peer element in the runqueue |
| 4: when the group is scheduled, take group from the runqueue |
| and goto 2: |
| 5: no more groups on the runqueue, iteration ends |
| |
| |
| NOTES: |
| |
| - We need a GList instead of a single bufpen to hold buffers |
| for multi-out elements. |
| |
| - We probably need to set a limit on the maximum number of |
| recursions and size of the bufpen list. |
| |
| - elements run non-preemptively, a group is done scheduling when all |
| elements in that group complete their chain/loop function. |
| |
| - can we only have a stack overflow when there is a loop in the |
| pipeline? I think so. |
| |
| - putting groups twice on the runqueue is not a good idea, we |
| need to check a flag or something, maybe give the group a |
| higher priority? |
| |
| - what about starvation? We'll probably have to put the group |
| at the end of the runqueue. |
| |
| - multi-out elements can introduce significant latency. consider: |
| |
| (-group1----------) (-group2-----------------) |
| ! ! ! ! |
| ! *filesrc -> mad --> *%identity -> osssink ! |
| (-----------------) (------------------------) |
| |
| mad produces N output buffers, they are queued in the bufpen, when |
| group1 is done, group2 is scheduled to empty the bufpen queue. |
| |
| The time it takes for the first buffer to arrive at osssink equals |
| the time needed to decode the N-1 other buffers. |
| |
| Of course, given the right buffersize for filesrc, N can be reduced |
| to 1. mad can also suggest a buffersize to filesrc with the |
| BUFFER_SIZE event. |
| |
| |
| |
| Ungrouping |
| ---------- |
| |
| Ungrouping might be the hardest part. especially in the case where |
| an element is still running (state changes and pipeline modifications |
| in element callbacks). |
| |
| Ungrouping involves two tasks: |
| |
| - breaking up groups. This can happen when a pad connection is broken |
| so that the group contains two clusters of unconnected elements. |
| - breaking up chains. This happens when a pad connection is broken |
| so that the chains contains two clusters of unconnected groups. |
| |
| |
| case1 |
| ----- |
| |
| The most simple case is where two elements are disconnected and one |
| of the elements does not belong to a group. In this case, we don't need |
| to do anything at all. |
| |
| case2 |
| ----- |
| |
| if the elements are part of different groups, we need to check if the |
| chain needs to be broken. A chain is broken if it contains two sets |
| of unconnected groups. |
| |
| To test this case, we will create a new chain and recursively move |
| one of the groups with all of its connected groups to it. If the |
| origial chain is empty after this operation, there was still a connection |
| and the new chain replaces the old one, else we end up with two chains. |
| |
| case3 |
| ----- |
| |
| When the elements are part of the same group we check if both elements |
| still have a connection to some other element in that group. The elements |
| without connections are removed from the group. |
| |
| It is possible that when an element still has a connection with some other |
| element in the group, the group has to be split up anyway. This can happen |
| in fakesrc ! indentity ! identity ! fakesink when we break the connection |
| between the two identity elements. We have to be careful here in the cothread |
| case that we don't take away the running cothread from under the elements. |
| In the non-cothread case we can just move the elements to another new group. |
| |
| |
| Interrupt with cothreadless optimal scheduler |
| --------------------------------------------- |
| |
| Interrupts are usually performed when a blocking _get based source or |
| decoupled element is unlocked for a state change. The idea of the |
| interrupt scheduler call is to return to the main execution stack frame |
| ASAP so that the state change can take place. For cothread based |
| implementations of the scheduler this is not a problem as one can jump |
| to the main cothread context without problems. For non cothread based |
| schedulers we need to follow this scheme: |
| |
| get <-> chain based connection. |
| |
| - enter get group scheduler |
| - call _get function on source, this can block internally |
| - app performs state change, element is told to unlock itself |
| - lock inside the get based function unlocks and _get function |
| return GST_EVENT_INTERRUPT |
| - group scheduler notices the event, unrefs it and jumps out of the |
| get_group_scheduler function, a scheduler INTERRUPTED flag is set. |
| |
| get <-> loop based connection |
| |
| - enter loop group scheduler |
| - loop based element does pull on sinkpad |
| - _get blocks |
| - app performs state change, _get is unlocked and returns |
| GST_EVENT_INTERRUPT. |
| - loop based function receives INTERRUPT event and exits its loop |
| ASAP using gst_element_interrupt. |
| |
| loop/chain <-> loop based connection |
| |
| - when returning from the recursive call to the scheduler, the state |
| of the scheduler is checked, if it was interrupted, a |
| GST_EVENT_INTERRUPTED event is returned to the loop based element. |
| - the loop based element exits its loop ASAP. |
| |
| |
| This technique will unwind the stack of scheduled groups ASAP and returns |
| to the main execution stack frame where the iterate() function can return |
| and the state change can take place. |
| |
| Another alternative could be implemented when the _push and _pull functions |
| would return a result code... |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
