| Probes |
| ------ |
| |
| Probes are callbacks that can be installed by the application and will notify |
| the application about the states of the dataflow. |
| |
| |
| Requirements |
| ------------ |
| |
| Applications should be able to monitor and control the dataflow on pads. We |
| identify the following types: |
| |
| - be notified when the pad is/becomes idle and make sure the pad stays idle. |
| This is essential to be able to implement dynamic relinking of elements |
| without breaking the dataflow. |
| |
| - be notified when data, events or queries are pushed or sent on a pad. It |
| should also be possible to inspect and modify the data. |
| |
| - be able to drop, pass and block on data based on the result of the callback. |
| |
| - be able to drop, pass data on blocking pads based on methods performed by |
| the application thread. |
| |
| |
| Overview |
| -------- |
| |
| The function gst_pad_add_probe() is used to add a probe to a pad. It accepts a |
| probe type mask and a callback. |
| |
| gulong gst_pad_add_probe (GstPad *pad, |
| GstPadProbeType mask, |
| GstPadProbeCallback callback, |
| gpointer user_data, |
| GDestroyNotify destroy_data); |
| |
| The function returns a gulong that uniquely identifies the probe and that can |
| be used to remove the probe with gst_pad_remove_probe(): |
| |
| void gst_pad_remove_probe (GstPad *pad, gulong id); |
| |
| The mask parameter is a bitwise or of the following flags: |
| |
| typedef enum |
| { |
| GST_PAD_PROBE_TYPE_INVALID = 0, |
| |
| /* flags to control blocking */ |
| GST_PAD_PROBE_TYPE_IDLE = (1 << 0), |
| GST_PAD_PROBE_TYPE_BLOCK = (1 << 1), |
| |
| /* flags to select datatypes */ |
| GST_PAD_PROBE_TYPE_BUFFER = (1 << 4), |
| GST_PAD_PROBE_TYPE_BUFFER_LIST = (1 << 5), |
| GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM = (1 << 6), |
| GST_PAD_PROBE_TYPE_EVENT_UPSTREAM = (1 << 7), |
| GST_PAD_PROBE_TYPE_EVENT_FLUSH = (1 << 8), |
| GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM = (1 << 9), |
| GST_PAD_PROBE_TYPE_QUERY_UPSTREAM = (1 << 10), |
| |
| /* flags to select scheduling mode */ |
| GST_PAD_PROBE_TYPE_PUSH = (1 << 12), |
| GST_PAD_PROBE_TYPE_PULL = (1 << 13), |
| |
| } GstPadProbeType; |
| |
| When adding a probe with the IDLE or BLOCK flag, the probe will become a |
| blocking probe (see below). Otherwise the probe will be a DATA probe. |
| |
| The datatype and scheduling selector flags are used to select what kind of |
| datatypes and scheduling modes should be allowed in the callback. |
| |
| The blocking flags must match the triggered probe exactly. |
| |
| The probe callback is defined as: |
| |
| GstPadProbeReturn (*GstPadProbeCallback) (GstPad *pad, GstPadProbeInfo *info, |
| gpointer user_data); |
| |
| A probe info structure is passed as an argument and its type is guaranteed |
| to match the mask that was used to register the callback. The data item in the |
| info contains type specific data, which is usually the data item that is blocked |
| or NULL when no data item is present. |
| |
| The probe can return any of the following return values: |
| |
| typedef enum |
| { |
| GST_PAD_PROBE_DROP, |
| GST_PAD_PROBE_OK, |
| GST_PAD_PROBE_REMOVE, |
| GST_PAD_PROBE_PASS, |
| } GstPadProbeReturn; |
| |
| GST_PAD_PROBE_OK is the normal return value. DROP will drop the item that is |
| currently being probed. GST_PAD_PROBE_REMOVE the currently executing probe from the |
| list of probes. |
| |
| GST_PAD_PROBE_PASS is relevant for blocking probes and will temporarily unblock the |
| pad and let the item trough, it will then block again on the next item. |
| |
| |
| Blocking probes |
| --------------- |
| |
| Blocking probes are probes with BLOCK or IDLE flags set. They will always |
| block the dataflow and trigger the callback according to the following rules: |
| |
| When the IDLE flag is set, the probe callback is called as soon as no data is |
| flowing over the pad. If at the time of probe registration, the pad is idle, |
| the callback will be called immediately from the current thread. Otherwise, |
| the callback will be called as soon as the pad becomes idle in the streaming |
| thread. |
| |
| The IDLE probe is useful to perform dynamic linking, it allows to wait for for |
| a safe moment when an unlink/link operation can be done. Since the probe is a |
| blocking probe, it will also make sure that the pad stays idle until the probe |
| is removed. |
| |
| When the BLOCK flag is set, the probe callback will be called when new data |
| arrives on the pad and right before the pad goes into the blocking state. This |
| callback is thus only called when there is new data on the pad. |
| |
| The blocking probe is removed with gst_pad_remove_probe() or when the probe |
| callback return GST_PAD_PROBE_REMOVE. In both cases, and if this was the last |
| blocking probe on the pad, the pad is unblocked and dataflow can continue. |
| |
| |
| Non-Blocking probes |
| -------------------- |
| |
| Non-blocking probes or DATA probes are probes triggered when data is flowing |
| over the pad. The are called after the blocking probes are run and always with |
| data. |
| |
| |
| Push dataflow |
| ------------- |
| |
| Push probes have the GST_PAD_PROBE_TYPE_PUSH flag set in the callbacks. |
| |
| In push based scheduling, the blocking probe is called first with the data item. |
| Then the data probes are called before the peer pad chain or event function is |
| called. |
| |
| The data probes are called before the peer pad is checked. This allows for |
| linking the pad in either the BLOCK or DATA probes on the pad. |
| |
| Before the peerpad chain or event function is called, the peer pad block and |
| data probes are called. |
| |
| Finally, the IDLE probe is called on the pad after the data was sent to the |
| peer pad. |
| |
| The push dataflow probe behavior is the same for buffers and bidirectional events. |
| |
| |
| pad peerpad |
| | | |
| gst_pad_push() / | | |
| gst_pad_push_event() | | |
| -------------------->O | |
| O | |
| flushing? O | |
| FLUSHING O | |
| < - - - - - - O | |
| O-> do BLOCK probes | |
| O | |
| O-> do DATA probes | |
| no peer? O | |
| NOT_LINKED O | |
| < - - - - - - O | |
| O gst_pad_chain() / | |
| O gst_pad_send_event() | |
| O------------------------------>O |
| O flushing? O |
| O FLUSHING O |
| O< - - - - - - - - - - - - - - -O |
| O O-> do BLOCK probes |
| O O |
| O O-> do DATA probes |
| O O |
| O O---> chainfunc / |
| O O eventfunc |
| O< - - - - - - - - - - - - - - -O |
| O | |
| O-> do IDLE probes | |
| O | |
| < - - - - - - O | |
| | | |
| |
| |
| Pull dataflow |
| ------------- |
| |
| Pull probes have the GST_PAD_PROBE_TYPE_PULL flag set in the callbacks. |
| |
| The gst_pad_pull_range() call will first trigger the BLOCK probes without a DATA |
| item. This allows the pad to be linked before the peer pad is resolved. It also |
| allows the callback to set a data item in the probe info. |
| |
| After the blocking probe and the getrange function is called on the peer pad |
| and there is a data item, the DATA probes are called. |
| |
| When control returns to the sinkpad, the IDLE callbacks are called. The IDLE |
| callback is called without a data item so that it will also be called when there |
| was an error. |
| |
| If there is a valid DATA item, the DATA probes are called for the item. |
| |
| |
| srcpad sinkpad |
| | | |
| | | gst_pad_pull_range() |
| | O<--------------------- |
| | O |
| | O flushing? |
| | O FLUSHING |
| | O - - - - - - - - - - > |
| | do BLOCK probes <-O |
| | O no peer? |
| | O NOT_LINKED |
| | O - - - - - - - - - - > |
| | gst_pad_get_range() O |
| O<------------------------------O |
| O O |
| O flushing? O |
| O FLUSHING O |
| O- - - - - - - - - - - - - - - >O |
| do BLOCK probes <-O O |
| O O |
| getrangefunc <---O O |
| O flow error? O |
| O- - - - - - - - - - - - - - - >O |
| O O |
| do DATA probes <-O O |
| O- - - - - - - - - - - - - - - >O |
| | O |
| | do IDLE probes <-O |
| | O flow error? |
| | O - - - - - - - - - - > |
| | O |
| | do DATA probes <-O |
| | O - - - - - - - - - - > |
| | | |
| |
| |
| Queries |
| ------- |
| |
| Query probes have the GST_PAD_PROBE_TYPE_QUERY_* flag set in the callbacks. |
| |
| |
| pad peerpad |
| | | |
| gst_pad_peer_query() | | |
| -------------------->O | |
| O | |
| O-> do BLOCK probes | |
| O | |
| O-> do QUERY | PUSH probes | |
| no peer? O | |
| FALSE O | |
| < - - - - - - O | |
| O gst_pad_query() | |
| O------------------------------>O |
| O O-> do BLOCK probes |
| O O |
| O O-> do QUERY | PUSH probes |
| O O |
| O O---> queryfunc |
| O error O |
| <- - - - - - - - - - - - - - - - - - - - - - -O |
| O O |
| O O-> do QUERY | PULL probes |
| O< - - - - - - - - - - - - - - -O |
| O | |
| O-> do QUERY | PULL probes | |
| O | |
| < - - - - - - O | |
| | | |
| |
| For queries, the PUSH ProbeType is set when the query is traveling to the object |
| that will answer the query and the PULL type is set when the query contains the |
| answer. |
| |
| Use-cases |
| --------- |
| |
| Prerolling a partial pipeline |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| .---------. .---------. .----------. |
| | filesrc | | demuxer | .-----. | decoder1 | |
| | src -> sink src1 ->|queue|-> sink src |
| '---------' | | '-----' '----------' X |
| | | .----------. |
| | | .-----. | decoder2 | |
| | src2 ->|queue|-> sink src |
| '---------' '-----' '----------' X |
| |
| |
| The purpose is to create the pipeline dynamically up to the |
| decoders but not yet connect them to a sink and without losing |
| any data. |
| |
| To do this, the source pads of the decoders is blocked so that no |
| events or buffers can escape and we don't interrupt the stream. |
| |
| When all of the dynamic pad are created (no-more-pads emitted by the |
| branching point, ie, the demuxer or the queues filled) and the pads |
| are blocked (blocked callback received) the pipeline is completely |
| prerolled. |
| |
| It should then be possible to perform the following actions on the |
| prerolled pipeline: |
| |
| - query duration/position |
| - perform a flushing seek to preroll a new position |
| - connect other elements and unblock the blocked pads. |
| |
| |
| dynamically switching an element in a PLAYING pipeline |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| |
| .----------. .----------. .----------. |
| | element1 | | element2 | | element3 | |
| ... src -> sink src -> sink ... |
| '----------' '----------' '----------' |
| .----------. |
| | element4 | |
| sink src |
| '----------' |
| |
| The purpose is to replace element2 with element4 in the PLAYING |
| pipeline. |
| |
| 1) block element1 src pad. |
| 2) inside the block callback nothing is flowing between |
| element1 and element2 and nothing will flow until unblocked. |
| 3) unlink element1 and element2 |
| 4) optional step: make sure data is flushed out of element2: |
| 4a) pad event probe on element2 src |
| 4b) send EOS to element2, this makes sure that element2 flushes |
| out the last bits of data it holds. |
| 4c) wait for EOS to appear in the probe, drop the EOS. |
| 4d) remove the EOS pad event probe. |
| 5) unlink element2 and element3 |
| 5a) optionally element2 can now be set to NULL and/or removed from the |
| pipeline. |
| 6) link element4 and element3 |
| 7) link element1 and element4 |
| 8) make sure element4 is in the same state as the rest of the elements. The |
| element should at least be PAUSED. |
| 9) unblock element1 src |
| |
| The same flow can be used to replace an element in a PAUSED pipeline. Of |
| course in a PAUSED pipeline there might not be dataflow so the block might |
| not immediately happen. |
