RTP client subsystem proposal.
Ramon Garcia Fernandez

A brief description of the RTP protocol.

The RTP protocol uses two connections: one for passing data and another for control. The control connection is used for starting, finishing, and passing statistics of lost packets. On the data connection, packets are transmitted containing the media. These packets have a 7 bit field that says what codec is the data transmitted with. This codec is variable. However data cannot change from audio to video in packets of the same connection. All the packets belong to the same logical stream, that is, for instanace, to the same speech, or to the same song. Different codecs in the same stream is used, for instance, to insert comfort noise.

Each codec is packeted in a specific way in RTP packets. This is necessary to minimize the damage produced by lost packets. Therefore, packets should be arranged so that each packet can be decoded independently. If a packet is lost, that shouldn't preclude the decoding of following packets.


Suggested implementation:

Tbe implementation that I suggest contains an rtpdec element. This element has one sink pad for the data, one src pad for the decoded data, and a pair of src and sink pads for control messages. The decoded data that comes from rtpdec has no RTP dependency. It is in the format expected by the codec that it decodes it. Therefore, rtpdec must do codec specific processing of data to take into account that different codecs are stored differently in RTP.

A posible pipeline is:

udpcon name="data" udpcon name="control" data.src!rtpdec.datasrc  control.src!rtpdec.controlin control.sink!rtpdec.controlout rtpdec name="rtpdec" ! mp3decode ! osssink

udpcon is an element (not yet written) that provides a src and a sink pad to read and write to a socket. There are two udpcon, one for reading the data socket and another for reading and writting to the control socket.

To handle variable types, the following would work (the UDP part left out for clarity is identical to the one above):

[udp] rtpdec ! spider ! osssink

rtpdec has a property that tells it the mapping of the payload bits to mime type. This is necessary, because that mapping is not fixed. It has to be obtained at runtime through other mechanisms, such as a SDP file obtained by http or rtsp.

A posible implementation of rtpdec is a bin that contains two elements: a media independent _rtpparse and a media dependent rtp-dec-mp3. Thus the pipeline would be

udpsrc ! _rtpparse ! rtp-dec-mp3 ! mp3decode ! osssink
       \________________________/
             rtpdec bin

Another posible implementation is, that rtpdec opens a plugin that contains the code to convert the codec specific packets into the input that the normal decoder for that codec expects.

Ronald said that this mechanism would be complicated because it would require to duplicate the functionality of the plugin loader. It shouldn't be like that because it should use the normal plugin loader. However, this is an issue that I do not fully understand.

Ronald suggested to use inheritance. Thus the user should insert a codec specific rtp element in the pipeline. Such as:

udpsrc ! rtp-mp3 ! mp3decode ! osssink

Reuse of RTP logic would be achieved through inheritance.

This looks more logical, because inheritance reflects the fact that rtp-mp3 "is an" specialization of rtp. However, there are several issues. As stated above, it is posible in RTP to switch the encoding of the media at any time. If this happens, some state must be kept, such as statistics of packets received and sent.