docs/random/wtay/interactivity

This document contains some ideas that can be used to implement
interactivity in media content.

Possible application: DVD navigation, flash,...

Requirements
------------

- capture mouse clicks, mouse position, movement occuring on
  a video display plugin
- transport these events to the interested plugins
- allow for automation (ie, the technique should work without
  a video plugin too)
- the core doesn't care

Capturing events
----------------

- the videosink element captures mouse events
  - event is encapsulated into a generic data structure
    describing the event (need to define a caps?)
  - event is signalled to the app?.
  - event is sent upstream?

  * videosink has to add something to the main_loop to
    be able to grab events
  * thread issues?
  * does the app need to know about the events?
  
- app captures mouse events
  - no idea if that's possible 
  - app sends events upstream

Sending events to plugins
-------------------------

- are sent upstream using the event methods
  * more generic
  * less app control

- are sent to the appropriate plugin by the app
  * app needs to know what plugins are interested,
    less generic.
  * more app control

automation will always work, the app can construct navigation
events and insert them into the pipeline.

What about flushing to minimize latency?


Defining an event
-----------------

some ideas:

GST_CAPS_NEW (
   "videosink_event",
   "application/x-gst-navigation"
     "type", "click",
     "x_pos", 30,
     "y_pos", 40
   )
     
GST_CAPS_NEW (
   "videosink_event",
   "application/x-gst-navigation"
     "type", "move",
     "x_pos", 30,
     "y_pos", 40
   )
     
...

do we need a library for this?

do we use custom events and use the mime type to detect the 
type? do we creat a GST_EVENT_NAVIGATION?

can we encapsulate all events into a GstCaps? I would think so

Random thoughts
---------------

- we're basically defining an event model, I don't think there is
  anything wrong with that.
- how is our coordinate system going to work? do
  we use normalized values, 0-1000000 (or floats)
  or real pixel values? real pixel values require scalers to adjust
  the values (I don't think I like that)