Last update: 2011/08/21
A comprehensive framework to deal with interactions between real and virtual worlds
Digital Media started with the digitisation of audio, video and images. But Digital Media is about more thanaudio, video and images. Implementing this plan has been the mission of MPEG-4 which has been adding more and more media types: composition, characters and font information, 3D Graphics etc. Other standards, like MPEG-7 and MPEG-21 have added more aspects of Digital Media. 3D Graphics, in particular, has led to very significant examples of "virtual worlds" some giving rise to new businesses, as in the case of much-hyped Second Life where people, numbering millions at one time, can create their own alter ego and create/buy/sell assets.
Digital media content may also need to stimulate senses other than sight and hearing involved in traditional audio-visual media, e.g. olfaction, mechanoreception (i.e. sensory reception of mechanical pressure ), equilibrioception (i.e. sensory reception of balance ), or thermoception (i.e. sensory reception of themperature). These additions to the audio-visual content (movies, games etc.) involve other senses that enhance the feeling of being part of the media content to create a user experience that is expected to have a greater value for the user.
Virtual worlds are used in a variety of contexts (entertainment, education, training, getting information, social interaction, work, virtual tourism, reliving the past ecc.) and have attracted interest as important enablers of new business models, services, applications and devices as they offer the means to redesign the way companies interact with other entities (suppliers, stakeholders, customers, etc.).
The “Media context and control” standard, nicknamed MPEG-V [11], provides an architecture, illustrated in Fig. 5, and the digital representation of a wide range of data types that enable interoperability between virtual worlds – e.g., a virtual world set up by a provider of serious games and simulations – with another virtual world. Of course both virtual worlds may have interactions with the real world, via sensors and actuators.
Fig. 1 – The general MPEG-V model for real and virtual world interaction
In the MPEG-V model described in Fig. 1 the virtual world at the left hand side receives Sensor Effects that have been generated by some sensor and generates Sensory Effects that eventually control some actuators. In general, however, the format of the information entering or leaving the virtual world is different than the one used by actuators and sensors, respectively. For example, the virtual world may wish to communicate a “cold” feeling but the real world actuators may have different physical means to realise "cold". The user can specify how Sensory Effects should be mapped to Device Commands and Devices can represent their capabilities: for instance, it is not possible to change the room temperature, but it is possible to activate a fan.
Fig. 2 provides a specific example illustrating some technologies specified in MPEG-V Part 3 “Sensory Information”, namely 1) the Sensory Effect Description Language (SEDL) enabling the description of “sensory effects” (e.g. light, wind, fog, vibration, etc.) that act on human senses, 2) the Sensory Effect Vocabulary (SEV) defining the actual sensory effects and allowing for extensibility and flexibility and 3) Sensory Effect Metadata (SEM), a specific SEDL description that may be associated to any kind of multimedia content to drive sensory devices (e.g. fans, vibration chairs, lamps, etc.).
Fig. 2 – Use of MPEG-V SEDL for augmented user experience
