In its quest to provide the necessary standards for the Digital Media industry, MPEG started the MPEG Multimedia Middleware (M3W), a complete set of standards defining the software environment of a multimedia device. When responses to the M3W CfP were received MPEG selected the proposal coming from the Universal Home API (UHAPI) consortium that had developed a pretty complete solution.
M3W is ISO/IEC 23004 and nicknamed MPEG-E. It is organised in eight parts
Part 1 “Architecture” describes the M3W architecture and APIs. In M3W there are 3 distinct layers:
- Applications
- Middleware consisting of M3W and other middleware. M3W can be separated into two parts
- Functional providing the applications and other middleware with a multimedia platform. (‘Functional’ indicates multimedia ‘functionality’)
- Extra-functional providing the means to manage the lifetime of, and interaction with, realisation entities. Furthermore this enables management of extra-functional (‘support’) properties, e.g. resource management, fault management and integrity management;
- Computing platform: the API is outside of the scope of M3W.
Fig. 1 – The software structure of an M3W system
The scope of M3W is limited to the specification of the M3W API and realisation technology. In Fig. 1, this means the L-shaped yellow part and its interfaces depicted by the green line.
Part 2 “Multimedia API” specifies access to the functionalities provided by conforming multimedia platforms such as Media Processing Services (including coding, decoding and trans-coding), Media Delivery Services (through files, streams, messages), Digital Rights Management (DRM) Services, Access to data (e.g. media content), and Access to, Edit and Search Metadata.
Part 3 “Component Model” specifies a technology enabling cost effective software development and an increase in productivity through software reuse and easy software integration.
Part 4 “Resource and Quality Management” specifies a framework for resource management aiming to optimise and guarantee the Quality of Service that is delivered to the end-user in a situation where resources are constrained.
Part 5 “Component Download” specifies a download framework enabling controlled download of software components to a device.
Part 6 “Fault Management” specifies a framework for fault management with the goal to have a dependable/reliable system in the context of faults. These can be introduced due to upgrades and extensions out of the control of the device vendor, or because it is impossible to test all traces and configurations in today’s complex software systems.
Part 7 “System Integrity Management” specifies a framework for integrity management with the goal to have controlled upgrading and extension, in the sense that there is a reduced chance of breaking the system during an upgrade/extension or to provide the ability to restore a consistent configuration.
Part 8 “Reference Software and Conformance” is the usual complement as with the other MPEG standards.
Part 2 of M3W does not define a multimedia API but how one can be called from an M3W environment. The job of ISO/IEC 23006 Multimedia Service Platform Technologies ( MPEG-M) standard is to provide that multimedia API and more.
Let’s start from Fig. 2, used as a reference in Part 1 “Architecture“.
Fig. 2 – The MPEG-M architecture
When an Application calls the API defined in part 2 of MPEG-M “MPEG Extensible Middleware” (MXM) to access the middleware functionbalities, different possibilities exist:
- The App calls just one local Technology Engines (TE). A TE is a module providing defined functionalities, such as a Media Framework to play a video. MXM defines some high level API and provides placeholders to define new ones. Part 3 of MPEG-M “Reference Software and Conformance” provides software implementations of a range of TEs released with a Berkeley Software Distribution (BSD) licence;
- The App calls a chain of local TEs. This TE serialisation is called “Technology Orchestration”;
- The App calls just one Protocol Engine (PE). A PE is an implementation of an Elementary Service (ES) such as Create Licence which in turn calls just one or a sequence of local or remote TEs. Part 4 of MPEG-M “Elementary Services” defines a set of ESs and the corresponding PEs;
- The App calls a chain of PEs. Part 5 of MPEG-M “Service Aggregation” defines a machine-readable representation of the PE workflow that represents the “Service Aggregation” implied by the sequence of PEs.
Figure 3 shows an Application calling the PEa→PEb→PEc Aggregated Service and where PEa calls just one TE, PEb calls 3 Orchestrated TEs and PEc calls 2 Orchestrated TEs. Typically a special TE called Orchestrator drives the sequence of TEs to accomplish the goal.
Fig. 3 – MPEG-M Aggregation and Orchestration
The example of Fig. 4 explains how Elementary Services can be aggregated to provide full-fledged services. Assuming that there is a Service Provider (SP) for each Elementary Service, a User may ask the Post Content SP to get a sequence of songs satisfying certain Content and User Descriptions.
Fig. 4 – A possible services chain centred around Post Content SP
End User would contact Post Content SP who would get appropriate information from Describe Content SP and Describe User SP to prepare the sequence of songs using its internal logic. He would then get the necessary licences from Create Licence SP. The sequence (“titles”) of songs would then be handed over to Package Content SP. Package Content SP will get the songs (“Resources”) from Store Content SP and hand over the Packaged Content to Deliver Content SP who will stream the Packaged Content to End User.
Part 4 Elementary Services specifies a set of standard Elementary Services and related protocols to enable distributed applications to exchange information about entities playing a role in digital media services (end users and service providers), e.g. Content, Contract, Device, Event and User, and the processing that a party may wish to execute on those entities, such as Authenticate, Create, Deliver, Describe, Identify, Negotiate, Process, Request, Search and Transact.
It is clear that in the real world Service Providers would not be able to exploit the potential of the standard if they were confined to only offer “Elementary Services”. Therefore MPEG-M part 5 provides the additional standard technology that allows the standard combination of Elementary Services to create Aggregated Services. In the example of Fig. 3 Post Content SP may wish to bundle, for example, the roles of Describe User and Package Content. MPEG-M Service Aggregation provides the means to set up a chain of Services, from one or more SPs, to respond to Users’ requests.