Sntarting from the early 2000s, several sessions were held to discuss about “what else does industry need from MPEG?” in the typical MPEG anticipative fashion. Finally at the Munich meeting (March 2004) MPEG came to the conclusion that while MPEG had produced many component standards, the integration of technologies, save for the canonical Audio-Video-System integration, had been left to implementers. The result has been that, e.g. ATSC uses MPEG-2 Systems and Video but a different Audio than specified by MPEG, and DivX uses MPEG-4 Visual, MP3 and AVI. Therefore while (some) components are standard and provided interoperability, the complete service package or application may not be interoperable.
Obviously implementers want to retain the freedom to make individual decisions in terms of what components technologies they should use for their products and services, but this is paid by end users who are confronted with incompatibilities between different system implementations.
So MPEG decided to enter the “system integration” area. This was done because MPEG has knowledge about (most of) the technologies needed by complete digital media solutions, internal expertise to do the integration and the appropriate industry representation.
Multimedia Application Formats (MAF) is the name of the new standard suite that bears the nice looking ISO number 23000, obtained through the well-known channels. MAFs specify how to combine metadata with media information typically, but not exclusively, packaged in an MPEG-4 file format to enable media interchange, management, editing, and presentation.
Part 1 “Purpose for Multimedia Application Formats” is, like MPEG-21 Part 1, an informative document outlining the scope of the ISO/IEC 23000 suite.
Part 2 “Music Player Application Format” (MPAF) specifies a format to carry MP3 coded audio content in MPEG-4 File Format augmented by a JPEG image for cover art and simple MPEG-7 metadata commonly expressed in ID3 tags, such as Song title, Album title, Artist, Year, Comment, Track, and Genre.
Figure 1 – Creation of a Media Player Application Format file
Assume that you have an MP3 file with ID3 metadata. An MPAF encoder extracts the ID3 tags and converts the MP3 bitstream to a format that allows the definition of Access Units and creates an MP4 file that can also contain JPEG images. The MPAF standard also enables album functionality by making reference to MPEG-21. It allows to collect several song files in the above described song file format into one album file. The second edition of the MPAF has added protection feature by incorporating AES-128 counter mode encryption as default protection tool and MPEG-4 IPMP-X. MPEG-21 IPMP and REL for protection and governance description.
Part 3 “Photo-Player Application Format” (PPAF) specifies a format to carry JPEG images and their associated MPEG-7 metadata in an MPEG-4 file to enable creation, sharing, searching and viewing of digital photo albums.
The supported metadata include image-acquisition parameters (e.g. date, time and camera settings) expressed as EXIF metadata and MPEG-7 visual content descriptions expressed as binary MPEG-7 metadata. The latter allow new, content-enhanced functionality, such as intelligent browsing, content-based search or automatic categorisation.
The following standard technologies are employed:
- MPEG-7 Visual tools to describe visual properties of the images
- MPEG-7 MDS tools to carry simple generic metadata
- MPEG-7 System tools to binarise the metadata
- MPEG-4 File Format
- JPEG
- EXIF (EXchangeable Image format)
The last is not an MPEG standard but it is supported because of its universal use.
The following EXIF tags are mapped into MPEG-7 descriptions:
- Artist
- ImageDescription
- UserComment
- GPS Latitude/Longitude/Altitude
- FileDateTime
The following MPEG-7 Visual Metadata are used in PPAF:
- Dominant Color Descriptor to characterize the color information when there is a small number of colours
- Scalable Color Descriptor for image-to-image matching and retrieval based on colour features
- Color Layout Descriptor for image-to-image matching and to visualise image appearance
- Color Structure Descriptor for image-to-image matching
- Edge Histogram Descriptor to retrieve images with similar semantic meaning
- Homogeneous Texture Descriptor for accurate search and retrieval.
Part 4 “Musical Slideshow Application Format” (MSSAF) is a superset of MPAF and PPAF in the sense that it combines the features of both AFs. Additionally MSSAF employs MPEG-4 Part-17 “Streaming Text Format” for timed text and MPEG-4 Part-20 “LASeR” Mini Profile.
Figure 2 – An example of MSSAF content.
Part 5 “Media Streaming Application Format” specifies how to use some MPEG technologies to build a full-fledged media player for streaming governed and ungoverned content. By referring to appropriate standards, MSAF defines the data formats exchanged between a number of devices used in a media streaming scenario: a Content Provider Device, a Licence Provider Device, an IPMP Tool Provider Device, a Domain Management Device and a Media Streaming Player.
In the most general case a Media Streaming Player obtains streaming content from a Content Provider Device using a Content Access Protocol. In order to use that content, a Media Streaming Player obtains a licence from a Licence Provider Device using a Licence Access Protocol. Further, to actually process the content, a Media Streaming Player may need to obtain the appropriate IPMP Tools from an IPMP Tool Provider Device using an IPMP Tool Access Protocol, as shown in Figure 1.
Figure 2 – Reference diagram of MSAF standard
Part 6 “Professional Archival Application Format” (PAAF) provides a standardised packaging format for digital files. The format is an implementation of the information package specified by the Reference Model of Open Archival Information System (OAIS), a framework for long-term digital information preservation.
PAAF specifies metadata formats to describe
- the original structure of digital files archived in a PAAF file
- context information related to a PAAF file and digital files archived in it
- information required to reverse the pre-processing applied to digital files prior to archiving them in a PA-AF file
and it specifies a file format for carriage of the metadata formats and digital files.
The following technologies are used:
- Technologies from Music Player MAF
- MPEG-21 IPMP Components Base Profile
- MPEG-21 REL MAM Profile
- ISO Base Media File Format 2nd Edition for:
- ‘stsd’ (sample description box)
- ‘sinf’ (protected scheme box) – to protect audio track and signalization of the protection format
- ‘ipro’ (item protection box) to protect metadata
- ISMACryp 1.1 and OMA2.0 DCF to enable random access to encrypted content
- AES128 CTR encryption tool (default encryption)
Part 7 “Open Access Application Format” (OAAF) is a packaging format designed for the release and exchange of content whose rights are owned by users who wish to release the content so that other users can freely access it, butwithout making the content “public domain”. This standard is the result of a proposal of a DMP use case.
OAAF packages different content types into a single container file and provides a mechanism to attach metadata information, by using a series of technologies, in particular
- MPEG-7 to describe the resource
- MPEG-21 REL to model the intentions of the licence
- MPEG-21 Event Reporting to provide a feedback mechanism, which can notify the author, when a user wants to derive a content or extract an item out of the container file.
Part 8 “Portable Video Application Format” (PVAF) defines a format for the use of video files on portable devices to give users the possibility to use a local resource interactively as shown in the figure below
Figure 3 – Portable Video AF content
PVAF enables playback of content
- Generates by user on his own for PVP (downsize and encode), e.g. UGC content
- acquired from package media
- acquired from the internet
PVAF uses the following technologies:
- ISO Base Media FF, MPEG-4 / AVC FF
- MPEG-4 AVC Baseline Profile, Level 1.2
- MPEG-4 HE-AAC Profile, Level 2, Stereo 48kHz
- MPEG-4 Streaming Text Format for subtitles
- MPEG-7 Metadata for movie information like film title, etc.
- JPEG ISO Standard for still images of e.g. movie posters
- MPEG-7 MDS: UsageHistory DS, HierarchicalSummary DS
- MPEG-21 File Format and DID
- MPEG-4 LASeR.
Part 9 “Digital Multimedia Broadcasting Application Format” (DMBAF) specifies how to combine the variety of resources and associated information of the mobile TV Digital Multimedia Broadcasting (DMB) service. Users can acquire and consume information anywhere, in a well-defined file format that facilitates interchange, management, editing, and presentation of DMB content as illustrated in Figure 4.
Figure 4 – Digital Multimedia Broadcasting AF scenario
Part 10 “Surveillance Application Format” (SAF) specifies the combination of audio and video content drawn from MPEG technologies, related metadata and file format, suitable for surveillance.
Part 11 “Video Stereoscopic Application Format” (VSAF) defines a format for a creator to take and for a service provider to distribute stereoscopic images, enabling users to have more realistic experiences (with or without special glasses) and to store the stereoscopic content for possible redistribution.
Part 12 “Interactive Music Application Format” (IMAF) defines a format to package interactive music content with audio tracks before mixing, in order to allow users to freely control individual audio tracks. The producer can create several versions (producer mixing 1, producer mixing 2, karaoke, rhythmic etc.) with just one piece of music, using the metadata structure for mixing information. Figure 5 illustrates an example of IMAF use.
Figure 5 – Interactive Music AF example
More parts are being developed but they will be introduced later.