While MPEG was busily developing more and more video compression standards, China decided that it could well afford to have its own national digital audio-video-system technology and in June 2002 established the Audio and Video Coding Standard Workgroup of China (AVS). The group achieved its goal and in December 2003 Wen Gao, the long-time Chinese HoD to MPEG, asked me to have AVS recognised as an MPEG standard.

In spite of our friendship my answer could only be that the recognition as such of external standards was not MPEG policy. On the other hand something that could look like an equivalent result could be conceived.

The request had come at a time I was already reflecting on the fact that, in spite of the MPEG “firepower” in terms of technical expertise and hence of quality of its standards, there were more and more cases of private companies developing their own proprietary codecs. Even though the performance of these was non yet a threat to MPEG’s supremacy, each of these standards had some good elements and, for different reasons, were supported by some service or device.

So the idea was to develop a standard that would enable the building of new video codecs starting from some form of standardised tools. Unlike what is done with traditional video coding described in Figure 1 where encoder and decoder share the syntax and the semantics of the video coding algorithm

Figure 1 – Basic Video Coding Model

the intention was to enable the sharing of the syntax and semantics of the way a specific video coding algorithm is described, as depicted in Figure 2.

Figure 1 – Basic Reconfigurable Video Coding Model

I made this proposal to the Munich meeting (March 2004) and Euee S. Jang agreed to lead the “Reconfigurable Video Coding” group.

The group produced two standards

1. A language to describe decoders (ISO/IEC 23001-4 or MPEG-B pt. 4)
2. A library of video coding tools employed in MPEG standards (ISO/IEC 23002-4 or MPEG-C pt. 4).

The language defined in MPEG-B part 4 is RVC-CAL. Using this language one can describe a particular decoder including the connection of video coding tools and the bitstream syntax and parsing.

Figure 3 – An implementation of RVC

As depicted in Figure 3 a specific RVC decoder is a software built using a Video decoder description that assembles video coding tools drawn from a standard Video decoding tool library. The software would run on a programmable device.

Figure 4 shows how a specific decoding solution can be built using the decoder description and the tools drawn of the Media Tool Library (MPEG-C pt. 4). The Tool-Box is a collection of Functional Units (FUs), i.e. modularised video decoding units.

Figure 4 – The RMC Tool-Box and a Functional Unit

The decoder description is given to an RVC decoder which creates an Abstract Decoder Model. From this a decoder implementation is created using an MPEG Tool Library, a repository of video decoding tools for a specific platform.

Figure 5 – Building an RVC-based decoding solution

The library of MPEG-C pt. 4 is not static as it can be augmented with new tools coming from either MPEG standards or other tools submitted by interested parties to MPEG that have been shown to provide improvements in at least one decoder configuration. Figure 6 shows how an existing decoder (line above) can be updated (line below) by replacing an existing Functional Unit (2nd in 2nd line) and by adding a new Functional Unit (3rd in 2nd line).

Figure 6 – RVC decoder update

Assume now that an MPEG library (toolbox 1) and two proprietary libraries (toolboxes 2 and 3) have been developed for a specific platform: A service provider can distribute video content for three types of decoders implemented in that platform (see Figure 7)

Figure 7 – How to build different decoders based on the RVC standard

• Decoder 1 is a decoding solution based on MPEG-B pt. 4 that employs tools drawn from the MPEG tool library of MPEG-C pt. 4 (toolbox 1)
• Decoder 2 is a decoding solution based on MPEG-B pt. 4 that employs tools drawn from the MPEG-C pt. 4 tool library (toolbox 1) and a proprietary library (toolbox 2)
• Decoder 3 is a decoding solution based on MPEG-B pt. 4 using tools drawn from a proprietary tool library (toolbox 3)

All the three decoders can be defined to be “MPEG decoders”, with the following understanding,

• Decoder 1 may conform to MPEG-B pt. 4 and MPEG-C pt. 4 and to a specific MPEG standard if the decoder solution uses only the tools prescribed in that MPEG standard
• In any case decoder 1 conforms to MPEG-B pt. 4 and MPEG-C pt. 4
• Decoders 2 and 3 only conform to MPEG-B pt. 4

Therefore a decoder solution based on the RVC standard can have 3 levels of conformance

1. To MPEG-C pt. 4
2. To MPEG-C pt. 4 and MPEG-B pt. 4
3. To MPEG-C pt. 4, MPEG-B pt. 4 and to a specific MPEG Video coding standard.

The RVC work has been extended to other media, in particular 3D Graphics Coding.