MPEG-1 is a great standard, but there is a potential problem in its practical adoption. Imagine I am a manufacturer and I choose to be in the business of making MPEG-1 encoders and decoders. I believe I have faithfully implemented all normative clauses in ISO/IEC 11172-1, -2 and -3 and I have checked that my decoder correctly decodes content generated with my encoder. Now, a customer of mine buys my MPEG-1 decoder and starts using it to decode content produced by an encoder manufactured by a competitor. Unexpectedly he encounters problems. My customer talks to my competitor and he is shown that content generated by his encoder is successfully decoded by his decoder. Who is right? Who is to blame? My competitor or myself or both?
This problem is not new to ISO and IEC. The “Procedures for the Technical Work” prescribe that a standard must contain clauses that enable users of the standard to assess whether an implementation conforms to the standard. One could even say that a standard is useless if there are no procedures to assess conformity to it. It would be like issuing a law without having courts one can make recourse to assess “conformity” of a specific action with the law.
The conformance problem used to be less well known to ITU because, when telcos were a regulated business providing a “public service”, they performed the “conformity” tests themselves to make sure that terminals from different manufacturers would interoperate correctly on their networks, without exposing subscribers to the kind of incompatibilities I have just described. They used to put a “seal of approval” on conforming terminals. Telcos used to do this because it was part of their public service licence, but also because keeping subscribers happy and not letting them suffer from incompatibilities was “good for business”, an old wisdom that too many proponents of new business models seem to disregard or forget.
When the telecommunication business became deregulated, independent Accredited Testing Laboratories (ATL) were set up. For a fee ATLs issued certifications to products that had successfully passed the conformance test. But even ATLs are a byproduct of the traditional “public service” attitude of the telcos.
In the IT and CE domains, which MPEG-1 ends up also sort of belonging to, the attitude has always been more “relaxed”. If you buy a mouse and you discover that it does not work on your computer, what do you do? If you are lucky the shop you bought it from will refund your money, if not you are stuck with a lemon. The same if you buy a component for your stereo. Sure, the consumer is protected, because if he is dissatisfied he can always take legal proceedings… The attitude of the IT and CE industry has always been one of either not claiming anything or, at most, of making “self-certification” of conformity.
That, however, is something that may work well when there is a market with large companies, producing mass market products, possibly not terribly sophisticated and where the product itself depends on a key technology licensed by a company that puts conformity of the implementation – to be verified by the licensing company – as a condition for licensing. Licensors have an interest to make sure that licensees behave correctly because they are interested in the good name of the technology and, again, because that leads to satisfied customers and hence more revenues. This has regularly been the case of major CE products.
Virtually none of these conditions apply to MPEG-1, and certainly not the last. There are multiple patent holders for the MPEG-1 standard, but none has the authority or interest to become the “godfather” and oversee the correct implementation of the standard. Therefore the approach adopted by MPEG has been to develop a comprehensive set of tools to help people make independent checks of an implementation for conformance.
Part 4 of MPEG-1 “Conformance” gives guidelines on
- how to construct tests to verify bitstream and encoder conformance
- test suites of bitstream that can be used to verify decoder conformance.
Encoder conformance can be verified by checking that a sufficient number of bitstreams generated by the encoder under test are successfully decoded by the reference software decoder. Decoder conformance can be verified by bitstream test suites or by decoding bitstreams generated with the reference software.