All posts by admin

The Impact of MPEG-2

The radio spectrum is sliced for use by a multitude of services and the VHF/UHF band assigned to terrestrial television broadcasting is very narrow. Spectrum scarcity used to make television a great business to be in, first because the number of broadcasters in any given market is reduced to a handful of Service Providers (SP), and second because the business has been given the means to sustain itself: public broadcasters by imposing the payment of a licence fee to citizens owning a receiver and commercial broadcasters by relying on advertisements as a source of revenues. Some countries even have the best (so to speak) of both worlds: public broadcasters not only charge licence fees, but they can also get further revenues from advertisement. 

That easy life began to change in the 1980s when some SPs, fancying what at that time looked like a virtually infinite bandwidth, started using satellite for television broadcasting. All of a sudden, the number of simultaneous programs, until that time reduced, could easily grow by orders of magnitude. Practically, however, those who wanted to watch satellite TV had to put up with annoying disruptions in their homes and make substantial expenses in antennas, wirings, and Set Top Boxes. Not really a deterrent, but the viscosity of habits established in decades of TV viewing works wonders. 

If there were some relaxed faces in broadcasting companies, there were also concerned people who understood that digital television was still years away, but one day it would come and that day the wind of change would blow like a hurricane. On that day the number of television programs that could be broadcast on the terrestrial network would become, say, 5 times as many. Great news, you would say. Not really, they said – and I say. 

The business of a broadcaster is fine tuned to provide its audience programs, either self-produced or acquired, with a degree of attractiveness (somehow related to cost) such that the audience is sufficiently high and characterised to entice advertisers and provide revenues to offset the cost of operation and program production or purchase. Leaving some positive margin is a necessity, of course, more than it is desirable. But if one day digital Fairy Morgana suddenly multiplies the number of channels by a factor of 5, are the revenues going to multiply by 5? This is not likely, and this is true for both public and commercial broadcasters. On the cost front, network managers will have to manage 5 times the transmission capacity, but the operation cost of the physical infrastructure will hopefully be multiplied by less than 5. On the other hand, program production and purchase costs are probably going to have something near to a 5-fold increase, if the level of program quality remains the same. The conclusion is that, finding a satisfactory new equilibrium point in multichannel terrestrial television may not be straightforward. Ways could be found to manage the transition, but not if the total numer of programs is multiplied by 5, but by a smaller number. This, however, would mean to return unused channel to the state (so called digital dividend), a prospect most broadcaster would balk at.

Some people are more forward-looking than others. Instead of mourning the eventual demise of their business, NHK engineers set out to develop a new television system, which they called HDTV, with a bandwidth 5 times standard television’s. In doing this they were simply continuing the excellent tradition of broadcast engineering, which had begun with the introduction of colour, to extend the capabilities of television. 

The model set by NHK, CCIR wrangling notwithstanding, found other converts. The MAC solution pursued by Europe was clearly less bold than Japanese HDTV, but HD-MAC looked like a solution that combined the best of both worlds: an evolutionary path that targeted the eventual goal of HDTV while providing an immediate, compatible improvement to today’s television. 

A similar pattern of goal setting drove the American broadcasters but with a very different outcome. A move by the FCC to hand the 700 MHz band allocated to television broadcasting over to cellular telephony, triggered the request by the broadcasting industry to upgrade the general USA broadcasting technology field. By requesting an evolutionary path to HDTV, that industry not only secured the preservation of the existing frequency allocation status, but succeeded in putting an option on yet another portion of spectrum, the one used for the differential signal designed to provide HDTV. 

Given the background, one can understand how the worldwide broadcasting industry must have watched with trepidation what was being done by this uncontrolled group of technologists gathered under the name of MPEG (and I would indeed subscribe to the word “uncontrolled”, if I think of some of the active figures in that group, some deeds of one of which I have already told) to provide a digital television broadcasting standard. But, as much as the blue-suit people from IBM made a deal with that group of long-haired programmers working for what would become the Microsoft that we know, so the worldwide broadcasting industry sort of “made a deal” with the uncontrolled MPEG technologists. As an aside, it is worth remembering that, much as in the case of the IBM-Microsoft deal referred to above, the deal that MPEG “signed” was non-exclusive to broadcasting. 

Ronald Bedford, then with the Independent Television Association in the UK, Taiji Nishizawa, then Director of the NHK Science and Technology Research Laboratories and Chairman of CCIR SG 11/B, and Robert Hopkins, then the Executive Director of ATSC, attended all the critical meetings in which the MPEG-2 features took shape. They did not so much attend the Video group where the “uncontrolled” ruled, as the Requirements group that set the guidelines for the development of the technology. 

Much as the blue-suit people from IBM did not question the low-level DOS function calls that the long-haired Microsoft employees of those early years were defining, so these gentlemen from the worldwide broadcasting industry did not question the prediction modes or packet field choices that the MPEG technologists were making in the red-hot Video and Systems meetings (but the experts from their compnies probably did). Eventually the MPEG-2 standard provided all the functionalities that this industry requested – by no means in a homogeneous fashion – and at no detriment for the functionalities that other industries demanded. 

The first MPEG-2 customer was probably the US ATV project through its involvement in the process starting from the time of the Tarrytown meeting. At the end of 1996, the FCC selected part 1 (Systems) and part 2 (Video) and Dolby AC-3 for Audio for use in digital broadcasting of HDTV on the USA terrestrial network where the data would be carried by the 8-VSB modulation system. A few other countries such as Canada, Korea and Taiwan adopted the same system. Interestingly, the standard does not mandate any display format and indeed there are broadcasters transmitting in interlaced and others in progressive format. 

The second MPEG-2 customer came from an unexpected quarter. In the USA, where analogue satellite broadcasting had not taken hold at all, Hughes Electronics was aggressively working on a major digital satellite broadcasting project called DirecTV, later to become a separate company with that name. For this initiative the MPEG-2 compression, far from a curse, was a blessing because it lowered the cost of developing the digital technology, enabling a wider offering of programs than could ever have been possible before because of spectrum scarcity and hence service cost. Against all odds DirecTV turned out to be a significant success.

The third customer came from an even more unexpected quarter. In spite of the technical success of European HDTV in the form of HD-MAC, as demonstrated at the winter Olympics in 1992 in Grenoble, Europe started having second thoughts about digital television. At the initiative of Peter Kahl of the German Federal Ministry of Posts and Telecommunication, interested parties started gathering in 1991 as the “European Launching Group” of a “European Project on Digital Broadcasting”. The first talks were directed at creating the European equivalent of the American ATV initiative, obviously skipping the analogue prelude. A technical group within the project started mulling ideas about solutions that employed scalable video coding ideas. However, the results, as with scalable solutions already tried in MPEG, were not up to expectations and the plans for a scalable road to digital television were shelved. 

In 1993, the European Project on Digital Broadcasting abruptly decided to redirect its interests to a new direction, viz. digital satellite broadcasting. The political environment, which until that time had been dithering while the project was drafting its charter, finally decided to endorse it. The initiative, under the new name of Digital Video Broadcasting (DVB), was then formally kicked off. 

At the top of the hierarchy there was a Steering Board (SB), a body with a limited number of elected members representing European PTT Administrations and industrial members, not necessarily European. Below it there were four committees, the Commercial Module (CM), the Technical Module (TM), the IPR Module and the Promotion and Communication Module. DVB takes pride in being “market driven” and therefore the TM does not (or should not) undertake technical work before the CM considers it from the market viewpoint and formally communicates its support. Technical specifications that are produced by the TM are then submitted to the SB for approval, sent to the Comité Européen des Normes (CEN), the European counterpart of ISO; CENELEC, the European counterpart of IEC; and ETSI, the European counterpart of ITU, where appropriate, for rubberstamp and published as European Norms (EN). 

In a rather short time, DVB produced the first specification of a complete digital satellite broadcasting system. The audio and video source coding were MPEG-2 Audio and Video, multiplexing and transport were provided by MPEG-2 Systems. The technology specifically developed by DVB was the QPSK modulation scheme for satellite broadcasting. Continuing along this line, DVB later produced modulation standards for CATV (64 QAM) and terrestrial television (COFDM) in different versions. Other standards developed subsequently concern return channels for different media, Service Information (SI), a description of which programs are available where and when and the Multimedia Home Platform (MHP). So DVB became another major “customer” for the MPEG-2 standard and the first to adopt the full Audio-Video-Systems triad. 

The fourth MPEG-2 customer was the DVD. In 1994, a number of CE manufacturers and Hollywood studios led by Toshiba and Warner Bros. defined a new CD format, quickly overcoming a competing format by Sony and Philips. The new format could pack more bits than CD Audio by using shorter laser wavelength, finer track pitch and inter-pit pitch, by gluing two discs back-to-back and by allowing the disc to be played at variable speed. A two-hour movie could be stored on one of these discs in MPEG-2 Program Stream where the video part is encoded at variable speed. 

Bits on a DVD may be encrypted. This was the result of protracted discussions between the technology and content companies. The former companies wished to develop a format like CD Audio in which bits are clear-text while the latter were too wary of providing their bits in a form that would lend itself to easy copy and distribution when moved to an IT environment.

The fifth MPEG-2 customer was the Japanese broadcasting industry. One should have expected that, with their investments in analogue HDTV, the land of the rising sun would see anything digital applied to TV as anathema. Not that the development of MPEG-2 should have pleased the bureaucrats at the Ministry of Post and Telecommunication (MPT), but the NHK Science and Technology Center, with a remarkable foresight, were working on a thoroughly conceived digital broadcasting system called Integrated Services Digital Broadcasting (ISDB).

Gradually MPEG-2 was in universal use in Japan with digital satellite broadcasting in SDTV and HDTV, CATV and Digital Terrestrial Television. Japan is probably the only place where the audio component of television is encoded using MPEG-2 AAC. 

MPEG-2 was a recognised success and, because of it, the audio-visual landscape completely changed. Billions of satellite, cable, and terrestrial STBs, and DVD players – hardware or software – have been sold. 


Beyond MPEG-2 Audio And Video

In 1992 and 1993, I was looking with interest at the developments in Europe bearing the name “European Project on Digital Broadcasting”. Seeing what was brewing in Europe, the VADIS Strategic Advisory Group asked me to organise a “European Seminar on Digital Television”. This was held in Geneva hosted by the EBU in February 1993 and attended by more than 100 participants, representing different companies and a dozen or so relevant European projects.

My (private) comment when I first heard of the European Project on Digital Broadcasting was that the initiative was good, but for two letters, the E and the B. When the project had become Digital Video Broadcasting (DVB), I said to myself (how insatiable I am) that again the initiative was good, but for two letters, the V and the B. At that time, however, my involvement in MPEG, VADIS and my company duties occupied all my time and no time was left to go beyond such lazy thoughts. 

Indeed, I had no time for a deeper involvement. Until the November 1993 MPEG meeting, my overriding goal was to make sure that MPEG would indeed produce the MPEG-2 CD as planned. If that goal had been missed, MPEG could very well have lost the confidence that the worldwide television industry had clearly placed on it: providing the technology to overhaul the 50-year old analogue television world with its billion receivers and its established business models, mostly dictated by the scarcity of spectrum, to the digital world with its unknown paradigms, fearful to some where “spectrum wealth” would probably be the rule. The digital television technology was being prepared while Europe, Japan and United States were handling the analogue-to-digital transition with largely incompatible strategies. If the complexity of the technical endeavour, viz. multi-program transport layer, video coding layer supporting interlace and scalability, backwards-compatible multichannel audio coding layer and DSM-CC, is added to the fray, it should be no surprise that I had been single-mindedly locked in until the Seoul meeting in November 1993, when MPEG-2 achieved the official Committee Draft (CD) status. 

Upon my return from Seoul, for the first time in 6 years I had some time to think about the foundation of the work that I had started in 1988 with MPEG-1. I realised that the enabling technology role that MPEG had defined for itself was crucial to enable the development of a common digital television technology, a continuation of the IVICO idea in other – forced – forms. The work, however, even if successful, was insufficient to guarantee complete End-to-End interoperability within the same applications, across different regions of the world or, across different applications. 

A few weeks after my return from Seoul, I had gone a long way from my original idle thoughts about my preferred use of letters in acronyms of European initiatives and I had already identified the missing link: to provide – at the terminal and service level – the kind of interoperability that MPEG standards had enabled at the signal processing level. To provide an example, an MPEG Video decoder can decode both 625/50 and 525/60 video, but what if the display only supports one of the two video formats? Because of that last missing link the user will be unable to get the information, and knowing that “inside”, “in the digital domain” everything works fine, will definitely not comfort him. 

These kind of obstacles were bound to appear all over the place, because the systems to be built with digital technologies were so complex and there were just to many opportunities to recreate – maybe unwillingly – the same kinds of incompatible systems that our forefathers had – maybe willingly – designed and built in. End-to-End interoperability had to be a requirement explicitly set in the system design. To achieve these further goals, some extra “glue” had to be standardised. Specifically, and in my order of perceived importance, these were: information representation, mid-layer protocols and physical layers. 

The place to develop this “glue” was the next question. In an ideal world an appropriate committee of a formal SDO should have settled this issue. Unfortunately, the “glue” spanned technology areas that are the competence of all three main international standards organisations and, within each of them, the competence for it was scattered across multiple committees. SDOs were and still are organised along traditional vertically integrated systems to serve specific industries, with objective difficulties in addressing cross-industry issues. I saw no chance of achieving the allocation of the work I had in mind in a reasonable amount of time and measurable amount of effort. 

So I came to the conclusion that I needed a new organisation where the different business interests represented by Broadcasting, CE, IT and Telecommunication industries could coexist and operate in a synergistic way. The operating principles of the new organisation would be similar to those of MPEG but, unlike MPEG’s mission of technology development, the new initiative would have a largely “system integration” mission. 

In a document that I drafted during the 1993 Christmas holidays, when the idea had already taken shape, I find the following text: 

The Digital Audio-Visual Council (DAVIC) is an International Organisation established to promote the timely and coordinated development of audio-visual applications and services and of the hardware and software components making them possible in a way that maximises interoperability across services/applications and countries. 

So, instead of following the established procedure and proposing a New Project (NP) as per the ISO/IEC directives, a person who had been engaged in formal international standardisation for years had to join the growing list of founders of the “forum du jour”. But in my view, DAVIC was meant to be the body that would serve the needs of these manifold industries; it could never have been a Technical Committee of ISO/IEC or a Study Groups of ITU, at least not in a time span of practical interest. So started the DAVIC initiative. 

The idea of a new organisation was proposed and accepted at a “founding” meeting in Geneva (March 94) hosted by Pierre-André Probst, then of Swisscom. Pierre-André was the chairman of CCITT SG XV, but he, too, understood that DAVIC’s goals could not be achieved within a known formal standardisation environment. The meeting saw the participation of some 50 delegates representing 40 companies from 17 countries. The industries represented broadcasters, telcos, IT and CE companies, research establishments and others. 

Resolution no. 2 of that meeting stated: 

The vision is to create the conditions that will enable producers of Digital Audio-Visual (DAV) information to reach the widest possible audience, users to have seamless access to DAV information, carriers to effectively transport DAV information and makers of hardware and software products to provide the means to support unrestricted production, flow and use of DAV information. 

A preparatory meeting was held at Chester and Newark, NJ (April 94), followed by the “DAVIC Opening Forum” held in San José, CA (June 94). On that occasion, the scope of DAVIC was discussed through workshop-style presentations of more than 50 papers. The organisation was then formally established in Geneva as a not-for-profit organisation (August 94). The statutes were derived from the ATM Forum charter. Me Jean-Pierre Jacquemmoud, a lawyer in Geneva, took care of the legal side of the registration and his law firm provided the domiciliation of the organisation. 

The DAVIC organisational structure was based on a General Assembly (GA) of members, chaired by the President. The GA had all the powers, in particular those of approving work programs and specifications. In addition there was a Board of Directors (BoD) managing the day-to-day business of the association. Reporting to the BoD was the Management Committee (MC), in charge of supervising the work of the Technical Committees, which were in charge of developing the technical specifications. I was appointed as President and Chairman of the Board.

The first CfP was issued at the Paris meeting (September 94) and sixty responses were received and discussed at the Tokyo meeting (December 94). The submissions were further considered and the scope of the first DAVIC specifications defined at the Orlando, FL meeting (January 95). While the technical work continued, a Strategic Planning Advisory Committee (SPAC) was set up at the London meeting (March 95) to advise on strategic lines of specification development. The Procedures for Technical Work were developed and approved at the Melbourne meeting (June 95). The long-term DAVIC work plan, identifying topics covering the next two years of work, was developed through a process of inquiry of members’ opinions and approved at the Hollywood meeting (September 95). Finally the DAVIC 1.0 specification was approved and the CfP for the next phase of work drafted and published at the Berlin meeting (December 95), exactly at the time that had been decided and announced at the founding meeting 20 months before. 

The DAVIC 1.0 specification is an outstanding piece of technology integration. DAVIC 1.0 defines a System Reference Model depicted in the figure below.

DAVIC_Reference_Model

DAVIC System Reference Model (from the DAVIC 1.0 specification)

The Reference Model comprises three separate systems, the Content Provider (CP) System (CPS), the Service Provider (SP) System (SPS) and the Service User System (SUS), interconnected by Delivery Systems. The first (CPS to SPS) part was not initially considered and attention concentrated on the SPS to SUS part. Nine interfaces, A1 to A9 were defined (another critical A0 interface in the end-user device was added later) and most of them defined.

DAVIC worked out the 5 types of signals – S1 to S5 – that are required between a server and a client linked by a telecommunication network. 

DAVIC_SPS_to_SUS_model

The DAVIC SPS to SUS model (from the DAVIC 1.0 specification)

The following entities were defined:

  1. Service Related Control, providing all control for the services that are offered by the Delivery System. 
  2. Network Related Control, providing control functions for network configuration, connection establishment and termination, and information routing. 
  3. Core Network, responsible for accepting information flows and transmitting the information, error-free, from a source location to a destination location. This network can be characterised as a high speed digital network or series of networks. 
  4. Access Network, the final delivery segment of the system. It can take many forms depending on the specific implementation. The entity is composed of the following sub-entities:
    1. Access Node, responsible for processing information flows in preparation for transport through the selected distribution network. 
    2. Distribution Network, the transport medium utilized to deliver information flows from a source location to a destination location. 
    3. Network Termination, an optional entity used to adapt information flows from one network configuration to another. This device, if used, serves as a bridge for the system. 
  5. Management, responsible for the operation and maintenance functions of the network. In most instances, the Management Entity will communicate using S5 information flows.

DAVIC adopted the MPEG-2 standard for the audio-visual component, DSM-CC User to Network for session set up, DSM-CC User to User for interaction with content, MHEG for multimedia composition and other technologies. A great effort was made at aligning different physical layer technologies for delivery of MPEG-2 content: Fiber to the Curb (FTTC), Hybrid Fiber Coax (HFC) and Asymmetric Digital Subscriber Line (ADSL). 

The last technology in the list justifies a short digression. At the first Kurihama meeting in November 1989, Charlie Judice, then with Bellcore, organised a dinner inviting a selected number of people and talked about some research work called Asymmetric Digital Subscriber Line (ADSL) that was being carried out by his organisation. I heard the ADSL acronym for the first time on that occasion and I became an immediate convert. Upon my return to CSELT, I widely promoted the idea, just to discover that my words were falling on deaf ears. I did get one positive response at the next COST 211 meeting, where the chairman Helga Séguin of France Telecom took the matter to heart and triggered the Eurescom project called Interactive Multimedia Services at 1 Mbit/s (IMS-1) that was approved and went as far as setting up a demonstrator. 

At that time, telcos never did much with ADSL, beyond a few trials, such as the Italian Video Magic VoD trial service of Stream, a STET company, that eventually became part of Sky Italia, then Sky Group owned by Comcast). While ADSL later became a major way to deliver “broadband”, at that time the technology never got the opportunity to achieve the critical mass that would trigger sufficient interest to invest in its industrialisation. The reasons will be understood from this other short story.

In 1993-94 I was advising an Italian manufacturer of telecommunication equipment, part of the ATET Group, who wanted to develop a strategy for interactive video services, as part of my work at CSELT. When I heard of their intention to use ADSL, my old interest was revived. I was too quick because my words were quickly dismissed. “We do not like 1st generation ADSL” – they said – “that is too low quality. We want 8 Mbit/s that will be provided by the new generation of ADSL”. The result was that 1st generation ADSL was not industrialised and the 2nd generation never really took off in those years. Later, ADSL was widely deployed at bitrates spanning the 1 to 20 Mbit/s range. The problem is that in the mean time no broadband services were deployed and when they did, the deployment followed the rules set by other industries, not by the telcos’. 

As for the other initiatives in which I had taken part, my group had not been sitting idle. Starting from the MPEG-2 decoders built in the VADIS project, my group realised a complete End-to-End DAVIC system (the SPS to SUS part of DAVIC 1.0) under the name ARMIDA and proudly showed it at the MPEG meeting in Florence in March 1996. 

The two years it took to build an organisation and develop such complex specifications as DAVIC 1.0, that cut across the businesses of all players in the audio-visual world, are proof that SDOs – whether formal or not – can provide – if there is a will – timely responses to standardisation needs, no matter how complex and “converging” the issues are. 

The DAVIC experience has been unique. For the first time representatives of all industries – not just the researchers – had the opportunity to sit together, agree on a common reference model, and specify the entire delivery chain and the interfaces and protocols that were needed to achieve the seamless flow of information that the DAVIC Statutes advocated. I particularly appreciate the work of Mike Carr, then of British Telecom Laboratories, who developed the work leading to the DAVIC System Reference Model.

The year 1995 was very demanding at the personal level. Just after the MPEG Singapore meeting in November 1994, I had been affected by a very serious illness that had prevented my attendance at two key DAVIC meetings in December 1994 and January 1995. Thanks God my health started recovering in spring 1995, but the negative influence of that illness extended well into mid-1995. At the professional level in that year I was engaged in the preparation and chairing of 6 DAVIC meetings and 3 MPEG meetings that dealt with the conclusion of MPEG-2 and the start of MPEG-4, in addition to my customary work of promoting my different initiatives. At the company level, my employer was getting more and more interested in the DAVIC work because of its Socrate Project aimed at providing VoD services nationwide. 

The DAVIC membership continued to grow and reached 220 corporate members at the time of the December 1995 meeting. Unfortunately the excitement of the first months of 1994 began to subside when  accountants in the telcos started assessing the likely ROI date in DAVIC-enabled VoD services. The high costs included in particular the ATM technology, which had been taken as the basis for deployment of the telecommunication networks underpinning DAVIC services. These costs looked all the more unjustified when compared with the advantages of choosing the much cheaper IP-based equipment provided by the IT industry that in that time frame was leaving its original “internet for research” ghetto in which the telcos’ OSI support policy had confined it. 

Infighting on petty technology choices had started delaying the work in some Technical Committees very much as is customary in other bodies that I used to quote as examples not to be followed. Finally the BoD and MC duality of roles, the result of the original need to provide enough relevant positions to all major players, started to take its toll. 

At the December 1995 meeting in Berlin, after the GA had approved DAVIC 1.0, I announced my resignation. I had given the best of myself to the DAVIC initiative, probably much more than I have ever given to an initiative in such a short period of time, but I did not feel like I wanted to continue to be involved in it. I thought that having produced the first epoch-marking specification, and having left a comprehensive work plan that covered two years of work after my departure, would have allowed DAVIC to continue playing the central role for which I had created it with such a wide support. 

This did not happen. After I left, DAVIC contravened the most fundamental “one functionality-one tool” principle of standardisation by adopting another audio coding technology next to one already standardised by MPEG. The publication of DAVIC as ISO/IEC 16500 through the ISO Publicly Available Specification (PAS) process turned out to be a pure publishing effort. A poison-pill clause that I had inserted in the DAVIC statutes mandated a positive vote of 2/3 of its members after 5 years for the organisation to continue for another 5 years. The membership chose not to give its vote of confidence in the organisation and DAVIC was wound up.

The thirty years passed since that mid December evening in Berlin do indeed allow is to make an assessment of the industrial audio-visual landscape in two different ages. MPEG-2 was designed to be as a neutral technology that should have assigned equal changes to all industries. DAVIC was conceived as a security policy that the end user would not become the the earthenware vessel among many iron vessels.

What is left of the hopes of those years. We will need to get more evidence in the next chapters to draw a conclusion. 

 


The Need For Standards

The MPEG-1 and MPEG-2 cases described above demonstrate the power of standards as communication enablers. But what is a standard? Even today, there is so much confusion around this word that I want to use this page is to try and dissipate it. 

If you take the Webster’s under the entry “standard” you will find the following definition: 

a conspicuous object (as a banner) formerly carried at the top of a pole and used to mark a rallying point especially in battle or to serve as an emblem. 

This refers to the original meaning of the word and is indeed a good definition. A standard is a model, a reference to be followed, if one feels to be well served, by following that guide. Of course that presumes one is given the freedom to choose one’s guide, which is not always the case. 

In the same dictionary, one finds another definition immediately after: 

Something that is established by authority, custom or general consent as a model or example to be followed. 

Most of this definition is also OK. Custom or general consent are subjective matters and one should be free to decide whether to adhere to them or not, although it is true that there are cases when customs can become rather constraining. The problem is with the word “authority”. 

The attitude of some Public Authorities, which sometimes feel they should impose the use of certain communication standards by the force of law and verify compliance with law-enforcement officers, is unfortunately included in this definition. The legal nature of some communication standards is a problem, and a particularly severe one in some industries. 

The broadcasting industry has traditionally been regulated in all countries and its technical standards have – or used to have – a legal status in many countries. A similar approach used to be followed in the telecommunication industry. This is reflected in the fact that the ITU, obviously including its ITU-R and ITU-T branches, is a Treaty Organisation (and an Agency of the United Nations), i.e. one where governments used to be the signatories and had the right to be represented. Today things have changed even there with private concerns acquiring the right to become ITU members and vote. 

This mixing of technical, political and legal matters is one reason why certain standards never got approved, or took ages to be approved, see the ITU-R HDTV case. This is also why in the early 1990s ITU-R SG 10 “Audio Broadcasting”, that should have felt “deprived” (in the inverted logic of SDOs) of a piece of work that rightfully belonged to them, viz. audio coding for Digital Audio Broadcasting, actually welcomed the work that eventually led to the MPEG-1 Audio standard. They did so because they were realistic in their assessment that the explosive mixture of technical and political issues that dominates their committees would hardly have allowed them to deliver. 

This does not mean that the ITU is unable to produce effective standards. The video coding work in ITU-T did produce some excellent standards like H.261, but this has been achieved by a group of “Experts”. 

My basic rule in standards development is that technical and political problems should not be addressed at the same time. Good engineering sense suggests that, if the problems are uncorrelated, at least as a first approximation, it is better to “solve one problem at a time”, or “de-factorise” the problems. Once the first, technical, problem of developing standards, is solved, one can attack the second, understandably more difficult, of converting technical standards into law, assuming that such a step is still needed. Luckily, that is done by a different group of people.

All the above has been said for the purpose of introducing my definition of standard: 

the documented agreement reached by a group of individuals who recognise the advantage of all doing certain things in an agreed way. 

Forget about law and authority. An agreement is a compromise between total satisfaction of one side and total dissatisfaction of the other. If one accepts a compromise it is only because the perceived advantages earned by entering into an agreement exceed the perceived disadvantages caused by the accepted limitation of freedom. Standards are like contracts regulated by the Civil Code. You use them if you want to buy a pair of oxen or for anything else: you sign them if you like them, you shun them if you don’t like them. 

It is to be noted that my definition of standardisation contains the one found in the Encyclopaedia Britannica as a special case: 

Standardisation, in industry: imposition of standards that permit large production runs of component parts that are readily fitted to other parts without adjustment 

This is the manufacturing-driven view of standards, apparently different from the interoperability-driven view because it amounts to the same effect embodied in my definition. People accept standards because it is more convenient to, say, use nuts and bolts that can be chosen from a finite set of sizes, thanks to the possibility of achieving economies of scale. We do certain things in an agreed common way because we, the parties in the agreement, see a benefit in doing so. In the example, it is more convenient to have an assured source of nuts and bolts or electronic components developed once and for all independently of specific needs, instead of having to develop the components every time they are needed. 

There is one thing that I do not like in Britannica’s definition, though. The word “imposition” is misplaced. Following a standard should be a free choice, not the result of “imposition” by anybody on anybody. This is again a proof of how this innocent word “standard”, that should sound libertarian because it should be the result of a free adherence to an idea, possibly motivated by the most down-to-earth reasons, is again associated with some remote authority using the word as if it were wielding a weapon. 

This does not mean that Public Authorities should have no role in communication matters. I believe that richness of information sources is a prerequisite for upholding democracy. Therefore, when a monopoly of information supply is being established, based on some restrictive technology, they should not give their nod just because the case has been made that otherwise that particular content distribution business cannot sustain itself. These are excuses because the companies making the claim are exactly those who have thwarted all attempts at developing standards that would have enabled a sustainable, although different, type of business. The answer of public authorities to the creation of monopolies should just be: “no way, make and use the right standards”, instead of saying: “you are a naughty boy, but you can get away with it, this time (and the next)”. More about this later.

My definition of standard is not particularly revolutionary and is perfectly in line with its original meaning. Even the very organisations that drive the work in international standards setting follow it. An example? Most countries in the world use A4-size papers, an element of an ISO standard (ISO 216:2007). The American National Standards Institute (ANSI) is the ISO NB for the United States; still ANSI does not use A4-size paper. Inconsistency? Absolutely not. Simply, for ANSI as an organisation, the perceived advantage earned by the international (in this case) agreement of using A4-size papers falls below the perceived disadvantages caused by the limitation of freedom of keeping on using letter size paper. 


Patents And Standards

Johannes Gutenberg had the great idea of developing a system to print books using movable characters. He had not just the idea, which had been floating around for a long time (actually, it had been in use in China for centuries), but also the skill and will to develop a set of technologies capable of transforming the idea into a system working practically, efficiently and economically. 

To achieve that goal, he needed money to make his investigations and trials and he found a wealthy person (an investor, we would call him today) willing to provide it. However, being a prudent man, Johannes hid the secrets of his developments from everybody, including his financial backer. In those remote times the patience of financiers was probably more robust than today’s, but not infinite. After some years and a considerable amount of money spent, his financial backer started a law suit. Gutenberg lost it and was financially ruined. 

If Gutenberg had lived in a culturally more advanced city like Venice or Florence, which had already started issuing litterae patentes (“open letters”, whence the word “patent” comes from) since a few decades, he could have protected his invention with a patent. Then he would have started negotiations with some wealthy person and agreed on how to split the revenues. Unfortunately, life in 15th-century Mainz was not yet as sophisticated as in those two Italian cities and Gutenberg had only one way to protect his invention from robbers of what we call today his Intellectual Property (IP): hiding his secrets. If he had had his IP protected, everybody would have gained: he could have led a better life, his financial backer would have had a share of the revenues and the world at large would have gained from the mastery of his printing technology. The printing art might have had a different history. 

Fortunately for Gutenberg, his story did not have a wholly unhappy end. After his genius had been revealed through the books printed using his invention, Gutenberg could escape the common fate of those times like ending his life as a beggar, thanks to the kindness of the Archbishop of Mainz, Great Elector of the Holy Roman Empire, who bestowed on him a tax-free sinecure with a life annuity of “cloth, grain and wine”. 

The correlation between that world of 15th-century Germany and today’s is not straightforward, but the basic elements are still there: inventions happen, inventors wish to protect their IP, the state of the invention in the beginning is too crude for an exploitation, inventors need capital to transform their inventions into something exploitable, (venture) capitalists develop methods on how to manage their risks, etc. 

One thing does not correlate, though. We no longer have Archbishops, whether of Mainz or not, whether Great Electors of the Holy Roman Empire or not, who are kind enough to recognise and concretely support a ruined inventor. And I am not sure it is not a good thing they are no longer around (for this purpose, not to elect the emperor of the Holy Roman Empire). 

Patents were the timely answer to a necessity felt by a society that was becoming increasingly more complex and their use quickly spread throughout all of Europe. In those early years, patents took the form of a privilege granted by a sovereign (not necessarily, as in Venice, a republic) for a limited period of time (14 years in 17th century England). The two major revolutions that happened at the end of the 18th century – the ones that led to the formation of the United States and the Republic of France – gave patents a firmer legal ground in those countries that spread to other countries. 

The USA Constitution of 1787 explicitly links private incentive derived from the patent “monopoly” to societal progress. It gives Congress the power “to promote the progress of the useful arts, by securing for limited times to inventors the exclusive rights to their discoveries”. The French bourgeoisie, imbued with Illuministic principles, drafted the French Constitution of 1791 with an explicit declaration of the natural right of an inventor, as an inherent right of Man, to the exclusive right to his invention. 

Therefore, more than 300 years after Gutenberg, L.-J. Daguerre, S. Morse, A. G. Bell, T. A. Edison, A. and L. Lumière, G. Marconi and a host of other inventors took care of filing patents of their inventions with their respective Patent Offices and they, their business associates and possibly their heirs gained conspicuous benefits from those inventions. 

Those were the hay days of the individual inventor, who harvested all the benefits from his invention, but in the 20th century the centre of gravity progressively shifted from the individuals to the companies hiring them. Today the split of economic returns from patents produced by an inventor hired by a company varies from all benefits going to the company to a proportion of benefits also going to the inventor. The only thing that is guaranteed in all cases is the recognition of the status of inventor to the person who made the invention. Countries that had allowed employers to take all benefits from their employees’ inventions are having second thoughts, seeing that less patents are filed in their countries.

In the second half of the 20th century, many new products and services became possible thanks to electrical, magnetic, electronic and, more recently, digital technologies. The frenzy of “colour television” contaminated even respectable companies. Governments were called in (actually, they called themselves in) for “help”. In the late 1960s, the prospects of exploitation of digital technologies in imaging, audio and video became more concrete and many companies and organisations started making or funding research in digital signal processing and specifically in audio and video coding. As a result, the current numbers of valid patents in these fields are counted by the tens of thousands. There would be many more valid patents, had not the earlier ones lost their validity because R&D in this area started so early. Still many inventors are good at repackaging expired patents in ways that allow them to file an “old” patent again.

Companies producing innovations tend to protect their IPR with patents that tend to become major ingredients for many standards. The result is that IEC, ISO and ITU, the three main international standards organisations, have become well equipped to deal with patents in standards and have actually agreed on a common policy. Traditionally, the work of these organisations used to be the formal ratification of some technology required for an electric appliance or a machine tool or a communication device that had been invented and possibly already exploited by an inventor. 

But because standards organisations do not exist to ratify unregulated monopolies, for an international standard to specify the use of a patent for a conforming implementation of the standard, the rights holder must either release the rights to his patent or commit to give license to his patent “on fair and reasonable terms and non discriminatory conditions” (so-called RAND or FRAND ). Besides ISO, IEC and ITU, other regional and national standards organisations, apply a similar policy, possibly with a slightly different wording. Needless to say, standards organisations have no intention to delve into the meaning of what makes terms and conditions “fair and reasonable”. In case of major problems, such as the case of a rights holder not complying with the policy, the matter can be brought to the highest instance of the SDO and can even lead to the withdrawal of the standard. We will have to return to this point in due course.

In the past, adoption of a standard based on a necessary patented technology did happen – just take the case of the telegraph, the telephone and the radio – but this was before companies started investing in R&D to develop new technologies for new communication products. As a result, SDOs have begun to produce standards starting from a situation in which there are different industry-grade technologies, very often developed by major players, that exist or are even in use, possibly already in fierce competition in the market place. 

In these conditions, the task of producing a standard, which has to incorporate one or more than one of these proprietary technologies, becomes hard because the role of the standards committee shifts from the place where discussions deal with technical merits of a solution, obviously driven by commercial requirements, to a place where a solution is defined based on fitness of a technology to the current plans of companies, bundling with other technologies, terms of exploitation, etc. Of course there is nothing wrong with technology deals between companies or with close alignment of standards to products within companies – God forbid the thought of prohibiting it – but this is damaging if it is done in a standards committee (or in the hallways of the building where a standard is developed). 

Working in this space is a tricky business. Blessing one, or more than one, specific solution that has already achieved product maturity obviously exposes people to the wrath of those excluded because “being a winner” entails considerable economic benefits. Not only because those wishing to operate in that particular business will have to get a license for the relevant IPR – if there is IPR, the right to use it must be acquired – but because at such a late stage of deployment of the solution, the basic IPR is often inextricably linked to other product features and these, too, end up being part of the license deal. Therefore it is only natural that antitrust authorities have been scrutinising this standard-setting method and associated licensing deals. 

The selection process is the tricky part. In abstract terms everybody agrees that standards should provide just one way of doing things, like my definition of standard goes, but the implementation of the principle is often less than thorough. The original desire of the committee to select just one winner from a number of candidates may still be there, but preserving that determination to the end is the hard part. When people working for a company are in a standards committee, their determination to pursue the shared goal to the end dwindles, if they see competing technologies to their company’s prevail in the favours of the committee. Most often the outcome of a dialectic battle lasting anywhere from an hour to ten years is to compromise the intellectually accepted principle and, voilà, a “limited” number of solutions, often disguised as “options” are anointed and all of them are made part of the “standard”. 

Usually, it is too costly to implement all options in products and so implementers try to make up for the failure of the committee making their own choices among the “limited number” indicated by the standard. But it is a sure bet that different manufacturers will make different choices and communication between equipment of different makers will no longer be guaranteed. 

Sometimes it is difficult to make people who have grown accustomed to this loose method of work understand that having a “limited” number of solutions or options in a standard very often heralds the failure of that standard. Because of too many signaling options it took 10 years for European ISDN to achieve a decent level of interoperability between different operators and, within the same operator, between equipment of different manufacturers. Because of too many options, many standards were stillborn because the critical mass that would have justified the necessary investments by the industry could not be reached. 

Other, more enlightened people are not blind and understand the damage they are inflicting on their industries and companies, but they shrug their shoulders as if they wanted to show their resignation to “the inevitable burden of civilised life” – namely options or multiple choices in standards – that society forces humans to carry. 

The first encounter of MPEG with patents happened in 1990 when, as part of my ISO duties at that time, I started collecting patent statements from companies that had taken part in the development of MPEG-1. An annex to all parts of that standard (and of those following it) gives the list of companies that have declared to be willing to adhere to the ISO patent policy for all IPR owned by them that is required to implement the standard. Setting aside MPEG-1 Audio, for which a licensing organisation (used to) exist, I am not aware of anything comparable that offered forms of collective licensing for patents necessary to implement MPEG-1 Video and Systems. In any case this is probably idle talk as way more that the canonical 20 years have passed since the publication of MPEG-1.

Some time later, MPEG had to deal with MPEG-2 patents. Many companies had participated in the MPEG-1 development not so much because they necessarily had plans to make products in that space, but because they were keen to be around and help tune up the MPEG machine for the next big MPEG-2 deal. At the London meeting in November 1992, some National Bodies had started to draw MPEG’s attention to the fact that, if a large number of patents were needed to implement MPEG-2, for each of which the licensing conditions were “fair and reasonable” – the reasoning went – the licensing terms and conditions of the combination of those patents would not necessarily be “fair and reasonable”, not to mention the fact that striking many licensing deals with with all the licensors would be a very costly and time-consuming exercise which probably only big companies could afford. The irony in these National Body documents was that they knew very well that MPEG was unable to respond positively to these requests because dealing with patent licensing is explicitly forbidden by the ISO/IEC directives. Still, something had to be done. And I did.

In January 1993 I took the initiative to invite to the Rome MPEG meeting Mike Smith, head of the Information Technology Task Force (ITTF), the group inside the ISO Central Secretariat in Geneva looking after JTC1 matters. Mike provided advice on how this difficult issue could be treated without trespassing the limits set by the ISO/IEC directives. The MPEG success owes a lot to the help Mike (and his colleague Keith Brannon) gave to solve this and the many different procedural problems over the years. I do regret Mike’s untimely death a few weeks before his retirement date.

Then, at the following Sydney meeting in March/April 1993, I took the initiative to convene an informal meeting, open to any interested party, to discuss which steps could be undertaken to make progress. Eight possible alternatives were identified but, unfortunately, none looked particularly attractive. 

In the fall of 1992, I received the visit of Dick Green, President of CableLabs, a newly formed laboratory funded by the North American cable industry. This was one of the many meetings that I was having in my drive to promote MPEG-2 to different industries in different countries. Even though Dick, in his former position at the Public Broadcasting Service (PBS), the public USA broadcaster, and a long-time participant in ITU, had a good understanding of standard matters, in that pre-convergence era he was not fully aware of the ISO process and wanted to personally make sure that a solution coming out of MPEG would satisfy the due-process requirements that the CCIR and his industry required for acceptance. 

In July 1993, Baryn Futa, then CableLabs CEO, started attending MPEG meetings and particularly the “ad-hoc group on technical support for IPR”. The ad hoc group, chaired by Greg Wallace who acted as Head of the US Delegation after Cliff Reader had left Samsung at the beginning of that year, received the following mandate: 

To provide technical support for ongoing discussions on IPR matters, to assist with efforts inside and outside of MPEG. 

This ad hoc group was re-established for the last time in September 1993 in Brussels because MPEG members felt that, after a seminar of parties interested in MPEG-2 patents to be held on the Saturday after the end of the Brussels meeting, the MPEG-2 patent issue could have – finally – a life of its own. Baryn took the initiative in his hands, I severed my already tenuous ties with the MPEG-2 patent discussions and could finally relax and think of other, technically more rewarding challenges. 

In the event, an MPEG-2 Patent Pool was established, managed by MPEG LA. In spite of that company bearing “MPEG” in its name, there was no formal relationships at all between NPEG and that company. MPEG LA acted as a “one-stop shop” for MPEG-2 Video and Systems patents (Audio is not part of the licensing), but a few companies wanted to license their technologies separately. Originally the number of patents found necessary to implement MPEG-2 Systems and Video was about 30 (which is different from the number of patent holders), but hundreds of different patents were considered necessary (because the same patents has been filed in different jurisdictions). Licensing of the patents managed by MPEG LA could be obtained by paying 2.5 USD per decoder or encoder box (originally the fee was 4 USD). Interestingly, the amount to be paid for patents in an MPEG-2 decoder stayed constant, while the number of patents was increasing. For DVD a different licensing scheme was adopted: 0.035 USD per printed DVD, and 2.5 USD per decoder. These numbers are provided to give an idea of the amount MPEG-2 licensing. Reportedly, MPEG-2 Video and Systems royalties generated a cash flow of about 1 billion USD per year.

Important notice: this was a rough summary of a complex issue provided for information. The licensing terms used to be publicly accessible at the MPEG LA web site. A few years ago, MPEG LA was acquired by Dolby’s licensing company. 


The MPEG Way Of Standards Making

It used to be difficult to make a person outside the group understand that MPEG was not in the business of running beauty contests, i.e. of choosing one or a few established solutions from a set of candidates. Indeed MPEG did not typically choose complete solutions, but it used to assemble them from individual technologies, possibly at different levels of granularity. The selection was made using objective criteria, such as numerical results, video or audio quality assessed formally or by collective expert viewing or hearing sessions. By building the solution in the standards committee using objective criteria, MPEG was shielded from the danger of being accused of anti-competitive practices. 

MPEG could develop standards in this way because it operated at a different time position than most other standards groups: as a rule MPEG developed a standard before industries had a declared need for it. At that time, technologies are in general not fully developed and, therefore, the committee can assemble and possibly improve them on the basis of measurable technical merits. “A priori standardisation” – the qualification MPEG used to give to this notion – was one of the MPEG ground rules. 

This positioning avoided certain risks but created others. As has been shown before for the MPEG-1 case, sometimes companies themselves make gross mistakes in their product development plans: the attraction of a media-related product or service as seen at a board meeting or at the drawing board is often quite different from the attraction felt by a consumer when he has to reach into his wallet to buy the product on the shelf or subscribe to the service offered. Why should MPEG have succeeded where big companies failed? 

One answer to this question is that MPEG did not develop products, but standards, and generic standards at that. An MPEG standard could very well not be fully suitable for making products. This implied that sometimes adopters pick some of the technology pieces of the standard and very often they need other technology pieces from elsewhere to make their products. The second answer is that MPEG had a well structured process to to develop requirements by including industry representatives for later conversion into Call for Proposals, both exposed to industry at large.

An additional danger is that, even if the right standard has been spotted, it may be developed too early, and in this case the standard may be superseded by a better technology at the time industry really needs the standard. Conversely, the standard may arrive too late, at a time when companies have made real investments for their own solutions and are not ready to discount their investments. The seriousness of this danger can be measured by the state of development of the relevant technologies in the industry.

In general we can say that while MPEG had a track record of very successful standard, the imprint of success was not necessarily inherited from the MPEG name. Indeed there were MPEG standards that did receive little or no attention. This was an unavoidable consequence of the MPEG time positioning. 

In the following I will give a general description of the steps followed by MPEG when it develops a new standard. This will hopefully confirm that the process is sound and produces the expected results. 

Step  Description 
Idea  It is hard to codify the steps that led to the idea of a new standard. Ideas could spring up from discussions and interactions during a meeting, from a specific proposal issued by one or more than one member or a National Body or from the common feeling that more technologies had been developed outside of MPEG, possibly in some laboratories of companies already represented in MPEG, that it might be good to consider for inclusion in one of the existing MPEG standards or in a new one. 
Involvement  When the decision to develop a standard was made by the group, the affected communities, if not already well represented, were alerted and invited to join in the effort. In the case of MPEG-1 this was a long and difficult process because MPEG was an unknown organisation with no track record. Later, things got much easier because there was already a network of liaisons with the main industry organisations. 
Requirements  The requirements the standard was required to satisfy were developed with the participation of all communities involved. In the case of MPEG-1, the approach was to consider different applications enabled by the standard from which “implications” were derived and from these “technical features”. The process, with changed names, remained basically in place. Typically, there were 4 documents: “Context and Objectives”, “Use Cases”, “Applications” and “Requirements” that were constantly updated as discussions continued. In the requirements development process, a legitimate requirement proposed by an industry was always included, possibly after a long debate to understand if this was really a new or just an extension of an already identified requirement. This was an essential rule to preserve the “multi-industry” qualification of MPEG. 
Approval  In parallel to this internal process of clarification of the precise purpose of the new standard, there was a need to obtain an approval to start  a New Project (NP) leading to a standard from the ISO hierarchy. This was done by the SC 29 Secretariat (the committee above MPEG) and involved two levels of approval. The first at the level of the NB members in SC29 and the second at the level of NB members in JTC1 (in ISO a NB may be member of a Technical Committee but not of an SC and vice versa). The NP was approved if there was a simple majority of yes votes and if at least 5 National Bodies committed to work on it. 
Timeline  The NP proposal also contained a standard development timeline and MPEG took pride in sticking to it. This was part of the basic idea that standards are the “goods” that the standards committee “sells” to its customers. As for a company it is clear that the goods have to be of high quality and conform to the specification issued by the customers but, first and foremost, they have to be delivered by the agreed date. If a company makes a plan to go to market by a certain date with a certain product that requires a certain standard technology, and makes the necessary investments for it, the company – i.e. the buyer vis-à-vis the standards committee – is not going to be happy if the standards committee – i.e. the supplier vis-à-vis the company – at the due date reports that they are “behind schedule”. Therefore “stick to the deadline” was another of the key MPEG precepts. 
CfP  When the requirements were sufficiently mature, usually MPEG issued a Call for Proposals (CfP) of technologies satisfying the requirements (a CfP was always issued when the NP involved audio and visual coding). Later, as there were existing MPEG   standards for audio and video, it was found necessary to go through the phase of a Call for Evidence (CfE) before issuing a CfP, to get confirmation of the group’s feeling that new technology exists that was worth standardising. 
Testing  Proposals had to contain a description of the proposed technology with a level of detail enabling MPEG to make an implementation. In the case of audio and video coding standards, the CfP also requested the submission of complementary evidence in the form of coded material. The audio or video material selected by the group for the CfP (or the CfE) was provided to those responding to the call and the encoded material was tested according to the procedure described in the CfP. For MPEG-1, MPEG had the great advantage of being able to use the JVC facilities in Kurihama for the video tests and the Swedish Radio facilities in Stockholm for the audio tests. Therefore proponents did not have to pay to have their proposals tested. In the case of MPEG-2 Video the only cost incurred was the preparation of D1 tapes used in the tests. Later, it was not uncommon to entrust the tests to a commercial organisation and in this case all proponents are asked to share the cost of the tests. 
Ranking  Submissions were analysed and ranked along several axes, e.g., subjective quality, complexity, coding delay, robustness to errors, etc. The ranking guided the extraction of the best features from the best proposals to create something new that was based on elements present in the different submissions. This was called Simulation Model (SM) in MPEG-1 and Test Model (TM) in MPEG-2. 
CE  Then the Model evolved from meeting to meeting using the Core Experiment (CE) methodology. A CE was designed so as to enable a collective assessment of the improvement offered by a particular technology providing a certain feature, with all other elements of the CE kept unchanged. At least two companies had to be part in the CE in order to have results that qualified for consideration. MPEG was very strict when it accepted technologies and based its decisions on the “one functionality – one tool” principle, so that no unnecessary duplication of technologies was made. As part of this exercise there was constant attention to assess implementability of the standard. 
Reference Software As a rule CEs were carried out using software. In later years (starting with MPEG-4), the originator of the CE of a successfully executed and accepted CE donated the software to ISO which became part of the Reference Software for that standard. The software, duly integrated with the rest of the Model, was scrutinised by all members, in the best tradition of Open Source Software (OSS) development. This method of work allowed all participants to try different options, and the group to assess the results and select the best solution. It also ensured that there were no “black holes” in the standard that only some participants know and could exploit. The obvious weak point of this method was the time it took for MPEG to develop a standard. However, the very idea of the MPEG standard development process was based on the assumption that the work was carried out before industries had a need for the standard according to a timeline agreed at the beginning. 
WD When the work reached sufficient maturity, a Working Draft (WD) was produced and updated at every meeting. 
CD When the WD was considered sufficiently mature and ready for balloting with the NBs, the WD became a Committee Draft (CD). NBs considered the CD in their national committees within 3 months from the publication of the CD and casted their ballots with comments. At times several hundred technical comments were received and studied by the group. A comment might be accepted or rejected, but the group had to provide a justification for each choice. This appeared in a separate document called Disposition of Comments (DoC).
DIS The revised draft standard was published again as Draft International Standard (DIS) and underwent another ballot with the NBs lasting 5 months. Ballots and comments were again treated as for the preceding ballot. 
FDIS The new version of the draft standard was called Final Draft International Standard (FDIS). At this time the document achieved its final (technical) shape.
IS There was another ballot with the National Bodies, very short and without any possibility of changing the technical content of the standard that had to remain in the state it had at FDIS level. After this last ballot the document achieved its final status as International Standard (IS), identified by a 5-digit number.
VT In parallel with the balloting, MPEG carried out the so-called Verification Tests (VT). Once the work neared completion, it made sure that the standard did indeed satisfy the requirements (“product specification”) originally agreed to by the different affected industries (“customers”). VTs had the purpose of ascertaining how well the standard produced met the specification in terms of quality. This was obviously also an important promotional tool for the acceptance of the standard in the market place. 
Corrigendum Standards are like living organisms. The group itself or an industry user could discover that, in spite of all attentions, an error had crept into the standard. In this case a “corrigendum” was issued. This was balloted only once and became immediately effective as soon as the first ballot was successfully resolved. 
Amendment It could happen that the group found that it was possible to enhance the standard with new features. In this case an “amendment” was issued. An amendment was a standard for all practical purposes and, as the other standards, it was balloted twice  with the following names: Proposed Draft Amendment (PDAM), Draft Amendment (DAM) and Final Draft Amendment (FDAM). 
Withdrawal It never happened for MPEG, but one day it could have happened that a certain standard was considered obsolete. In this case the standard might have to be withdrawn. As MPEG defienes media formats, I expect that no MPEG standard will ever be withdrawn.

Not systems but tools is another MPEG precept that was developed in the way MPEG tried to serve multiple industries. By definition, industries making end-user products need vertically integrated specifications so that they can make products satisfying their needs. Audio-visual decoding may well be a piece of technology that can be shared with other communities, but if industries need to sell a DVD player or a digital satellite or Over The Top (OTT) receiver then they require integrated standards. But if different industries need the same standard, quite likely they will have different end systems in mind. Therefore only the components of a standard, the “tools”, as they were called in MPEG, could be specified in an effort designed to serve multiple industries. 

The implementation of this principle required a change of the nature of standards from “system” standards to “component” standards. Industries will assemble the tool specifications from the SDOs and build their own product specification. What constitutes a tool, however, is not always obvious. Single channel and multichannel audio or SDTV/HDTV are components needed in AV systems. Defining a single “tool” that does the job of coding both single channel and multichannel audio or conventional television and HDTV may be impractical because the technology has to be designed and manufactured to do things to an extent that some customers do not need. The “profile/level” philosophy successfully implemented by MPEG provided a solution: within a single tool, different “grades”, called “levels” were defined. 

Specify the minimum” should be a general rule of standardisation. In some environments, however, there is a mixture of technologies that are absolutely indispensable to enable communication and of those components that bring a standard nearer to a product specification. This is, for instance, the case for “industry standards” or when standards are used to enforce the concept of “guaranteed quality” that used to be so dear to broadcasters and telecommunication operators because of their “public service” nature. This is not a good practice and should be abandoned – as far as standards are concerned – in particular when a standard is to be used by multiple industries. Only the minimum that is necessary for interoperability must be specified in a standard. 

Finally “Relocatability of tools” was another MPEG ground rule, again dictated by its multi-industry nature. When a standard is defined by a single industry there is generally agreement on where certain functionalities reside in the system. In a multi-industry environment this is not always the case. Take the case of encryption used in digital television, a tool that has an important value-added function. Depending on your role in the AV distribution chain, you might like to have the encryption function located where it best serves your role in the value chain. If the standard endorses your business model you will adopt the standard, if it does not you will antagonise it. Therefore, not only must the technology be defined in a generic way, but also in such a way that the technology can be located at different points in the system. 

Lastly we have the case in which an industry or a vocal member does not want a standard to exist. This is a logic that MPEG did not accept, because of its multi-industry nature. Standardisation is a service offered to those who need it, not a tool to prevent other people to have the standard they need. Those who do not want a particular standard should have no obligation to participate in its development much less to use it. And, also, they have no right to tell other people that a certain standard should not exist.


An MPEG Meeting

Starting from the “marginal” idea of compressing digital video to fit it in the low throughput of 1.4 Mbit/s of a CD, MPEG had to develop newer areas of expertise to provide standard component technologies to build complete solutions. The size of the group (some 300 participants from early on, 500 experts at the time of the High Efficiency Video Coding (HEVC) standard development and 600 participants in it last days) and the span of technologies considered prompted the establishment of a subgroup structure so that work could proceed more expeditiously. 

Following the order of the MPEG workflow, two examples of which (MPEG-1 and MPEG-2) have already been provided, at the beginning of the 2010s the subgroups were: 

  1. Requirements (RQ) to produce requirements for standards, both new or already under development. 
  2. Systems (SY) to develop standards for the coding of the combination of individually coded audio and video, including related information so that the resulting combination can be used by an application. 
  3. Video (VD) to develop standards for coded representation of moving pictures of natural origin. 
  4. Video Coding (VC) to develop the High Efficiency Video Coding (HEVC) standard (joint with ITU-T).
  5. 3D Video (3V) to develop 3D Video extensions for AVC and HEVC (joint with ITU-T).
  6. Audio (AU) to develop standards for coded representation of audio information.
  7. 3D Graphics Coding (3D) to develop standards for coded representation of audio and moving images of synthetic origin.
  8. Test (TS) to develop methods for subjectively evaluating, tests and verify the quality of coded audio and moving picture.

In addition: 

  1. Convenor Advisory (CA) to advise on general matters related to the group, specifically meeting schedule and organisation.

MPEG was one of the first groups to exploit the power of the Internet not just to exchange emails and set up discussion groups, but also to create a repository where members could upload and download input documents, so that all other members could download the submissions, study them and discuss the issues raised on email reflectors even before a meeting started.

This facility enabled the group to increase its productivity to levels unthinkable before, when submissions could only be considered after the beginning of the meeting, actually after the host had made hundreds of copies for each of the input documents. Several hundred input documents (even in excess of 1,00) were uploaded at every meeting as input contributions. The upload had to be made at least 6 days before a meeting started.

The idea to put in place this new method of work started at the New York meeting in July 1993. Managing input documents, MPEG’s most valuable asset, was a nightmare since early times. Many participants were already making good use of email, but not all of them had access to it and, for those who had, adding a Word document as an attachment to an email would often lead to unpredictable results spurring a ping-pong of well-intentioned emails designed to solve the problem of “format incompatibility” (a.k.a. lack of standards). For a long time the safest way to have a document considered before the meeting was to send preliminary copies of contributions by fax to members who were assumed to be interested in it and then bring an original to the meeting. The host would then make 200-300 copies of each of the 100-200 contributions. Assuming an average of 8 pages per contribution and adding the copies of the documents that were produced at the meeting, the host had to often produce more than half a million pages of photocopies, with the record of 1 million copies first reached in November 1993 in Seoul, which included output documents on paper and on transparencies used to present a document at plenaries. 

The other problem was document distribution inside the meetings. The British Standard Institute, who hosted the November 1992 meeting in London, discovered at its expense that the standard “pigeon holes” method, besides being very costly to administer, simply did not work for MPEG. The only method that seemed to work was making piles of copies for each of the documents from where delegates would draw those of their interest. Jacques Guichard, former Director of Human Interactions at France Telecom R&D, and a long-time attendant at the COST 211 and CCITT meetings, seeing the rush of delegates to get a copy of an important document, once remarked that the level of civilisation in MPEG was lower than CCITT’s. Maybe so, but it was likely a consequence of the fact that MPEG was a large, totally self-supported organisation (besides being populated with unruly members). And so it continued to be until it was decided by some powers that be that MPEG non longer served their interests. 


Life In ISO

A free translation of the fake Latin proverb “Homo homini lupus, femina feminae lupior, sacerdos sacerdoti lupissimus” could go like this: man behaves like a wolf toward another man, a woman behaves with another woman in a more ferocious way than a man with another man, and a priest behaves with another priest in an even more ferocious way. 

Of course the proverb should not be taken too literally. The meaning is that, while men are prone to be physically aggressive toward other men as wolves are (or are thought to be) between themselves, women can be more aggressive toward other women, but in a more intellectual than physical fashion and priests can be even more aggressive with one another, of course only in a context where there are different opinions on how to act “ad maiorem Dei gloriam”. 

Until the time ITTF (Information Technology Task Force), the body inside the ISO Central Secretariat looking after JTC 1 matters was managed by people caring about IT standards – I hve already mentioned Mike Smith and now I mention Henry Cuschieri – I could hardly associate the third part of this proverb to the members of this huge organisation called ISO. Now I do. 

When, in January 1988, JTC 1/SC 2/WG 8 established the three experts groups JBIG, JPEG and MPEG, the matter did not escape the watchful attention of the British Standards Institute (BSI), the UK National Body. They immediately circulated a document within SC 2 requesting a discussion on the matter of the three experts groups at the following SC 2 meeting. 

It is not known what prompted BSI to behave in this way, but the matter was resolved at the SC 2 meeting in June 1988 thanks to the strong intervention of Dara Hekimi, then the Secretary General of the European Computer Manufacturers Association (ECMA) and an influential SC 2 figure at that time. During the same meeting, the three WG 8 Experts Groups had their terms of reference endorsed and, in the case of MPEG, enlarged to read “Coding of Moving Pictures and Associated Audio and their Combinations for Digital Storage Media”. This was achieved because I had attended that SC 2 meeting as Italian representative with Hiroshi Yasuda to provide a larger number of National Bodies supporting the WG 8 move – just in case. 

The “Associated Audio” part was more or less silently resented by Audio experts for several years. The CCIR even established a group called “Audio and Associated Video”. At the Dallas meeting in November 1995, “Associated” was finally removed from the title of MPEG, much to the relief of the Audio group.

When I had started gathering people with the goal of developing audio coding alongside video coding, Hiroshi and I had considered the possibility that the IEC, which had a consolidated presence in the area of audio and had already issued digital audio standards for CD and Digital Audio Tape (DAT), might already have started work on compression coding of digital audio. The two of us then went to Eindhoven (then Philips-city) to meet with Kurt Herrmann of Philips who acted as secretary for IEC SC 60A, the IEC Subcommittee in charge of audio recording, and learned that they had indeed in mind to do work on the subject, but concrete plan for it had yet to be hatched. This was taken as an implicit “go ahead” for the audio coding activity in MPEG. 

At the WG 8 meeting in Kurihama, the Multimedia and Hypermedia Experts Group (MHEG) was established. Francis Kretz explained his vision of multimedia enabled by a common technology applicable across a large number of domains, using very much the same generic approach that MPEG had taken for audio and video. A New Proposal (NP) request for a new standard was made at Kurihama and approved at JTC 1 level a few months later. Then the American National Standards Institute (ANSI), which also holds the JTC 1 Secretariat, saw this NP as a competing technology to what SC 18 Document Processing and Related Communication was doing with their Open Document Architecture (ODA) on audio and video added to textual information. In an unprecedented move, JTC 1 overruled the results of the MHEG NP ballot and convened an “Ad-hoc Technical Study Group on Multimedia and Hypermedia” in New York (mid-December 1990). The recommendation of that ad hoc group was to split the MHEG NP in two parts, one assigned to SC 18 and the other to WG 8. The part assigned to SC 18 (actually SC 18 itself) went nowhere.

In the meantime, the combined size of WG 8 and its Experts Groups (EG) was steadily increasing and I thought that the importance of the technology that WG 8 handled justified the promotion of WG 8 to Subcommittee level. At that time Hiroshi’s position in NTT did not allow him enough freedom to attend all the various international meetings where the proposal to create a new SC on Video, Audio, Multimedia and Hypermedia was on the agenda. I had no such limitation and could attend all environments in which the issues of the new SC were discussed .

At the Washington, DC meeting of SC 2 in April 1991 a coalition of National Bodies approved the secession of WG 8 from SC 2, much to the relief of the SC 2 old guard who had been working for decades splitting character coding hairs and had no interest in a group working on these strange new technologies that had to do with such alien and unstructured information types as audio and video. The WG 8 Experts Groups were elevated to WG level and became WG 9 (JBIG), WG 10 (JPEG), WG 11 (MPEG) and WG 12 (MHEG) with WG 8 retaining a “coordination” role. So the premises for the creation of a subcommittee were laid down but, in a world of (virtual) wolves (or, maybe, priests), this interim situation could have prompted another SC to hijack the activity. 

The NB of Japan who was eager to get another badge to their panoply of Subcommittes of which it hekd the Secretariat, strongly supporterd the move and finally, in November 1991, at the time of the second Kurihama meeting, the new SC 29 was established, with Information Technology Standards Committee of Japan (ITSCJ), the JTC 1 NB of Japan, as the Secretariat and Hiroshi Yasuda as Chairman. The four new SC 2 WGs  were moved to SC 29. Later WG 9 and WG 10 merged and the combined working group was called WG 1. WG 12 faded away after its original goal was reached. 

In the meantime JTC 1 gave itself the rule that the JTC 1 Chair and SC Chairs could hold only two three-year terms of office. Hiroshi stepped down in June 1998 and was replaced by Hiroshi Watanabe, at that time also with NTT and in 2006 by Kotaro Asai of Mitsubishi, an active MPEG member of which we will have more to say.


Craft, Intellect And Art

In primitive societies every able individual was capable of making for himself all that was needed for a living: bows, arrows, shoes, tents, pottery, etc. From early on, the social organisation, with a complexity originally so far from today’s, gradually evolved towards a specialisation of functions: the stronger ones became warriors, those good at making traps or hunting game went out to get food, the wiser ones became advisors to the tribe’s chief and the cunning ones were sent to negotiate with other tribes.  Even so, individuals had to rely on just themselves for most matters a lot more than is even conceivable today.

As society grew more complex, specialisation of functions affected the individual in a deeper way. While still generally capable of making most of the tools of their daily life for themselves, those in need of a sword had better go to a good blacksmith if they did not want to run the risk of finding themselves at a disadvantage in the battle field. To impress the beloved one, it was advisable to go to a capable craftsman and buy a nice necklace, otherwise one would risk one’s chances. If a visit from an important person was expected, hiring a good cook to offer the visitor a sumptuous meal was a necessity, if one wanted to get the much sought-after contract. 

From the oldest times and under a variety of civilisations and social orders, people have happily parted with some of their wealth for the purpose of laying their hands on some physical objects, produced by the skill of some craftsman, to achieve some concrete goal in the wider world or for their own personal enjoyment. 

With the growing sophistication of daily life, there were other cases where humans became ready to part with portions of their wealth for other, less physical, but for them no less important, matters. Somebody, realising his own lack of knowledge or inferior intellectual capabilities, might decide to “borrow” the missing abilities from somebody else who has a reputation for being endowed with them. In case of illness the medicine man might be called because his potions, magic formulas, dances and prayers were said to work wonders. After many months of drought, the shaman might be offered huge rewards if he only could get rain (but he was probably clever enough to request payment before his prayers took effect). The same ritual could happen if a tribe wanted abundant game or a good harvest. The making of these “wonders” required compensation to the person who effected them. 

With the further progress of civilisation, the ability to use words skilfully gave more and more benefits to those who mastered that skill. They could persuade somebody else to do something he had otherwise no intention of doing. The person able to give good advice to others would be considered a wise man and he could get many benefits if he exploited his skill. His ability to make strong arguments would convince people in trouble with the clan elders to enlist such services and defend themselves better. Great ideas about the current form of society would prompt the head of a social organisation to enlist the holder of such ideas in his ranks so that the status quo in society would be preserved. If the ideas were radical and promoted a new social order, and if the holder of these ideas was smart enough, he could himself become the head of a new social organisation. 

History provides countless examples. Demosthenes earned a living by offering counsel to Athenians in trouble with the law, but his political ambitions were thwarted when he failed to rouse Athenians against Alexander the Great. Cicero attacked Catiline and successfully exploited that case for his social standing and fame, but had to succumb to Anthony’s wrath when his party was defeated. Confucius preached his philosophy in the hope that one of the rulers of the Chinese kingdoms of his time would hire him. Moses became the leader of his people within a reshaped religion and Mohammed became the founder and head of a new religion. 

There is another type of intellectual ability that has existed ab immemorabili, possibly even before speech actually took shape. Humans use songs, often accompanied by rhythmic movements of the body or by some musical instrument, to convey a message that could otherwise not be expressed, or expressed poorly with words alone. This communication medium makes words more effective because rhythm heightens their effect as it allows performers to convey them with a passion that the rational product called “word” often cannot. 

Public Authorities, civil as well as religious, have always been good at getting control of those social moments when people forget their daily troubles in those opportunities of social relaxation called festivals. The authorities often lavishly remunerate those involved in the musical and scenic events because of the role that these moments have always played in preserving and fostering social cohesion and because of their functions in maintaining the essential elements of individuality of a social grouping. 

But, alongside the lucky ones, those groups of people or individuals who cannot get the support of some authority, or simply want to follow their own artistic inspiration without compromise, can likely meet with a very different fate. Artists and performers can be very good at playing a musical instrument or dancing or reciting a poem and people would even come from far away just to listen to or watch them. The artists and performers may personally be very fond of being surrounded by a crowd of people enthused by their performing ability. Those around, if they feel like that, might even happily give some coins, just to show the performers how much they like the way they sing, play, dance or recite. But others would give nothing thinking that the very fact that they have been listening to or watching them is sufficient reward to the performer. 

All this does not really matter if the artists and performers are wealthy persons and their only purpose in life is to express their creativity, possibly for their own exclusive pleasure. However, if they are poor or average persons, possibly with a family to raise, we can see a blurring of the border between an artist, receiving a reward for his creativity from the good will of onlookers, and a person who lives on the goodwill of others. 

This description of artists and performers is not something recalled from past antiquity, it is still with us today. The sad fact is that people take it for granted that they must pay for the work of a blacksmith or an attorney, but they consider it an option to pay for the performance of a singer or an actor. Maybe because art and business flair seldom go together, artists have often felt the need for somebody else to take care of the promotion of their work. 

Of course there are excellent artists who have a good life, but it is a sad fact that the excellence of too many great artists is recognised only after they are dead.


Fixating Works

Advancement in society – and ingenuity in developing appropriate technologies – created written works. If one takes the remaining samples of old written works, it is easy to imagine that they just record the textual parts of very complex events where different “media” were certainly involved: recitations, music, songs, dances, choruses, sacrifices etc. The magic formulae Chinese priests wrote 8,600 years ago on tortoise shells are just the textual component of what was probably a very elaborate ritual that must have comprised hymns, dances, burning of incense, special dress worn by priests, sacrificial offerings to the Gods, etc. We are left with the words, because they were “easy” (so to speak, because it took millennia to develop writing) to “fixate” on a physical carrier and sent to us across the millennia. 

Unfortunately we have completely lost the other components that would probably interest today’s ethnographers more than the words, and they would be better off to try and explain in which way we – modern humans – are more (sometimes less) than we were millennia ago. Some Egyptian papyri provide something more than written words, because they are supplemented by drawings depicting the power of the Pharaoh or the life of the farmer. Glimpses of another component – dance – can be obtained from Greek amphorae and temple bas-reliefs and similar remains from other civilisations. Extant Greek tragedies are composed of parts recited by actors and parts sung by the chorus, with the accompaniment of music, but the remaining textual components let us just guess the tensions that the Athenian society felt between a mythological past driven by the sense of the divine and a progressively lay society – in substance if not in form – while it would probably be easier to understand them if we could know how the chorus expressed its sharing of the actors’ sufferings. 

In the European Middle Age a method to represent music information was invented, but this was originally mostly used for the music that “mattered” at that time, i.e. religious hymns (Gregorian Chants). So we have largely lost how troubadours sang and played violas, even though we are able to read and play the music of later centuries. We are lucky to be able to read Kalevala – the Finnish epic poem – thanks to the efforts of Axel Gallén, the doctor who scoured the Finnish countryside in search of the scattered pieces of that epic poem, but we can only imagine how the two persons recited it sitting on a wood trunk and holding each other while moving in a rhythmic fashion. 

Still, preserving the written parts of those “multimedia” works was not a minor accomplishment. In antiquity, the facts of a life incomparably harsher than today’s did not really offer those inclined to writing many opportunities to exercise their art. Unless these people had considerable personal wealth, or they could find a patron who found pleasure in offering those individuals the possibility to manifest their talents (and being rewarded by the content of their writing), or some authority would hire them to support their plans of preservation, promotion, expansion or uprooting of a social order, it was impossible for most of them to engage in writing in a continuous fashion. Therefore the motivations of those ancient writers were regularly personal satisfaction in the work produced and the fame generated from it or a means to have a leisurely life. 

Under these conditions no one making a copy of a manuscript felt like he had to remunerate the author – probably the author would have taken such idea as an insult. Remuneration was not for the authors but for the amanuenses hired for that purpose. More likely the person making a copy thought that he was doing a favour to the author, because his work could reach farther, and a service generally to society, because he was propagating culture. And this was really needed, if one considers that in all of pre-Gutenberg Europe there were just a few tens of thousand “books”. 

This does not mean that in those early times some forms of commercial interest behind books did not exist. In Egypt a copy of the Book of the Dead was required at each funeral. Some surviving Babylonian tablets bear directions for getting a “book” from the library. In Rome Titus Pomponius Atticus employed a number of trained slaves to copy and produce books with retail branches of his business in the provinces of the empire. In 12th century China a publisher’s account for a book of 1,300 pages indicated a 3 to 1 ratio of selling price to production costs. All this shows that some business around “book copying” existed, but was limited to periods of well developed and stable social order, because otherwise it was hard to put in place and operate the value chain that would guarantee an economic return to an organisation that would look for and select manuscripts, produce copies, create and manage a distribution channel, and collect the proceeds of the sales. 

From very old times public authorities saw it as useful and dutiful to collect books. The royal libraries of the Assyrian kings were probably for the exclusive use of the sovereigns, but Ptolemy II Soter meant to help learned men find old works when he established the Great Library of Alexandria, because the great body of knowledge on which the Hellenistic world rested was scattered – literally in pieces – in hundreds of different places across the Mediterranean and the Near East. 

Qin Shi Huang-Ti, who ordered the burning of all works by Chinese philosophers in 213 BC, the Spaniards, who burnt all Aztec books in Mexico in 1520 and Hitler, who ordered the burning of books in 1933 all had a similar understanding of the importance of collecting books and applied the same ideas as Ptolomy II Soter, but in the negative, because their goal was the annihilation of a culture that was alien to them or considered as hostile. Keeping track of books published in a country is still a role played by many public authorities as in many countries one or more copies of each printed book must be deposited with the National Library. France has been innovative in 1975 when they created the “Institut National de l’Audiovisuel” (NA) with a similar mission for media content.

The invention of printing in the mid-15th century forced a radical change to the economic basis of written words because it provided the missing link that had often made the creation of a value chain impractical until that time. A new form of entrepreneur was born whose job was to select and edit manuscripts, design the material, arrange its production and distribution, and bear the financial risk or the responsibility for the whole operation. That this new entrepreneur played a useful role can be seen from the fact that, in barely 50 years after the invention of printing and with the low manufacturing capabilities of those times, the number of printed books in Europe amounted to 9 million. 

Printing, by offering books at a price that was orders of magnitude less than the equivalent for a manuscript, enabled broader layers of population to have access to culture and favoured the adoption of vernacular languages in Europe. Latin was the language of choice until that time also because it was one thing to find an amanuensis who could copy a manuscript, and another to find a translator. The market in books, therefore, from the very beginning became largely national and, until recently, national markets have been rather impervious to one another, save for those books that scored highly and qualified for translation into another language. This excludes science and humanities, where the market has been largely global since the very beginning of printing, moving from Latin to French, to German and, today, to English. 

Therefore, even though billions of books have been printed over the centuries, until quite recently it was still possible to keep oneself abreast of everything “new” through the use of catalogues published by individual publishers. Public libraries, often set up even in small communities, or their bigger equivalent in the larger cities, gave everybody access to a broad spectrum of culture.

Before movable type, manuscripts carried combinations of text and images. With the introduction of movable type, mass distribution of text became feasible, but at the expense of forfeiting the images of illuminated manuscripts. It was not until the integration of text and engravings came into practice that the expressiveness that illuminated manuscripts had achieved centuries before was approached. A similar phenomenon happened with the invention of phonography where the visual experience of the presenters was lost but the mono-media information – music – could be mass distributed. The invention of cinematography enabled the fixation of moving pictures, but for several decades the audio component could not be provided, until a standard technology supporting the missing medium was invented. 

With all the excitement caused by these new inventions that enable people to listen to music from a record or to watch a theatrical performance in a movie theatre, the hard economic facts were that fewer people would go to a concert, because they could buy a record and listen to it as many times as they wanted. This is just one example of how a new technology destabilises the status quo.


Rights

Being able to own “rights” to something is a fundamental concept of living beings: ants own the “right” to suck aphids, but they are “obliged” to feed them, if they want to continue sucking them. A dog pisses at all trees and poles in the area he happens to be to mark his “possession”. A hunting tribe considered as its own the area in which the tribe hunted and used to enforce this right by killing every intruder. William the Conqueror considered the land of England as his because of elaborated dynastic relations and because he had defeated the Saxons who already owned the right to it. On their part, the Saxons had grabbed it from somebody else before. It was not until 1866 that landowners in all the United States lost the right to own slaves and make them work “for free” in the cotton fields. 

Progress of society introduced several other forms of ownership. One that is of particular interest here is the right of somebody to be recognised as the author of a piece of artistic work. This is an old “right”, as can be seen from the use made by Martial, a Latin poet of the first century AD, of the word “plagiarius” (whose original meaning was “abductor”), to indicate a person who presented somebody else’s literary work as his. As we have said before, it should not appear strange that such a sophisticated concept could develop, while the apparently simpler and practical concept, such as rewarding the author of a manuscript with his rightful revenues when others made a copy of it, was unknown. 

Printing technology, and the consequent appearance of the printer/publisher entrepreneur, brought this issue to forefront, because the habit of making copies of literary works was not immediately affected by the invention of printing. Simply, what was done before by amateurs – copying manuscripts because “I need a copy for my own needs” and, by the way, “while I do this I spread culture” and “I am sure, the author would be happy if he knew that I like his work so much that I make a copy of it” – could be done with the tools offered by the new technology, admittedly with some economic rewards for those engaged in the process. 

Some entrepreneurs – most notably the Dutch printers – started doing exactly that, reprinting and selling books printed by others. The financially sensitive Britons reacted strongly and the first copyright law, known as the Queen Anne’s Act of 1709, was issued. The law starts with the grand design

to stimulate the flow of knowledge “for the encouragement of the learned Men to compose and write useful books”. 

but does not hide the real motivation, viz. that some printers and publishers

have of late frequently taken the liberty of printing, reprinting and republishing books without the consent of Authors or Proprietors of such Books. 

In the Anglo-Saxon world, this Act started a process that led to securing the position of printers/publishers in the value chain because the copyright law aims to protect entrepreneurs’ economic investments. The author, who has been so smart as to make a contract with the publisher, is also protected, the more so the smarter he has been in negotiating a good contract with the publisher. 

At the instigation of legislation enacted during the French Revolution, continental Europe saw instead the prevalence of “droit d’auteur” (author’s rights). This protects the rights of an author to be remunerated for his intellectual investment and confers  two types of rights to the author: moral rights, that give him the legal right to have his name, his qualification as an author and his work’s integrity respected and economic rights to exploit his work. The former rights correspond to those that Martial implied were affected by a “plagiarius” and the latter rights correspond to those granted by the Queen Anne’s Act, but applied to another player in the value chain. 

As was recalled before, artists are not necessarily good managers of their intellectual productions. The consequence has been that, in most countries, Authors and Composers Societies have been established to protect authors’ rights. Next to authors’ rights there are the so-called “neighbouring rights”, which refer to the rights of performers. For similar reasons, Performers Societies have also been established in most countries. 

The concept of rights associated to a literary work has gradually extended to artistic works “fixated” or imprinted on other carriers. Rights to the works fixated on the different media obey to a multiplicity of rules and conventions that depend on the way different communities in different countries have come to manage their technology-enabled businesses. For instance, in a sound recording bought by a consumer, rights belong to the composers of the music; the authors of the lyrics; the musicians for their performance; the designer of the record cover or sleeve; the author of the text on the sleeve; the photographer or artist if the sleeve contains a picture and, of course, the recording company, who may own some or all of the other rights. In a movie, the complexity of rights may even be greater, because of the generally much larger budgets involved in movie production that force the originators of the idea to share the risks with other individuals and the large number of people involved – usually quite special characters (not that people in the sound recording business are not). 

Behind the words “Proprietors of such Books” of the Queen Anne’s Act there is an implied understanding that, with the purchase of a carrier that has a literary or artistic work fixated on it, the buyer acquires rights to the atoms of the physical carrier but does not “own” the literary or artistic work whose “Proprietor” retains rights to when he, directly or indirectly, sells the book to a buyer. Of course, the publisher no longer has rights to the atoms making up the book, as for any other transaction involving physical objects. The specifics of these rights vary widely depending on the period of time considered, the type of carrier, the type of content and the country.

With the start of radio broadcasting in the early 1920s, millions of listeners could receive audio works in no time and music was one of the first forms of artistic works to be broadcasted by the so-called “radio heads”. With the start of television broadcasting some 30 years later, viewers could even receive audio-visual works. Physical carriers – mostly papers and vinyl discs at that time – so unwieldy because they needed physical delivery, ceased to be the only way to deliver literary and artistic works to end users. 

The new immaterial delivery system had every reason to be recommended, but for the absence of an immediate solution to the problem of remunerating rights holders. The new communication medium acted both as “killer” and “creator” of paying customers. “Killer” because some customers ended up no longer buying a work because they had already consumed it via the broadcast channel and “creator” because some others would be motivated to buy a work just because they had been made aware of something they had not known about, and they liked it. 

Most countries took the approach that this medium, often retained firmly in the hands of the state, was a tool to promote the well being of the masses, while some others, in the dark 1920s, ’30s and part of the ’40s, extended this view even more thoroughly to effectively control those masses. Parts of the license fees that most countries apply to this “public service” were used to pay for the right to broadcast literary and artistic works. For the countries that took the road of “commercial radio service”, broadcasters paid rights holders using the revenues from posting advertisements on their networks, but arrangements were also made so that content would be broadcast, thereby filling air space, as promotion. This, however, is by no means a complete representation of the network of agreements, some freely entered into, some brokered by governments and some mandated by law, of course all different from country to country, that has been built around this business over many decades. 

With the price of domestic recording appliances – first audio and then also video – quickly reaching levels affordable by the general consumer, anybody could make extended private libraries of audio and audio-visual works that had been received via the broadcast channel. Here, too, the problem already encountered with the appearance of broadcasting arose: what is the incentive for a consumer to buy a record or a videocassette, when a copy can be obtained for free by recording it from over the air? Without going into a debate, suffice it to mention that the ability to make copies of audio and video cassettes of broadcast works did provide consumers with the means to avoid buying the originals on disc or tape from a shop but did also create a completely new channel to deliver audio and video works for a consumption that was more personalised than was possible with broadcasting.

In general it is fair to say that these new devices found a more-than-decent place in the audio-visual business landscape by creating new outlets for the consumption of works. This is so true that the business of distributing content via Compact Cassette and VHS (and later CD, DVD and Blue-Ray)  became important components of the music and video markets.

But we will be seeing that new technologies will further change the media landscape.