MP3 has shown how the expectations of rights holders concerning the use of digital content clash with those of billions of end users. In this chapter I would like to summarise the views from the perspective of the different players. I will classify the attitudes in four categories: philosophical, technological, political and legal. 

Those who have been part of the creation of the Internet or those who, without having actually participated in it, have been raised in that spirit, take what I would call the “philosophical position“. This is greatly influenced by what has certainly been an exciting enterprise, viz. the creation of the entire technological infrastructure to enable a network of computers to exchange information seamlessly on a global scale. 

The majority of those who have taken part in this development come from an academic background. For them the project meant an unprecedented long-term sharing of resources, having the purpose of creating all the technical specifications and the software implementing them, without patents and copyright, which was eventually “donated” to mankind. This tradition is somehow continued by the OSS movement in which – take the development of Linux as an example – large-scale software projects open to anybody are realised whose results anybody can utilise, within the terms of a very broad license. 

A few of those belonging to this world make the extrapolation that, as much as the software for the network is shared, so should the bits moving through the network by virtue of that software. I am personally fascinated by the ultimate implications of this philosophical position. The idea that, at least from now on, all the results of human ingenuity, as soon as they are converted into bits, become ipso facto commonwealth of mankind, ready to be accessed, consumed, re-used and extended to create new works without waiting for the 75 or more years from the death of the author, is so innovative that it could push the evolution of mankind to totally unexpected directions. I would personally like to be a part of a global experiment of this nature. I would like to set a condition, though, that I can call myself out of this experiment if things go wrong.

Indeed, I would like to suggest that, before making reference to a philosophical position expressed in a way more reminiscent the famous ipse dixit used in Middle-Age Scholastic discussions than a dialectic and rational argument, some analysis should be made about what it would mean for an artist to be deprived ope technologiae of the incentive that, at least from historical times, has been the driver to express the fire that burned in him, the intensity of which often depended on the size of the expected remuneration. 

It is a case similar to my ancestors’ who cobble-stoned the paths criss crossing the hills because they wanted to help themselves, and a consequence, everybody, in their daily job of carrying heavy loads of firewood, grain or hay on their shoulders without any intention of sharing the loads with other farmers. I do not see why the fact that people unselfishly cooperated in building the “virtual cobble-stoned paths” of the internet should imply that the loads (the bits) moving on those paths should also become common property. 

I would also like to take this opportunity to spend some words on the deliberate, but often not so well perceived, ambiguity of the expression “free internet” (or free software or free anything, but for “free lunch” whose meaning is unequivocal), as if the network were the place (virtual, but nonetheless a place) where the sacred principles of Liberté, Égalité, Fraternité of the French Revolution at last triumph and not – at times a convenient ambiguity of the English language – the place where moving bits “does not cost” (apparently). 

The “technological position” is based on the assertion that, no matter what protection is applied to no matter what content, a sufficient quantity of human ingenuity of sufficient quality, seconded by an adequate amount of processing power, can remove it. From this comes the assessment of the futility of any attempt at protecting content to manage its consumption and hence its value, because, sooner or later, whatever the Technological Protection Measures (TPM), they will be removed. 

I have no intention of entering a dispute on the universal validity of such an argument, but I would like to start by just stating the obvious: a poorly protected piece of content can be hacked without much ingenuity and simple computers, while a smartly protected piece of content will probably take greater ingenuity and powerful computers.  I would also like to add that the position presupposes the continuation of today’s content distribution model, where users buy the right to consume content independently of the machine on which it will be consumed, which I do not think is a reasonable assumption because future content may only be consumable on secure and authenticated machines (which supporters of the philosophical position object to). 

The last comment should not be read to mean that I am supportive of legislation, proposed in some countries, that intends to impose that any processing device should contain security elements. I think these proposals are damaging progress in one of the most innovative technology fields of this century (and the past) which holds the prospect of unifying a lot of other disciplines and their exploitation. I am just saying that a rights holder (or a distributor) might wish to have content consumed only on devices that he trusts.

The “political position“, made by some civil liberty organisations and also utilised by supporters of the technological position when other arguments fail, centres around the figure of “fair use” to implicitly negate any attempt by rights holders to add TPMs to their content. The position stresses the fact that, if content is protected, it is no longer possible to do the same things that people have traditionally done with content in physical or analogue form.

Those who take this position are apparently not affected by the fact that, insisting on carrying over to the digital world acts that used to reasonable, leads to the conclusion that it is no longer possible to assign a monetary value to content. In the analogue world making a copy was always a tiresome and time-consuming effort and the copy itself was a deteriorated replica of the original that became useless after the process was repeated a certain number of times. Today the act of copying and distribution requires no effort, copies are perfect, and so remain after an indefinite number of copies. 

Starting from the Queen Anne’s Act, countless national laws have been promulgated and many international treaties signed that attempted to strike a balance between two opposing needs: providing rights holders with a remuneration for their intellectual and material efforts that produced the work, and allowing citizens some simplified or even free access and reuse of content. The inspiring principle was that, in a world where information was scarce, public authorities took upon themselves the task of easing access to information. Allowing this type of special access was in response to identified social needs and worked satisfactorily because, with analogue technologies, access, reproduction or extraction of a piece of work damages, often in a significant fashion, the effective use of the work. 

Still in the context of the political position, the claim is made by those civil rights advocates that security elements in the content consumption device are an attack to basic individual freedoms. The argument goes this way: these security elements allow those who are “at the other end of the wire” to know many things about the habits of the device owner, a clear attack to that individual’s privacy rights. Strictly speaking this is not specific of security technology, the concern exists whenever two devices communicate be they in a client-server, or a peer-to-peer configuration. 

This is the natural reaction to the “laissez faire” attitude that is characteristic of countries where there is no intention to regulate, say, the collection of data from the visits to a web site. The justification is that a market in the making, with unknown features, should not be stifled by regulation but only self-regulation should be encouraged. Originally this self-regulation worked well and indeed web sites used to publish their data handling policies and subscribers possibly decided to entrust their data to a site or not depending on the published policy. The dark side of the story is that many sites, pressed by economic difficulties or better prospects or simply because they see it more convenient, may unilaterally decide not to honour their published policies and sell or use their valuable databases in new ways. 

Therefore the problem is not so much the existence, or lack thereof, of security elements in the user device that allows those sitting “at the other end of the wire” to collect data, but the use that people who have collected the data put it to. People forget that from a century and a half there is a universally deployed system where those sitting “at the middle of the wire” know everything about those sitting “at this end of the wire” and that, since some 30 years there is a system that is more and more widely deployed where those sitting “at the middle of the wire” can even know where their subscribers are physically located when they are connected to the system. Discovering the privacy problem is not a show of maturity on the part of the discoverers. What it takes is appropriate standards, efficient legislative systems that protect the users, with appropriate deterrent measures for those abusing the system. Then, whether this should be done in a way that does not stifle a nascent market, is a separate issue. 

Among the positions taken by the different players in this space, the “legal position” of some rights holders and their representatives must also be mentioned. These respond with courts and injunctions to all those acts that they consider, in these times of shifting land, attacks to their properties. One cannot but observe that such an intransigent attitude is only matched by the unavailability to have a dialogue shown by the extremist supporters of “fair use” and civil liberties in general. At times one has the impression that some rights holders prefer a legal clash with their opponents instead of helping create a discussion place where issues are democratically, albeit dialectically, debated. This attitude is cordially reciprocated by the extremists on the other side.

I am obviously aware that each of the four major positions described above can be maintained forever with new arguments. I must also acknowledge that the key arguments of all four groups are right or at least appear to make a lot of sense.

• The philosophers are right in their claims that ubiquitous networks should benefit the citizens of the world, that intellectual progress is the result of the meeting of ideas, and that mankind should get more and not less from the innovation brought by digital techniques.
• The technologists are right that a product of a human mind can theoretically be cracked by another human mind, but so what? Do we stop putting locks on the doors of our apartments just because a skilled thief can break in?
• The politically-minded people are right to say that rights holders are such only because the authority of the state has made them so and that they have the social responsibility of giving access to their works under simplified conditions, but citizens at large have the obligation to respect in a substantial way the rights holders’ property, without hiding behind the outdated words of rights possibly enshrined in laws that are products of another age.
• Rights holders of course have the right to make recourse to the courts to protect their rights, but this is no reason for elected representatives of the people to sit idle and let tribunals decide the shape of the rights to digital media in future – actually present – society. 

Recognising that everybody is right is not of great help and is an indication that the level of discussion has been inadequate. All parties are making arguments that used to be the right ones in a different technology environment. They are no longer true today because many of the statements are reduced to empty statements of principle, without practical application.

After the memorable achievement of the PD 1.0 specification, I tried hard to convince SDMI people that the group should work on developing interoperable specifications that included specific technologies beyond the screening ones, or the entire initiative would be meaningless. My efforts were in vain and, at the Los Angeles meeting at the beginning of the 7th of July 1999 meeting – the very meeting that approved PD 1.0 – representatives of the different industry constituencies expressed one after the other their hostility to my proposal. 

That was a rough awakening. Probably thinking that representatives of large media and technology companies would work together to create a practical alternative to MP3 had been a naïve mistake on my part, but it was no time for recrimination. At that time I thought that SDMI would remain a milestone in the transition of media from analogue to digital but, unless its membership changed their mind, SDMI would at most be remembered as a precursor of the transition, not as the organisation that would actually manage to create the bright new world. Still I wanted to create the new world, even if the SDMI route was no longer attractive.

I had very intense summer holidays in 1999 and I will try to go back over the steps that led me to a formulation of a new strategy.

Looking back at centuries of history, artists have always endured a constant tension between their capability of expressing art and their ability to satisfy two basic desires: to be recognised as artists producing valuable work and to be remunerated for them (not necessarily in this order). Middlemen have always found a role propping up both desires of artists. They and their proxies have invented all sort of ways to promote artists and all sort of technologies to create, promote, deliver and enable consumption of their creations. Middlemen have often provided practical answers to the artists’ tension but, at the same time, have rendered the artists more and more dependent on the middlemen. 

With the arrival of digital technologies and the internet, people at first thought that creation, distribution and consumption of content would become radically easier and middlemen could be dispensed of. Soon, however, they discovered that these were hollow thoughts because digital technologies simply allowed the transformation of a piece of art into bits that were easier to distribute, but did not help solve any of other problems, certainly not of their remuneration.

The vision that I had when I agreed to be SDMI’s Executive Director was to empower anybody in the value chain and give them the possibility to talk to the owners of some other bits representing some interesting content, negotiate and make deals with them so those bits could be legitimately “used”. Depending on usage conditions, users could add something valuable to the bits whose rights to use they had just acquired and make something new and, hopefully, more valuable than the individual constituent parts. 

I wanted to make this process possible for all players in the value chain and let this “bit partnership” extend without limits. The more people could add bits, the more valuable the bit combination would potentially become. I did expect that, as the network of “rights to the bits” became more complex, so would the technologies making this possible. 

This idea of value chain where users add value is of course not new. If you replace the word “bit” with “content”, all that I have said above has happened for centuries if not millennia and evolved in ways that were originally not codified. But the physical world added constraints many of which can go away if digital technologies are used. 

In 1996 I first had the idea of experimenting with the creation of a marketplace of digital AV content for professional users. The European Commission funded ACTS ATMAN project was driven by the idea that, with the increasing number of content offerings that Service Providers (SP) would make over the multiplying number of delivery systems, there would be a need for more efficient means to acquire and stock up content than had been available with a reduced number of SPs handling analogue content. The purpose of ATMAN was then to define mechanisms to trade digital audio-visual material, develop and integrate hardware and software subsystems in support of the trading mechanisms so defined and test the full supply-handling-delivery chain by getting the involvement of real users. 

Fig. 1, taken from the project, was used a reference for its work. 

Fig. 1 – The ATMAN model

Content possibly created in remote places, say by a news gathering team, would be uploaded to the ATMAN server via satellite (or, if available, the fixed network) and the corresponding descriptions uploaded to the ATMAN database. Prospective buyers would search the latter for interesting descriptions and preview pieces of content stored in the former and a transaction could occur if the content, its terms of use, and the price were found interesting. Note that in 1996-97 ATM was still considered a technology that companies could make plans on for new developments. In the ATMAN case, however, ATM would only be used to upload and deliver AV content while IP would be used to search available content in the ATMAN database.

The thoughts of my 1999 summer holidays built on both the SDMI and ATMAN experience. The first opportunity to present them was offered by the WIPO International Conference on Electronic Commerce and Intellectual Property in September 1999. Then came the MPEG meeting in Melbourne in October 1999 where I presented my thoughts in a set of slides to the membership. The message extended the ATMAN idea with two constraints removed. The first was that the trading should not just apply to the Content Provider-Service Provider relationship and the second was that all types of digital content, not just AV, should be supported.

The slogan that I coined at that time was to build an infrastructure supporting electronic commerce of digital content to create a future where the (then) 6 billion humans would be enabled to take on any conceivable role that exists in the value chain: content provider, value adder, packager, service provider, consumer, reseller, etc. To make my message concrete, my slides advocated the need to create a “big picture” and make an inventory of the technologies underpinning the life cycle of media in the new digital environment. 

The big picture would be used to create a “Multimedia Framework” where the different technologies would find a place. In the framework, one would see which technologies already existed and which were missing. The latter would be developed by MPEG, if the necessary expertise existed. Otherwise, an appropriate competent body would be found and convinced to develop the missing technology. Lastly, and very much in line with its tradition, MPEG would attempt to integrate the technologies into a coherent whole. My presentation also suggested a name for this project: MPEG-21, from the century that was soon to begin. 

The proposal was well received and Keith Hill, then with MCPS, the UK organisation in charge of music licensing, a long time participant in MPEG, and an active participant in SDMI where he chaired the Functional Requirements (FR) WG, was assigned the task of converting my general ideas into a practical proposition. At the following Maui, HI meeting in December 1999, MPEG was already in a position to ask SC 29 to approve a proposal for a new project. The plan was to have a Technical Report describing the Multimedia Framework as Part 1, followed by a number of other Parts with normative value. 

A workshop was organised at the following March 2000 meeting in Noordwijkerout. The goal was to promote understanding of the project – inside and outside MPEG – with a comprehensive overview of the different issues and target industries. Several speakers from MPEG gave their views and other organisations were invited to contribute to the definition of the scope. The work formally started during the June 2000 meeting in Geneva when news of the approval of the project by JTC1 arrived. I also sounded out Keith Brannon of ITTF, my usual interface with ISO, whether it was possible to get a “good-looking” 5-digit number for MPEG-21. To my surprise it turned out that 21000 was “free” and could be allocated to MPEG-21 which then became ISO/IEC 21000 Multimedia Framework. 

At the basis of MPEG-21 there is the notion of Digital Item (DI), defined as a structured digital object with a standard representation and identification. A typical example of a DI could be a music compilation containing (references to) the music files, photos of the singers and performers, videos of the best concerts, animated graphics, lyrics, scores and MIDI files. These can be described as “static” components of the DI, but there could be “dynamic”, i.e. time-dependent components as well, such as real-time interviews with the singers, statements by opinion makers, ratings of an agency, position in the hit lists, etc. The DI, to be able to express its value to a full extent, would contain navigational information possibly driven by user preferences, rights information to use content etc. 

The DI is the basic unit of transaction between two parties as shown in Fig. 2 .

Figure 2 – Digital Items and Users in MPEG-21

Depending on the conditions of purchase, rights to the “music compilation” DI might mean to be able to access up to 2 interviews for the next 2 months, while statements, ratings and positions could be made free for the next 15 days and cost a fixed amount of money thereafter. News related to the song, such as bargains – another dynamic component – could be provided, e.g., for the next 6 months. 

In MPEG-21 “User” is any entity that interacts in the MPEG-21 environment or acts upon a DI. Therefore User is not just the “end-user”, but anybody performing such roles of acting on content to create, provide, archive, rate, enhance, aggregate, deliver, syndicate, (retail) sell, consume, subscribe, regulate and many more, very much like in my MPEG-21 vision launched in Melbourne… And the MPEG-21 mission was eventually formulated as “Defining the technologies needed to support Users to create, exchange, access, consume, trade and otherwise manipulate Digital Items in an efficient, transparent, and interoperable way”.

Part 1 of the MPEG-21 standard has the title “Vision, Technologies and Strategy“. It is not a standard but a Technical Report because it contains a description of part of the content of MPEG-21.

Digital Item Declaration (DID) is part 2. This normative part defines the technology supporting DIs. The purpose of the DID standard is to describe a set of abstract terms and concepts to form a useful model for defining DIs. The Digital Item Declaration Language (DIDL) is an XML language for defining DIs. By using DIDL the standard XML representation of a DI is obtained.

For each transaction we need a means to identify the object of the transaction. Part 3 of MPEG-21, called “Digital Item Identification” (DII), plays that role by providing the means to uniquely identify DIs. The role of DII is similar to the one played by International Standard Book Numbering (ISBN) for books, International Standard Serial Number (ISSN) for periodicals and ISRC for recordings. Its scope includes: 

• How to uniquely identify DIs and parts thereof (including resources); 
• How to uniquely identify IP related to the DIs (and parts thereof); 
• How to use identifiers to link DIs with related information such as descriptive metadata; 
• How to identify different types of DIs. 

Part 4 of MPEG-21 Intellectual Property Management and Protection (IPMP) Components provides the means to control the flow and usage of Digital Items throughout their lifecycle by specifying how to include IPMP information and protected parts of Digital Items in a DIDL document. The IPMP DIDL encapsulates and protects a part of the hierarchy of a Digital Item, and associates appropriate identification and protection information with it. This is related to MPEG-4 IPMP-eXtensions (IPMP-X), started in 1999 as part of MPEG-4 and completed in 2002. The same technology was applied to MPEG-2 and become part 11 of MPEG-2. IPMP-X defines standard ways of retrieving IPMP tools from remote locations, authenticating IPMP tools and exchanging messages between the tools used to protect a piece of content and a terminal that needs to process (e.g. decrypt, decode, present) the content. 

Already in the physical world we seldom have absolute rights to an object. In the virtual world, where the disembodiment of content from carriage augments the flexibility with which business models can be conceived and deployed, this trend is likely to continue. That is why part 5 of MPEG-21 Rights Expression Language (REL) was developed to express rights about a Resource in a way that can be interpreted and possibly acted upon by a computer. 

The MPEG REL data model for a rights expression consists of four basic entities and their relationship. This basic relationship is defined by the MPEG REL assertion “grant”. Structurally, an MPEG REL grant consists of the following:

• The principal to whom the grant is issued
• The right that the grant specifies
• The resource to which the right in the grant applies
• The condition that must be met before the right can be exercised

This is depicted in Figure 1.

Figure 1 – The MPEG-21 REL model

A right exists to perform actions on something. Today we use such verbs as: “display”, “print”, “copy” or “store” and, in a given context, we humans share the semantics of these words, i.e. what they mean. But computers do not and must be “taught” the meaning. That is why MPEG developed the  Rights Data Dictionary (RDD) as part 6 of MPEG-21 to give the precise semantics of all the actions that are used in the REL in addition to a lot more actions. However, the basic semantics is already in the REL standard. 

The digital world should give people the means to do more than just find new ways of doing old businesses. Content and service providers used to know their customers very well. They used to know – even control – the means through which their content is delivered. Consumers used to know the meaning of well-classified services such as television, movies and music. Today we are having fewer and fewer such certainties: end users are less and less predictable, the same piece of content can reach them through a variety of delivery systems and can be enjoyed by a plethora of widely differing consuming devices. How can we cope with this unpredictability of end user features, delivery systems and consumption devices? This is where Digital Item Adaptation (DIA), part 7 of MPEG-21, comes to help, because DIA provides the means to describe how a resource and/or description in a DI should be adapted (i.e. transformed) so that it best matches the specific features of the User and/or the Network and/or the Device. 

Figure 2 – The MPEG-21 DIA model

As shown in Figure 2, a Digital Item Adaptation Description specified by part 7 of MPEG-21 can be used by the (non-normative) “resource adaptation” and “descriptor adaptation” engines to produce adapted Digital Items.

Part 8 contains the usual Reference Software of the entire MPEG-21 standard. Part 9 is the MPEG-21 File Format, the “transport” format of a DI. The MPEG-21 file format inherits several concepts of MP4, as a DI may be a complex collection of information that contains still and dynamic media, information related to the DI such as metadata, layout information, etc. 

A DID is a static declaration defined using the DIDL. Digital Item Methods (DIM) are defined in Part 10 “Digital Item Processing” (DIP) to allow DI Users (authors, publishers, distributors, etc.) to add functionality to a DID, such as specifying a selection of preferred procedures by which the DI should be handled at the level of the DI itself. On receipt of a DID, a list of DIMs applicable to the DI is presented to the User. The User chooses one DIM that is then executed by the DIP Engine. As an example, for a music album DI an “AddTrack” DIM might be provided such that a user can add a new track in the preferred format. 

Back to part 3, getting an identifier for a DI is important, but how are we going to put a “virtual sticker” on it to carry the identification? This is where Persistent Association Technologies may be of help. SDMI struggled with the selection of very advanced “Phase I” and “Phase II” screening technologies and its task was made harder by the fact that no established methods existed to assess the performance of these technologies. MPEG-21 contains part 11 called “Evaluation Methods for Persistent Association Technologies” that does exactly that: a Technical Report, hence non-normative and similar to a “best practice” for those who need to assess the performance of watermarking and related technologies. 

Part 12 is called  Test Bed for MPEG-21 Resource Delivery. It is a comprehensive environment that can be used to test the effect of different conditions for delivery of media resources.

During the long study period that eventually led to the acquisition of the technologies required to develop a scalable video coding standard, it was thought that novel technologies would be required for such a form of video coding that would not fit in the MPEG-4 standard as, e.g., done by Advanced Video Coding (AVC). Part 13 Scalable Video Coding (SVC) was originally intended to host such a standard. However, when it became clear that SVC would be an extension of AVC, as opposed to a new standard, this part 13 was moved to MPEG-4 part 10 as an amendment to AVC and MPEG-21 Part 13 became void.

Conformance of an implementation is of course needed for MPEG-21 technologies as well. Therefore the purpose of Part 14 Conformance is to provide the necessary test methodologies and suites to be used to assess the conformity of a software that creates an MPEG-21 entity (typically an XML document) and a decoder (typically a parser) to the relevant MPEG-21 standard.

Many application domains require a technology that can generate an event every time a Digital Item is processed and an action identified by an action is performed. The technology achieving this is specified in Part 15 Event Reporting (ER).

Figure 3 – The MPEG-21 Event Report model

A User places an Event Report Request (ERR) in a DI. When the DI is received by a device, the ERR is passed to an ERR Receiver and parsed. An Event Receiver senses all internal and external events and passes them to an ER Builder that creates a message (Event Report) and dispatches it to the address indicated in the ERR.

Since a few years (starting from MPEG-7) MPEG has standardised a technology that allows the lossless conversion of a typically very bulky (because of its verbosity) XML document to a binary format, while preserving the ability to efficiently parse the binarised XML format. The technology was originally part of MPEG-7 Systems but was later moved to MPEG-B Part 1 Binary XML format (BiM). BiM is  essentially just a reference in MPEG-7 Part 1 Systems and MPEG-21 Part 16 Binary format.

There are cases where it is necessary to identify a specific fragment of a resource as opposed to the entire set of data. Part 17 Fragment Identification (FID) specifies a normative syntax for URI Fragment Identifiers to be used for addressing parts of a resource from a number of Internet Media Types.

While Part 9 provides a solution to transport a Digital Item in a file, Part 18 Digital Item Streaming (DIS) provides the technology to transport a DI over a streaming mechanism (e.g. in broadcasting when transport is done using MPEG-2 Transport Stream or over IP networks when RTP/UDP/IP is used.

DIS enables the incremental delivery of a Digital Item (DID, metadata, resources) in a piece-wise fashion and with temporal constraints so that the receiver may incrementally consume the DI. This is achieved by using the Bitstream Binding Language (BBL). BBL defines syntax and semantics of instructions applied to fragment a DI and maps it into a plurality of delivery channels each employing a specific transport protocol.

Figure 4 – The Bitstream Binding Language and Digital Item Streaming

Part 19 Media value Chain Ontology (MVCO) provides a normative core model of a knowledge domain that spans the full media value chain and that can be extended to represent other specialisations. Thus, the MVCO provides a common backbone for interoperable standard services and products (metadata, licenses, attribution etc.) offering new for new business model opportunities to a broad set of interconnected value chains and niches.

Part 20 Contract Expression Language (CEL) provides a standard structured digital representation of complete business agreements between parties. CEL may be used to represent contracts directly related to content or services.

The CEL features include

1. Identification of the contract and its parties,
2. Digital expression of the agreed permissions, obligations, and prohibitions, and the associated terms and conditions (deontic expressions) addressing the rights for the exploitation of intellectual property entities, including the specification of the associated conditions, together with other contractual aspects, such as payments, notifications, or material delivery.
3. The possibility to insert the textual version of the contract and/or of the specific clauses, especially when the original contract is written in natural language.
4. The possibility to add metadata related to any contract entity and encrypt the whole or any-sub-part of the contract.
5. As electronic format for a contract document, the agreement of the parties can be proved by their digital signature.

Part 21 Media Contract Ontology (MCO) specifies an ontology for expressing CEL contracts in a semantic representation.

Part 22 User Description (UD) provides standard descriptions of user, context, service and recommendation with reference to Figure 5.

Figure 5 – User Description Model

If the input (descriptions) and output (recommendation) data formats are standardised, it is possible for a user to process, e.g. combine, different recommendations to obtain one recommendations that is best suited to the user as depicted in Figure 6.

Figure 6 – Combination of standard recommendations

By 2003 the MPEG-21 project that I had kicked off three years before in MPEG, as a result of my failure to convince SDMI to move to technology specification after their successful achievement of the Portable Device specification, was in full swing. Still I could already see that no matter how successful the project would be in developing the Multimedia Framework, it would not be enough to provide the solution I was seeking. System-level interoperability, my quest since DAVIC times, could not be achieved by just making the pieces available, the complete system had to be put in place.

This realisation was mounting while my permanence at CSELT, Telecom Italia’s research centre that had just been renamed Telecom Italia Lab or Tilab, was getting harder with my mounting disagreement with the direction the center was taking after yet another change of the reference shareholder at Telecom Italia. While mulling about my future I began writing these pages collectively called “Riding the Media Bits” where I did not hide my dissatisfaction with the outcome of the grand project begun some 75 years before to move the world from analogue to digital, in particular as far as media was concerned. The potential of the digital media technologies created by the various initiatives I had started was impressive but its exploitation missed two main targets: end users, because too few of those technologies were actually being put to practical use, and creators and their proxies because in too many cases those technologies had damaging effects to them.

On my “Independence Day” (4th of July 2003) I left Telecom Italia, my employer of 32 plus years. The following day I disclosed to the vast circle of friends and acquaintances that I had created over the years the plan I had been spawning for some time, namely to start an initiative designed to bring more digital media technologies than compression to good use. Before engaging in a solution, however, there was a need to achieve a common awareness of the problem, as I had done for my other initiatives DAVIC, FIPA and OPIMA.

I could no longer rely on CSELT for that so I had to change, actually improve on, my old approach, also thanks to the internet and the WWW. So I decided to promote the creation of a grassroots movement to develop and publish a Digital Media Manifesto (DMM), a document that should identify the causes of the “digital media stalemate” and propose ways to overcome it. The virtual meeting points of the movement were the DMM email reflector, the DMM web site and a site to publish the contributions to DMM made by the community.

On the 30th of September 2003 the Digital Media Manifesto was published, after less than 3 months of work. To grasp the essence of the document I will quote the Executive Summary

Today Digital Media has been enabled by remarkably sophisticated technologies with potential opportunities for creativity, business, culture and enjoyment, as well as benefits for players all along the value chain.

However, the achievement of a full Digital Media Experience is stuck in a stalemate. There is too much at stake to simply bow to this stalemate as an inevitable presence to live with in the next few years to come, hoping wistfully that the mess will be sorted out some day soon.

The Digital Media Manifesto identifies the need for coordinated policy and technical actions needed to achieve this fuller realisation of Digital Media. The policy actions include reviewing the Digital Media standardisation process. The technical actions require, as explicit critical success factors, the development of specifications for interoperable Digital Rights Management (DRM) platforms technically open to value-chain players and for interoperable end-user devices, and the development of recommended practices for end-to-end conformance assessment.

Executing these actions is the mission of the Digital Media Project, a not-for-profit organisation whose establishment is proposed to create the policy and technical conditions for a sustainable Digital Media economy.

The text of the Manifesto goes in more analytic detail of the stalemate. The actions proposed are very clear and can be summarised by the table below

The first policy action was designed to develop effective means to enable the continuation in the digital space of the ensemble of usages, that the Manifesto calls Traditional Rights and Usages (TRU), made possible by laws and exceptions or simply practiced by the general public. Unless such usages are possible in some form users would see any digital media solutions as a poor proposition compared to the analogue solution or to the current digital solutions that are very much at the root of the “digital media stalemate”.

The second policy action was designed to leverage on the flexibility of digital technologies to phase out “analogue legacies” such as those represented by “levies” charged indiscriminately on certain devices or delivery systems just because of their potential use for digital media.

The third policy action was designed to consider the interrelation of technology, user demand, service provisioning, economics, legislation and politics, to generate a vision for improved broadband access that is synergistic with the DMM vision.

The fourth policy action was designed to create the conditions for a more efficient transfer of innovation to Digital Media via a more streamlined access to technologies that require standardisation.

The technical actions concerned the development of technical specifications for an interoperable Digital Rights Management (DRM) platform, including in particular end user devices and the means to test implementations for conformance. The vision for this was

…to make an improved Digital Media Experience economically rewarding on a global scale, legitimate for the multiplicity of players on the value chain and satisfactory for end-users, with the ultimate goal of realising a fuller Digital Media Experience.

After publishing the DMM a new DMP email reflector was established to continue practical discussions that led to the establishment of the Digital Media Project (DMM) on the 1st of December 2003 in Geneva, in the same rooms of Maîtres Jacquemmoud et Stanislas’s law firm that saw the birth of DAVIC, FIPA and MPEG-4 Industry Forum.

The DMP Statute signed by those attending the ceremony in Geneva was developed by the continuation of the DMM grassroots that so became the DMP grassroots.

Article 3 of the statutes is very clear about the key elements of the new organisation, namely

about the mission:

…to promote continuing successful development, deployment and use of Digital Media that respect the rights of creators and rights holders to exploit their works, the wish of end users to fully enjoy the benefits of Digital Media and the interests of various value-chain players to provide products and services, according to the principles laid down in the Digital Media Manifesto.

about the modus operandi:

…on the basis of open international collaboration of all interested parties: corporations and individual firms, partnerships, governmental bodies or international organisations, supporting the DMP mission and the means to achieve its goals…

about the means to achieve the goal:

…through the development of Technical Specifications and Recommended Practices enabling businesses that support new or improved user experiences, and Recommended Actions to appropriate entities to act on removal of barriers holding up exploitation of Digital Media.

and about what to do of the results of its activities

…contribute … to appropriate formal standards bodies and other appropriate entities whenever this is instrumental to achieve the general DMP goals.

While the DMP as an organisation was being set up, e.g. with the selection of Eurescom as the DMP Secretariat, the DMP grassroots started the development of a list of 88 TRUs all based on a standard template. This monumental task took almost all of 2004 to complete and was eventually endorsed by DMP.

The first DMP meeting was hosted by Eurescom at their premises in Heidelberg. The meeting developed several important elements, among which were a first list of requirements, plans for the “TRU workshop” at the following meeting in Los Angeles in April 2004 and the DMP work plan.

At the second and third meetings, respectively, a TRU workshop and a DAS workshop were successfully held. At the third meeting DMP produced the Call for Proposals on Portable Audio and Video (PAV) Devices. A large number of submissions were received at the fourth meeting where a successful PAL workshop was also held. The first Interoperable DRM Platform (IDP-1) specification was approved at the sixth meeting, 18 months after the establishment of DMP).

At the same meeting the responses to the second Call for Proposals on Stationary Audio and Video Devices were due. The second Interoperable DRM Platform (IDP-2) specification was approved at the ninth meeting, nine months after the approval of IDP-1.

The DMP went on approving the third Interoperable DRM Platform (IDP-3) specification at its fifteenth meeting DMP. The current version of the IDP specification is 3.3.

The DMM had already identified DRM interoperability, supported by open DRM specifications of appropriate protocols between value-chain users supporting the functions they perform as the practical goal to achieve. DMP has provided a definition of DRM interoperability:

The technical ability of value-chain users to perform functions through interfaces and using protocols of open specification that can be independently implemented and provide predictable results.

But “specification of DRM” is easier said than done for at least three reasons

1. There is no such thing as a “universal DRM system” to develop a standard for simply because there is no such thing as a “universal way of doing business with media”. We can only think of a range of implementations of DRM systems that satisfy the needs of specific value-chain users.
2. Digital technologies have forced changes on media value-chains and they are likely to keep on doing so. Therefore it is impossible to standardise functions performed in existing value-chains as we do not know how today’s value-chains will evolve in the future.
3. Standards are useful because they provide interoperability to users, but one must make sure that it will be possible to inject innovation into the system, because we do not know the functions that will be performed in future value-chains 

Therefore DMP engaged in the standardisation of protocols for functions at a more atomic level – that DMP calls primitive functions – which are executed between value-chain users. Examples are identification of content and devices, expression of rights to content, authentication of devices, access to content etc. Functions performed by value-chain users are obtained through combinations of primitive functions. The specification retains the ability to support new roles in the value-chain by combining existing and primitive functions.

As most of the work needed to develop primitive functions has already been done by MPEG, DMP has concentrated in the development of some missing technologies and in the creation of an integrated collection of technologies, the Interoperable DRM Platform. The resulting solution offers some distinctive advantages

1. The specifications are industry agnostic, i.e. Users are free to build a great variety of Value-Chains that suit their business models by combining just the Tools that are appropriate for them;
2. The capabilities of a Value-Chain or new Value-Chains can be extended by adding more Tools, possibly through additional standardisation;
3. The cost to access standardised Tools may be reduce because in general Tools have multiple usages and may be provided by multiple suppliers;
4. Full interoperability can be achieved within a Value-Chain;
5. An enhanced degree of interoperability can be achieved between different Value-Chains;
6. Innovation can be continuously fed into the system.

DMP has made significant investments in the development of the IDP Reference Software written in the Java language and released as Open Source Software. DMP has trademarked the name Chillout for its IDP Reference software.

DMP has also strived to bring to international standardisation the result of its activities. This is the complete list of its achievements on this space

1. Extension of the REL standard. This has triggered the development of Amendment 2, Dissemination & Capture (DAC) Profile of MPEG-21 Part 5: Rights Expression Language developed on the basis of submissions made to MPEG by DMP members
   
2. Conversion of the IPMP-X standard to an XML format. This has triggered the development of part 3 of MPEG Systems Technologies (MPEG-B) XML Representation of IPMP-X messages, again on the basis of submissions made to MPEG by DMP members
   
3. Submission of some 90% of the IDP specification to international standardisation. This has triggered the development of Part 5 of Multimedia Application Format (MPEG-A) Media Streaming Application Format on the basis of submissions made to MPEG by DMP members. This huge work has been completed in 15 months
4. Proposal for MPEG eXtensible Middleware (MXM). This has triggered the development of the ISO/IEC 23006 standard on the basis of submissions made to MPEG by DMP members. The 1st edition of Part 3 of MXM – Reference Software is entirely based on Chillout and Part 4 – MXM Protocols v. 1 is based on protocols developed by DMP. Version 2 of ISO/IEC 23006, rechristened as Multimedia Service Platform Technologies, with the MPEG-M acronym, was approved in April 2013.
5. Contribution of Open Media Marketplace (OMM) to MPEG. This has triggered a complete revisiting of MPEG-M Part 4, now called Elementary Services, and the development of Part 5 Service Aggregation. Both parts are fully based on DMP ideas and contributions made by DMP members.

DMP has also been active on the “political” front by responding to four consultations:

1. Comments on the Final Report of the High Level Group on Digital Rights Management
2. Response to European Commission on “Creative Content Online” consultation
3. Response to UK Intellectual Property Office’s request for comments on Proposed Changes to Copyright Exceptions”
4. Response to the European Commission Green Paper

In 2011 DMP started the Open Connected TV (OCTV) project with the idea that while there is a lot of talk about “Connected TV”, implementations are largely proprietary. This is a major impediment to the development of a promising new market of content, products, services and applications that can be overcome by creating the conditions for the development of such a market, i.e. by providing a specification and an industry-grade standards-based implementation. If adopted these can be used to enrich one-way TV services with support to interoperable multichannel two-way content access and delivery.

Therefore the purpose of the Open Connected TV project was defined as:

To develop a specification and an industry-grade implementation of an Open Connected TV platform based on international standards through an open collaborative process.

DMP has developed a specification and commissioned an implementation of OCTV. Since October 2012 the OCTV software is available to DMP members for commercial use subject to the OCTV licence.

In conclusion DMP has successfully executed its mission by accomplishing 3 phases of work:

1. Development of the Interoperable DRM Platform specification and experimentation with submission of proposals and contribution to the development of international standards
2. Development of the notion of and some supporting technologies for Open Media Marketplace, and submission of proposals and contribution to the development of international standards
3. Development of specification and implementation of Open Connected TV (OCTV)

The IDP specification is a set of 9 Approved Documents (AD), DMP lingo for specifications, addressing many, though not all yet, of the areas envisaged by the Digital Media Manifesto.

In the following a summary description of the 9 ADs constituting the Interoperable DRM Platform specification will be provided.

AD #1 – Value Chain Functions and Requirements

AD #1 documents the requirements upon which the Interoperable DRM specification has been developed. DMP has followed a clear process, open to any party, to develop requirements:

1. Ask a broad range of value-chain users (see below) to state their needs
2. Derive Functions (i.e. what the platform should enable) from the stated needs
3. Develop requirements for implementing Functions
4. Identify prominent use cases used to derive and test requirements
5. Issue Call for Proposals for technologies that implement Functions.

All documents produced at each of these steps have been made publicly available for comments on the DMP web site. Representatives of the following value-chain users have contributed to the process:

1. Civil Rights Associations
2. Association of People with Special Needs
3. Collective Management Societies
4. Device Manufacturers
5. Individuals
6. Producers
7. Public Service Broadcasters
8. Sheet Music Publishers
9. Telecommunication operators
10. Internet Service Providers.

AD #2 – Architecture

The Architecture document provides an overview of the operation of a Value-Chain and of the technologies that enable it.

A typical Value-Chain has the following main components:

1. Creation (including Adaptation)
2. Instantiation
3. Production
4. Content Provisioning
5. Service Provisioning
6. Consumption

Figure 1 – General schematic case of a Value-Chain

Value-Chains handle objects that represent IP Entities with Intellectual Property (IP) associated to them. In Figure 1 dotted arrows mean that the output of a User can be Used by the same type of User to make objects representing the same type of IP Entity.

For proper management in a Value-Chain it is useful to create combinations of different types of Resources with different types of Metadata and possibly other information types that DMP calls Content. DMP calls the digital Representation of Content, needed to Use Content on Devices, DMP Content Information (DCI). A User wishing to express conditions to Use a Content Item can associate a Licence to it Granting Permissions under specified Conditions. In this case the Content is called Governed Content. The party Granting Permissions is referred to as Licensor and the party receiving them is referred to as Licensee. A User who does not wish to express Conditions to Use a Content Item can do so by Releasing it without a Licence.

To enable a Device to interpret Permissions without human intervention, a machine readable language called Rights Expression Language (REL) is needed. The Licensee can be a Device, a User or a Domain (set of Devices or Users) and the Licence can be Bundled within the Content (i.e. it is part of the DCI) or not Bundled within the Content (i.e. the DCI references an external License). Other Content Elements (e.g. Resources) can be in-line or referenced. The Content Elements in Governed Content can either be in a form that allows immediate Processing, e.g. for Rendering by a Device (so-called Clear-text) or in a protected (i.e. Encrypted) form. Keys and related DRM information can be conveyed by various means.When the Device has limited capabilities it is useful to be able to make reference to a basic selection of Encryption Tools. However, when the Device does not have such restrictions it is important to be able to convey in a DCI blocks of executable code (called DRM Tools) that are required to Process various types of Governed Content.

XML is the technology selected by DMP to Represent Content. XML is very powerful and flexible but Content Represented by means of XML can easily become bulky and unwieldy. Therefore DMP has selected an XML binarisation technology that not only reduces the size of a DCI but also allows simpler Processing in a Device without loss of information. To Deliver Content between Device Entities it is necessary to Package a DCI (in binary form) and its referenced Resources. Two forms of Delivery are possible: as File (DMP Content File – DCF) and as Stream (DMP Content Broadcast – DCB – in the case of an MPEG-2 Transport Stream, and DMP Content Streaming – DCS – in the case of Real Time Protocol on Internet Protocol).

In general Users require Devices to perform the Functions proper to their role in the Value-Chain. To entrust their Content to a Device, Rights Holders must have assurance that the Device will execute Functions as expected. Device Certification is the process that is intended to provide such an assurance. This is performed by a number of organisations (Certification Agencies) that are appointed and overseen by a single root authority (Certification Authority) according to a process specified by IDP-3.3. DMP appoints the Certification Authority after approving the Authority’s Certification policies. The figure below depicts this three-layer arrangement.

Fig. 2 – Authorities appoint Agencies that Certify Entities

In general interacting Devices require the establishment of a Trust relationship between them, e.g. when they Deliver Content between them. A precondition for this to be possible is that a Device be Identified. The task of Assigning Identifiers to Devices is performed by a number of organisations (Registration Agencies) that are appointed and overseen by a single root authority (Registration Authority). DMP appoints the Registration Authority after approving the Authority’s Registration policies. The same three-layer arrangement used for Certification is also used for Identification. Content Items also require Identification. These are Assigned by Agencies appointed according to a process specified by IDP-3.3. When Identifiers are Assigned appropriate Metadata are also typically generated.

IDP-3.3 provides the following models

1. Creation Model identifying the major entities for which IP is attributed and describing their relationship to the digital objects involved in Content Creation;
2. Distribution Model identifying and describing the role of the principal Value-Chain Users engaged in distribution;
3. Delivery Model describing the two basic ways – File and Stream – in which Content can be Delivered between Devices;
4. DRM Tool Model describing the general operation of DRM Tools in Devices;
5. Domain Model describing the operation of Domains of Devices and Users;
6. Device Model identifying and describing the principal Devices employed by Value-Chain Users;
7. Import/Export Model describing how governed Content can be converted to Governed Content and vice versa;
8. Data Model identifying and describing the different types of Data specified by DMP.

AD #3 – Interoperable DRM Platform

AD #3 of IDP-3.3 provides the key technologies that are required to implement the walkthrough above. These are grouped in 3 categories: Represent, Protocol and Package.

Represent

IDP-3.3 provides a standard Representation of a number of Elements:

1. Represent Content
2. Represent Metadata
3. Represent DCI Signature
4. Represent DCI Hash
5. Represent Identifier
6. Represent Resource
7. Represent DRM Information
8. Represent DRM Tool
9. Represent DRM Tool Body
10. Represent Rights Expressions
11. Represent Rights Data
12. Represent Keys
13. Represent DRM Messages
14. Represent Authentication Messages
15. Represent Device Information
16. Represent Domain
17. Represent Domain Protocol
18. Represent Use Data
19. Represent Binary XML
20. Represent Event Report

The Represent Content technology is the DMP Content Information (DCI) that is based on the MPEG-21 Digital Item. Figure 3 offer a pictorial representation

Figure 3 – The DMP Content Information

Protocols

IDP-3.3 provides the following Protocols:

1. Protocols to Identify Entities
2. Protocols to Authenticate Entities
3. Protocol to Access
4. Protocols to Manage Domain
5. Protocols to Manage DRM Tools

Package

To deliver Content between Users it is necessary to Package Content in files or streams.

AD #4 – Use Cases and Value-Chains

AD #4 of IDP-3.3 provides a number of Use Cases showing how the IDP-3.3 Tools can be used to build Value-Chains implementing them. The Value-Chains are normative. This does not mean that IDP-3.3 inhibits different assemblies of Tools for the same or similar purpose, but only that Users who implement the Use Case by assembling the Tools as specified in AD #4 will be able to interoperate with other Users who will assemble the Tools in a similar way.

Here is a brief description of some of the Use Cases considered.

Open Release: This Use Case shows how it is possible to Release Content, e.g. on the web, in a Governed fashion, but without applying protection technologies. Open Release can, for example, enable a Creator to Release Content now with a very broad License of use without jeopardising future opportunities of other forms of Release. Open Release was contributed to MPEG and became the Open Access Application Format (ISO/IEC 23000-7).

Controlled Peer-to-Peer Distribution: This Use Case shows how it is possible to use IDP-3.3 Tools and build a community using a peer-to-peer network to share different types of Content such as User Generated Content released with, say, an Open Release Licence, and other professional Content released with, say, a more stringent Licence.

Personal Photography: This Use Case shows how IDP-3.3 Tools can be used to enhance privacy in the specific case of distribution of personal photographs.
Open Broadcast: In this Use Case a light-weight form of content governance is applied to a digital TV broadcast service that may satisfy some concerns of Content and Service Providers while not affecting the End User experience negatively.

AD #5 – Certification and Registration Authorities

Entities such as Devices (e.g. Creation Devices, Consumption Devices and Domain Managers) and DRM Tool Bodies have to be Certified before operation on the IDP. As an example, Certification constitutes an assurance for a Rights Holder to entrust his Content to a Device. Certification is carried out by a plurality of organisations dedicated to the task of Certifying Entities. To perform this task these organisations must be properly accredited by a root authority called Certification Authority. IDP-3.3 describes the process according to which DMP appoints a Certification Authority and oversees its operation. Further it provides the following elements

1. Qualification Requirements for a Certification Authority
2. Procedure to appoint a Certification Authority
3. Responsibilities of a Certification Authority
4. Responsibilities of Certification Agencies

The task of Identifying Entities such as Content, Devices and Domains is a critical one, e.g. in the case of Devices, where Identification constitutes a key element of Trust establishment. This Identification task is typically carried out by several organisations that are properly accredited by a root authority. While the operational details of Certification and Registration Authorities/Agencies are different, the process followed in appointing and supervising them is very similar.

AD #6 – Terminology

AD #6 of IDP-3.3 provides a set of terms and corresponding definitions that are used throughout all ADs. Providing these terms with consistent meanings will hopefully overcome the problem of DRM being a new field that impacts many existing fields with their own established and sometimes conflicting terminologies.

AD #7 – Reference software

The DMP Reference Software provides a normative reference software implementation of the DMP specification. This is written in Java and is available in source code under the terms of the Mozilla Public License Version 1.1 and is the goal of the Chillout project.

Chillout is organised in the following structure:

1. Core library: library of classes implementing the Primitive functions defined in the Technical Specification. This software is normative as much as the Technical Specification
2. Auxiliary library: library of classes encapsulating the functionalities of a number of modules which a Device may have as a hardware or software implementation. This library is not normative, in real Devices, these modules are supposed to be replaced by proprietary ones. However the interfaces made available by some or by all the components of this library may become part of the DMP Technical Specification.
3. Applications: a set of sample applications with the purpose of showing how to use the classes part of the Core and Auxiliary library and to provide a number of demonstrators of the DMP functionalities. This category includes a number of Devices: SAV, CCD, LPD, CPD, CID, TPD, DID, DMD. This library also includes testing applications such as those for testing Protocols, well-formedness of DCI, License, etc. The figure below shows the list of all available Devices and a possible way of inter-connecting them.

Chillout Devices currently run on Windows, Mac and Linux Operating Systems.

Figure 4 – DMP Reference software layers in Chillout

The DMP Reference Software is available from the Chillout Subversion repository. The code can be downloaded by any computer using a Subversion client, and it can be automatically built by Apache Maven 2 project management tool.

AD #8 – End-to-End Conformance

DMP Approved Documents provide Tools and guidance to Users on how to employ the DMP-specified Tools to set up Value-Chains. However, DMP Approved Documents #1 to #7 are in general not sufficient to respond to the following basic questions:

1. Has a given Content or Device Entity been made correctly according to the Technical Specifications?
2. Can the Device Entity that has passed the test of Question 1. be safely admitted to a Value-Chain, i.e. it does not compromise its integrity?

Being able to respond to the first question is a pre-requisite for building Interoperable Value-Chains based on Devices that are independently manufactured and Use independently Created Content. This is the first purpose of AD #8. The goal is achieved through the use of methodologies, test suites and software to test Entities for Conformance to the Technical Specifications.By using this material Users are in a position to:

1. Take a content item, feed it to a reference Consumption Device, i.e. a Consumption Device whose Conformance has been positively assessed, observe the performance of the Device and judge whether the content item is a Content Item;
2. Take an implementation of a protocol, install it on a device, observe the performance of the Protocol running on a reference Device that sends reference Content, i.e. Content whose Conformance has been positively assessed and judge whether the implementation of the protocol is a Protocol;
3. Take a device, e.g. a content consumption device, connect it to a number of reference Devices, e.g. Content Provider Device, Domain Management Device etc., and judge whether the implementation of the content consumption device conforms to the Technical Specifications by observing its performance when subjected to a number of test suites;
4.  Etc.

In order to build Value-Chains that Users can entrust their assets to because they expect that they will be Used as mandated, Users require means to obtain responses to the second question. This is a primary function of the Certification Authority and Agencies whose general role is described in AD #5 – Certification and Registration Authorities. Therefore AD #8 only provides general elements that help provide responses to Question 2, with the understanding that complete answers to this question with a level of satisfaction suitable for business practices in a specific environment can only be obtained from Certification Agencies.

AD #9 – Mapping of Traditional Rights and Usages to the Digital Space

The objective of AD #9 is to provide illustrative examples of:

1. Support of uses similar to those that users were accustomed to in the analogue age but using the technologies in the IDP and without compromising the integrity of a Value Chain that is based on it
2. New innovative ways of offering and consuming digital media largely inspired by TRUs that DMP calls Digital Media Business Models (DMBM).

For each TRU/DMBM supported the following is provided:

1. A rationale for the TRU/DMBM
2. One or more than one solution enabling support of the selected TRU/DMBM, each with
   1. A walkthrough complemented with the list of IDP Tools required
   2. Additional actions that may be required to enable TRU/DMBM support.

Below is a sample TRU handling Use Case #1 – Quote TRU. An important feature of digital vs. analogue media is the low threshold of entry to creation because virtually anybody is in a position to create Content of high (technical) quality using inexpensive devices and software. With more and more Content that is being posted, it becomes important for persons in their capacity of Creators to be able to make Quotes from Content that has been Released, possibly in a format that employs technical protection measures.

Tim wants to show 10 seconds from time code 1h 15m 25s of “My best quote of the year”, a movie that is only available as protected Content. Tim performs the following sequence of steps:

There are several ways in which this TRU can be actually implemented. Here follows a non-exhaustive list

1. In a given Jurisdiction the law may oblige Rights Holders to allow a User to make a limited number of Quotes of a given Content Item each with a maximum duration
2. In a given Jurisdiction a subscriber to a movie service may be entitled to make a limited number of Quotes of a given maximum duration
3. The Rights Holder of “My best quote of the year” may wish to promote his movie using the technology analysed in the walkthrough giving away a limited number of Quote Licences
4. A User may offer to send Quotes of the movie to a number of friends and be paid for the effort
5. ….

Open Media Marketplace

In 2008 DMP began discussions on Open Media Marketplace (OMM), an environment where Users perform actions on Entities using Services provided by Service Providers. The following subset of OMM requirements should indicate where OMM would lead.

1. OMM shall enable the provisioning of interoperable Services between OMM Users
2. OMM shall support Basic Services
3. OMM shall enable a Service Provider to combine several Basic Services into one Aggregated Service
4. OMM shall enable different Providers to offer the same Basic Services and the same or partially overlapping Aggregated Services
5. OMM shall enable the provisioning of interoperable Services between OMM Users

Figure 1 depicts a possible list of services that a user may need to provide services to another user.

Figure 1 – OMM services

In January 2010 MPEG issued a new Call for Proposals that included

1. OMM requirements as proposed to MPEG at the London meeting (June-July 2009)
2. Advanced IPTV Terminals (AIT) proposed by ITU-T

DMP members submitted a response to the call in April 2010 and contributed to the development of new parts 1-5 and part 5 of MPEG-M.

Open Connected TV

Open Connected TV (OCTV) seeks to enrich one-way TV services with two-way interoperable and multichannel content access and delivery. The OCTV Specification enables the creation and the OCTV software and accelerates the development of a broad market of products, content, services and applications that enrich access to and delivery of one-way TV services with two-way interoperable multichannel content.

Figure 1 shows the model OCTV chain that has been developed

Figure 2 – An OCTV chain

Currently DMP is considering a 4th phase, the development of a specification to enablea user to

1. Access any service using a device (e.g. Set Top Box, Connected TV set, Mobile handset) purchased from any manufacturer
2. Access any service using an EPG acquired from any developer
3. Search EPG instances for content referenced in the EPG
4. Purchase an object in an application or in an EPG used to access a service provider
5. (A user to be authenticated by any service provider or EPG developer)

as depicted in the figure below where

CP: Conrtent Provider
SP: Service Provider
STB: Set Top Box
EPG: Electronic Program Guide

Figure 3 – DMP phase 4 reference model

At the end of May 2005, I was invited to make a speech at an internal school of Mediaset (the main private broadcaster in Italy) in Cologno Monzese, near Milan. On that occasion I met Juan Carlos De Martin of the Polytechnic of Turin and Massimo Travostino, an attorney specialising in Intellectual Property. On our way back we discussed the idea of “doing something for Italy” in the space of Digital Media.

It took five months of preparation, but on the 5th of November we were able to open a workshop with a large participation from various sectors of the Italian industry and society. Workshop participants agreed on the name Digital Media in Italia, the acronym (including domain name) dmin.it and the mission

an interdisciplinary, open, non-profit group with the purpose to define action areas that enable Italy to play a primary role in the exploitation of the global “digital media” phenomenon.

By “digital media” dmin.it means any digitally-represented content that can be carried by digital networks and processed by digital devices, particularly if these are programmable.

This was for me an opportunity to put into practice a number of ideas that I had hatched in the last 20 years and that I have illustrated in the previous pages:

1. Digital media has the potential to disrupt existing media businesses: just look at the MP3 and DivX phenomena
2. Compression is the enabling factor of everything that relates to digital media: just look at the way TV channel multiplication changes the nature of broadcasting
3. Content management and protection is the key enabling technology for content distribution and monetisation: depending on whether management and protection technologies are open of closed radically different business models can be deployed

Italy was a very interesting case study for implementing my ideas: a midsize country, with a content industry of a significant size, but not enough to make it a sustainable global player. Left to market forces Italy ran the risk that it would become a “land of conquest” of the big international Content and Service Providers because the content distribution business models of the analogue world would simply be reproduced in the digital domain. Indeed already at that time it was already clear that the many innovation attempts made since introduction of digital technologies had failed and what remained were digital value chains that looked very much like old analogue value chains. Even new and promising open distribution methods such as Creative Commons licences have a hard time to show their innovation potential.

On the other hand, if the possibilities offered by digital technologies that I have illustrated in the previous pages were exploited with an open mind, Italy could hope to play a role by creating a homogeneous market underpinned by some technologies that the group would specify. The slogan adopted was

The goal of dmin.it is to maximise the flow of digital media.

The first workshop was followed by a series of meetings that led to the approval of the Draft dmin.it Proposal in September 2006. Then it took another 15 months before the Complete dmin.it Proposal could be given the final shape.

The dmin.it proposal provides the necessary technological, normative and governance tools to realise the objective that dmin.it had set to itself. More specifically the proposal identified, specified, and defined the conditions for an effective use of three key technologies:

1. Format for distribution of digital media (iDRM).
2. Access to broadband digital networks (iNet).
3. Pay and cash based on points (iPay).

dmin.it dodnot propose a standardisation of technologies that excluded the use of other technologies by the force of law. Indeed an action of this kind, even though its benefits were historically assessed, did not correspond to the trend. dmin.it simply proposed that, next to technologies selected by value chain operators to satisfy their needs, the interoperable dmin.it technologies be used to satisfy end user needs.

Concerning the format, dmin.it observed that Digital Rights Management (DRM) technologies, both for the purpose of management and protection, were used pervasively in the market for controlling distribution of digital media and that there were international treaties, obligations coming from the European Union and national laws that regulated rights and duties stemming from the use of DRM. Therefore dmin.it assessed that any proposal aiming at substantially altering such a legislative context would be unrealistic.

On the other hand it was widely recognised that use of proprietary DRM technologies entailed significant impediments to:

1. Contents providers (authors, composers, interprets and producers) who needed millions of consumption devices deployed – an extremely expensive undertaking – in order to be able to distribute their works.
2. Consumers who had to stand significant costs in order to be equipped of all devices that were required to access all contents of their interest and find it difficult to move content between different devices.
3. Service providers who typically found it difficult to hook up to value chains.
4. Italian companies who found it difficult to operate in the national market fragmented and dominated by multinational behemoths.

Therefore dmin.it proposed that the law determine that operators who release content with proprietary DRM technologies must release the same content also using the “interoperabile DRM” (iDRM) technologies specified by dmin.it, at conditions that are not discriminatory if compared with those used by the operator to release content with proprietary DRM technologies.

Concerning access to broadband digital networks, dmin.it proposed that the law determine that a broadband digital network operator can offer access to his network based on freely determined technical characteristics, provided a network user (content provider, intermediary or end user) may request and obtain from that broadband digital network operator the raw “service-agnostic” access to “big Internet”, according to dmin.it specification, with the technical features adopted by the operator his offer and at conditions not discriminatory if compared with those used by the operator for his own offer.

Concerning pay and cash, dmin.it proposes that anybody, within the terms of banking regulation, may offer “account” services that are not directly monetary (points, credits) for transactions connected to the use of digital media, as specified by dmin.it. Transactions could be effected between “accounts” that relied on payment instruments with guaranteed cashing, e.g. bank account, credit card, prepaid card, electronic purse etc. Synchronisation of an “account” with its payment only effected at fixed times or on demand.

dmin.it believed that making it possible for

• Authors/performers/interpreters/producers to release their content with the assurance that there is broad installed basis of consumption devices capable of interpreting the content
• Consumers to access and use a large content base
• Intermediaries to hook up easily to content value chains

was a potent factor enabling the attainment of dmin.it objectives.

Therefore dmin.it, instead of proposing an exclusive standardisation of format, access and pay/cash technologies, proposed to place next to the tools that operators freely decide to adopt for their needs those proposed by dimin.it. The value added brought by dmin.it tools lied in the fact that users were assured that an interoperable mode existed.

The dmin.it proposal identified an equilibrium point between the demand of operators to retain the freedom to select those technologies that best match the needs of their business and the demand of interoperability coming from the two end points of the value chains – authors, performers, interpreters and producers on one end and consumers on the other hand – and from intermediaries who decided to base their business on ’interoperability”.

dmin.it believed that its proposal could create a potential homogeneous market of 60 million people capable of stimulating both the national cultural industry and innovation in the market of digital media thus helping the national cultural and technology industry to achieve a leading position in the global market.

Format technologies (iDRM)

For format technologies, dmin.it proposed that an interoperable Digital Rights Management (iDRM) specification be adopted at the national level.

dmin.it adopted the NIST definition of DRM as “A system of Information Technology components and services that strive to distribute and control content and related rights with digital technologies”. The “i” prefixed to DRM meant that dmin.it OLNY supported the interoperable, i.e. standard, version of DRM. dmin.it DID NOT support (but was not and might not be against) the use of non-interoperable DRM if parallel to iDRM.

Therefore ,dmin.it proposed that the law determine that service providers who released content using a proprietary DRM technology should ALSO release the same content using the iDRM technology at not discriminatory conditions if compared with those employed to release content using the proprietary DRM technology. This is depicted in Figure 1.

Figure 1 – Distribution of content with proprietary and interoperable DRMs

The iDRM specification is based on the DMP Interoperable DRM Platform (IDP).

Anybody may (but is not obliged to) implement (hardware and software) devices and services, request and obtain, if the implementation qualifies, a conformity certificate and offer them to third parties with an “iDRM” mark so that no conflicts are created.

The iDRM specification was made public, and implemented in Open Source Software with a Mozilla Public Licence (MPL 1.1) licence. It did not prescribe specific business models, in line with the part above related to the meaning of “Management” in the iDRM acronym.

The iDRM specification was not comparable with the specifications of the “black box” and monolithic DRM solutions found in the market, because iDRM is a toolkit specification. This approach allowed users to implement value chains corresponding to a large number of business models with improved levels of interoperability, using the same basic tools configured to suit their needs.

Access technologies (iNet)

dmin.it proposed that for access to digital broadband networks:

1. Digital broadband network operators may offer bundled and/or unbundled access to their networks with technical characteristic of their choice.
2. A network user (content provider, intermediary or end user) could request and obtain from a digital broadband network operators the raw “service-agnostic” access to the “big Internet” according to the iNet specification within the technical characteristics already part of the operator’s offer at non discriminatory conditions if compared with the other offers by the operator.
3. Digital broadband network operators guarantee network service interoperability by agreeing on and providing specific levels of quality of service (QoS) at peering points so as to provide network users appropriate levels of QoS.

Figure 2 depicts the dmin.it proposal for access to digital broadband networks.

Figure 2 – Open access to digital broadband network

Pay/cash technologies (iPay)

dmin.it proposed for pay/cash of content that:

1. Anybody, within the terms of banking regulation, could offer not directly monetary (points, credits) “account” services for transactions connected to the use of digital media, according to the iPay specification;
2. Transactions are effected between “accounts” that rely on payment instruments with guaranteed cashing, e.g. bank account, credit card, prepaid card, electronic purse etc.;
3. Synchronisation of an “account” with its payment means was effected at fixed times or on demand.

Figure 3 depicts the proposal.

Figure 3 – System to record and convert points for digital media

dmin.it proposed its payment protocols to MPEG and they became part of MPEG-M Part 4 for “Transactions”.

Additionally, dmin.it explored changes required to the Italian legislation to accommodate the dmin.it proposals and found that only small changes to the copyright & telecom legislation were needed for iDrm and iNet, while a Virtual Account Service Provider (VASP) could be a Payment Institution of the EU Payment Service Directive.

dmin.it also identified the tasks of a body governing the dmin.it ecosystem:

1. To manage the system in the interest of all parties
2. To manage the certification process
3. To manage the specification life cycle
4. To resolve disputes between parties
5. To ensure financial sustainability

Starting from the early 2000s, several sessions were held to discuss about “what else does industry need from MPEG?” in the typical MPEG anticipative fashion. Finally at the Munich meeting (March 2004) MPEG came to the conclusion that while MPEG had produced many component standards, the integration of technologies, save for the canonical Audio-Video-System integration, had been left to implementers. The result has been that, e.g. ATSC uses MPEG-2 Systems and Video but a different Audio than specified by MPEG, and DivX uses MPEG-4 Visual, MP3 and AVI. Therefore while (some) components are standard and provided interoperability, the complete service package or application may not be interoperable.

Obviously implementers want to retain the freedom to make individual decisions in terms of what components technologies they should use for their products and services, but this is paid by end users who are confronted with incompatibilities between different system implementations.

So MPEG decided to enter the “system integration” area. This was done because MPEG has knowledge about (most of) the technologies needed by complete digital media solutions, internal expertise to do the integration and the appropriate industry representation.

Multimedia Application Formats (MAF) is the name of the new standard suite that bears the nice looking ISO number 23000, obtained through the well-known channels. MAFs specify how to combine metadata with media information typically, but not exclusively, packaged in an MPEG-4 file format to enable media interchange, management, editing, and presentation.

Part 1 “Purpose for Multimedia Application Formats” is, like MPEG-21 Part 1, an informative document outlining the scope of the ISO/IEC 23000 suite.

Part 2 “Music Player Application Format” (MPAF) specifies a format to carry MP3 coded audio content in MPEG-4 File Format augmented by a JPEG image for cover art and simple MPEG-7 metadata commonly expressed in ID3 tags, such as Song title, Album title, Artist, Year, Comment, Track, and Genre.

Figure 1 – Creation of a Media Player Application Format file

Assume that you have an MP3 file with ID3 metadata. An MPAF encoder extracts the ID3 tags and converts the MP3 bitstream to a format that allows the definition of Access Units and creates an MP4 file that can also contain JPEG images. The MPAF standard also enables album functionality by making reference to MPEG-21. It allows to collect several song files in the above described song file format into one album file. The second edition of the MPAF has added protection feature by incorporating AES-128 counter mode encryption as default protection tool and MPEG-4 IPMP-X. MPEG-21 IPMP and REL for protection and governance description.

Part 3 “Photo-Player Application Format” (PPAF) specifies a format to carry JPEG images and their associated MPEG-7 metadata in an MPEG-4 file to enable creation, sharing, searching and viewing of digital photo albums.

The supported metadata include image-acquisition parameters (e.g. date, time and camera settings) expressed as EXIF metadata and MPEG-7 visual content descriptions expressed as binary MPEG-7 metadata. The latter allow new, content-enhanced functionality, such as intelligent browsing, content-based search or automatic categorisation.

The following standard technologies are employed:

1. MPEG-7 Visual tools to describe visual properties of the images
2. MPEG-7 MDS tools to carry simple generic metadata
3. MPEG-7 System tools to binarise the metadata
4. MPEG-4 File Format
5. JPEG
6. EXIF (EXchangeable Image format)

The last is not an MPEG standard but it is supported because of its universal use.

The following EXIF tags are mapped into MPEG-7 descriptions:

1. Artist
2. ImageDescription
3. UserComment
4. GPS Latitude/Longitude/Altitude
5. FileDateTime

The following MPEG-7 Visual Metadata are used in PPAF:

1. Dominant Color Descriptor to characterize the color information when there is a small number of colours
2. Scalable Color Descriptor for image-to-image matching and retrieval based on colour features
3. Color Layout Descriptor for image-to-image matching and to visualise image appearance
   
4. Color Structure Descriptor for image-to-image matching
5. Edge Histogram Descriptor to retrieve images with similar semantic meaning
6. Homogeneous Texture Descriptor for accurate search and retrieval.

Part 4 “Musical Slideshow Application Format” (MSSAF) is a superset of MPAF and PPAF in the sense that it combines the features of both AFs. Additionally MSSAF employs MPEG-4 Part-17 “Streaming Text Format” for timed text and  MPEG-4 Part-20 “LASeR” Mini Profile.

Figure 2 – An example of MSSAF content.

Part 5 “Media Streaming Application Format” specifies how to use some MPEG technologies to build a full-fledged media player for streaming governed and ungoverned content. By referring to appropriate standards, MSAF defines the data formats exchanged between a number of devices used in a media streaming scenario: a Content Provider Device, a Licence Provider Device, an IPMP Tool Provider Device, a Domain Management Device and a Media Streaming Player.

In the most general case a Media Streaming Player obtains streaming content from a Content Provider Device using a Content Access Protocol. In order to use that content, a Media Streaming Player obtains a licence from a Licence Provider Device using a Licence Access Protocol. Further, to actually process the content, a Media Streaming Player may need to obtain the appropriate IPMP Tools from an IPMP Tool Provider Device using an IPMP Tool Access Protocol, as shown in Figure 1.

Figure 2 – Reference diagram of MSAF standard

Part 6 “Professional Archival Application Format” (PAAF) provides a standardised packaging format for digital files. The format is an implementation of the information package specified by the Reference Model of Open Archival Information System (OAIS), a framework for long-term digital information preservation.

PAAF specifies metadata formats to describe

• the original structure of digital files archived in a PAAF file
• context information related to a PAAF file and digital files archived in it
• information required to reverse the pre-processing applied to digital files prior to archiving them in a PA-AF file

and it specifies a file format for carriage of the metadata formats and digital files.

The following technologies are used:

1. Technologies from Music Player MAF
2. MPEG-21 IPMP Components Base Profile
   
3. MPEG-21 REL MAM Profile
4. ISO Base Media File Format 2nd Edition for:
   1. ‘stsd’ (sample description box)
   2. ‘sinf’ (protected scheme box) – to protect audio track and signalization of the protection format
   3. ‘ipro’ (item protection box) to protect metadata
5. ISMACryp 1.1 and OMA2.0 DCF to enable random access to encrypted content
6. AES128 CTR encryption tool (default encryption)
   

Part 7 “Open Access Application Format” (OAAF) is a packaging format designed for the release and exchange of content whose rights are owned by users who wish to release the content so that other users can freely access it, butwithout making the content “public domain”. This standard is the result of a proposal of a DMP use case.

OAAF packages different content types into a single container file and provides a mechanism to attach metadata information, by using a series of technologies, in particular

• MPEG-7 to describe the resource
• MPEG-21 REL to model the intentions of the licence
• MPEG-21 Event Reporting to provide a feedback mechanism, which can notify the author, when a user wants to derive a content or extract an item out of the container file.

Part 8 “Portable Video Application Format” (PVAF) defines a format for the use of video files on portable devices to give users the possibility to use a local resource interactively as shown in the figure below

Figure 3 – Portable Video AF content

PVAF enables playback of content

1. Generates by user on his own for PVP (downsize and encode), e.g. UGC content
   
2. acquired from package media
3. acquired from the internet

PVAF uses the following technologies:

1. ISO Base Media FF, MPEG-4 / AVC FF
2. MPEG-4 AVC Baseline Profile, Level 1.2
   
3. MPEG-4 HE-AAC Profile, Level 2, Stereo 48kHz
4. MPEG-4 Streaming Text Format for subtitles
5. MPEG-7 Metadata for movie information like film title, etc.
6. JPEG ISO Standard for still images of e.g. movie posters
7. MPEG-7 MDS: UsageHistory DS, HierarchicalSummary DS
8. MPEG-21 File Format and DID
9. MPEG-4 LASeR.
   

Part 9 “Digital Multimedia Broadcasting Application Format” (DMBAF) specifies how to combine the variety of resources and associated information of the mobile TV Digital Multimedia Broadcasting (DMB) service. Users can acquire and consume information anywhere, in a well-defined file format that facilitates interchange, management, editing, and presentation of DMB content as illustrated in Figure 4.

Figure 4 – Digital Multimedia Broadcasting AF scenario

Part 10 “Surveillance Application Format” (SAF) specifies the combination of audio and video content drawn from MPEG technologies, related metadata and file format, suitable for surveillance.

Part 11 “Video Stereoscopic Application Format” (VSAF) defines a format for a creator to take and for a service provider to distribute stereoscopic images, enabling users to have more realistic experiences (with or without special glasses) and to store the stereoscopic content for possible redistribution.

Part 12 “Interactive Music Application Format” (IMAF) defines a format to package interactive music content with audio tracks before mixing, in order to allow users to freely control individual audio tracks. The producer can create several versions (producer mixing 1, producer mixing 2, karaoke, rhythmic etc.) with just one piece of music, using the metadata structure for mixing information. Figure 5 illustrates an example of IMAF use.

Figure 5 – Interactive Music AF example

More parts are being developed but they will be introduced later.

The successive waves of audio-visual technologies experiences by consumers in the last few decades are teaching one important lesson: every new technology creates a discontinuity Those who have created Super8 films and did not create a VHS copy risk losing their past. The same may soon happen to those who did not convert their Betamax or VHS cassettes to DVDs. Blu-Ray players seem to give more assurance because they play CDs, DVDs and Blu-Ray discs (and what about DivX?), but what if in a few years devices based on solid state technologies will be the only practically used to record media? I mention the case of old PowerPoint 95 presentations that can no longer be opened with the latest PowerPoint programs to signal the fact that preservation does not concern just the “physical layer”, but also the data format layer.

What can be a nuisance in the life of a consumer can be a big headache for organisations with the mission to preserve multimedia content that represent the collective memory of a community. They need to perform processes such as digitisation of analogue content or migration of digital content from one generation to the other. Large organisations typically perform these processes in house but smaller organisations outsource these processes to service providers.

Many organisations – archives, libraries, musea, etc. – already have digital preservation systems in place and they often need to exchange multimedia assets and related metadata, for example:

• To exchange assets between preservation systems/repositories within the organization or with related organizations,
• To change/upgrade their preservation systems,
• To exchange content with service providers
• To provide preservation services for other organisations.

At the time multimedia assets are exchanged there is a need to include preservation metadata as well in order to enable the receiving organisation to assess the integrity and fidelity of the assets it receives and to establish a baseline for its own curation and use of the assets. In addition to these metadata, the receiving organisation also needs information about any past preservation processes the assets and their descriptions that may include metadata about content, structure, and quality, as well as technical, historical and editorial information, and about ownership and usage rights and conditions.

Multimedia Preservation Application Format (MP-AF) is a standard that defines the content and format of Multimedia Preservation Description Information (MPDI), in order to facilitate interoperability between preservation systems, ensure accurate understanding of the resources’ exchanges, and reduce the risks of corruption both in the exchange and thereafter.

MPAF codes the following concepts relevant for multimedia preservation and used in preservation processes.

• Provenance documents the chronology of events regarding the creation, modification, ownership and custody of resources. It provides information on the history of the multimedia content (including processing history);
• Context describes the circumstances that resulted in the production of the audiovisual resource and how it relates to other relevant resources (e.g. why and how the resource was created, from which resources it was derived,  the relationship to other resources available in the package etc.);
• Reference represents the information used for identifying and addressing the multimedia content and related resources. Reference information supports the linkage of identical or related resources that might be stored in separate repositories;
• Quality encompasses information related to the qualitative or quantitative measurements of a given resource. It supports reasoning and evaluation of how good the resources have been preserved;
• Fixity encompasses the information ensuring that resources are not altered in an undocumented manner.
• Integrity represents the state of an entity (e.g. digital item) indicating the quality of being complete.
• Authenticity encompasses information that enables an agent to verify if an object is correctly identified and free from (intentional or accidental) corruption;
• Rights encompass information concerning legal, regulatory or contractual provisions affecting ownership, control, access or use of resources, as they impact the long term preservation (e.g. intellectual property, copyrights, privacy, etc.).

Here is a non-exhaustive list of technologies used in MP-AF

1. MPEG-21 DII and DID supports preservation metadata such as provenance metadata, item identification and description
2. MPEG-21 DIsupports structural relationships between the items
3. MPEG‑21 Digital Item Semantic Relationships expresses relations between Digital Items
4. MPEG-21 Rights Expression Language (REL, part 5) and MPEG-21 Contract Expression Language (CEL)/Media Contract Ontology (MCO) support rights metadata.

Publish/Subscribe (PubSub) is an established communication paradigm where senders do not communicate information directly to the intended receivers but rely instead on a service that mediates the relationship between senders and receivers. Specifically senders (called Publishers) post information to and receivers (called Subscribers) declare their interest in specific information to a Service. Here the mediator service is called Match Service Provider (MSP).

It should be noted that “Publisher” is not necessarily a user who distributes media to end users and “Subscriber” is not necessarily synonymous with end user or consumer. Indeed Publisher may represent a user who has created a media item and wants to make it available to publishers and Subscriber may represent a publisher who is looking for media to be distributed. Therefore Publisher could very well be a creator who announces the availability of his latest work and Subscriber could be a publisher who is looking for new works to publish.

A typical steps of a Publish/Subscribe workflow in a media distribution context are given in the table below, where “Resource” is the media item that is announced (i.e. published) by Publisher and will be eventually consumed by a Consumer.

Note that steps 2 and 3 may also happen in reverse order, i.e. a Subscriber can look for information before the information is actually published. In other words there is no specified order for Publications and Subscriptions to be made.

Publish/Subscribe Application Format (PSAF) allows parties of the PubSub communication model to support functionalities that are specific to publication, search and consumption of media content (Resource). The PubSub communication parties are:

1. Publishers
   1. Create and store Resource Information (RI) on a publicly accessible location (e.g. an Information Centric Network)
   2. Create and send Publications Information (PI to Match Service Providers (MSP)
2. Subscribers create and send Subscriptions Information (SI) to Match Service Providers (MSP)
3. Match Service Providers (MSP)
   1. Perform matches of Subscriptions with Publications
   2. When a Match has been found send Notification Information (NI) to users listed in PIs and SIs
4. Consumers
   1. Receive Notification Information (NI)
   2. Access the Resource present in or referenced by the Resource Information (RI).

The PSAF standard specifies RI, PI, SI and NI in such a way that typical multimedia communication requirements can be supported. A non-exhaustive list of such requirements is

• Publisher shall be able to
  • Describe the Resource for cinsumption purposes
  • Define the contract that will bind the Publisher and a Consumer
  • Request a device to report use of the Resource
  • Describe the Resource for distribution purposes
  • Define the contract that will bind the Publisher and a Match Service Provider (MSP) of a match involving the Publication
  • Request the MSP to notify/not to notify a list of users
• Subscriber shall be able to
  • Query the Resource
  • Define the contract that will bind the Subscriber and a Match Service Provider (MSP)
  • Request the MSP to notify/not to notify a list of users of a match involving the Subscription

PSAF does not specify the protocols used to carry these payloads.

The Publish/Subscribe communication paradigm, adapted to creation, publication and retrieval of media content, is represented by the workflow and figure below (the numbers in the list refer to the same numbers in the figure)

Publisher Stores Resource (1) Creates RI Stores RI (2) Creates PI Publishes PI (3) Subscriber Creates SI Subscribes SI (4) Match Service Provider Finds Match between SI and PI (5) Notifies Publisher and Subscriber (6) Consumer Retrieves RI (7) Retrieves and plays Resource (8)

Figure 1 – A graphical representation of PubSub for multimedia

NB: Consumer is the user receiving NIs. It may or may not coincide with Subscriber or Publisher.

CEDEO has developed the main contributions t the PSAF standard and implemented PSAF in the context of the European FP7 project GreenICN, where it has developed WimICN, a video distribution platform for ICN depicted by Figure 1.

Figure 2 – The WimTV service platform

WimICN supports two forms of video services: on demand and live. The latter takes two forms: a normal live event and a scheduled event that is a combination of on demand videos and live events. Video upload and live event/scheduled program creation are performed by WimTV (the white blocks in the figure above), while PubSub, and the put/get of MPD and segments to/from ICN are performed by the other GreenICN system components (the light blue boxes in the figure above). User devices are mobile handsets running the WimCom (ccnx) and WimPeer (publication-subscription-consumption) on Android.

Of particular note is the ICN Gateway which operates in two modes

1. When a Publisher selects one of the videos that he has previously ingested to ICN from his private area (called WimBox), the Segmenter creates DASH MPD and segments and sends them to the ICN Gateway for storage to ICN. The PubSub Application receives the MPD on the Publisher’s ICN PubSub channel and creates RI and PI using other metadata provided by Publisher.
2. When a Publisher selects one of the live or scheduled programs that he has previously created from his private list, WimTV sends the Publisher information regarding the live event or scheduled programs. The PubSub Application creates RI and PI using other metadata provided by Publisher.