MPEG-4 Inside – Graphics

MPEG-4 Inside – Graphics

At the Tokyo meeting in July 1995 Hiroshi Yasuda showed up and proposed to address the coding of information that is partly natutal (e.g. a video, a music) and partly synthetic (e.g. 2D and 3D graphics). Work started in earnest, first as part of the AOE group and then, after Cliff Reader left MPEG, in the Synthetic-Natural Hybrid Coding (SNHC) subgroup. Peter Doenges of Evans and Sutherland, a company that had played a major role in the early years of development of the 3D Graphics industry, was appointed as its chairman. The results of the first years of work were 2D and 3D Graphics (3D mesh) compression, Face and Body Animation (FBA), Text-To-Speech (TTS), (Structured Audio Orchestra Language (SAOL), a language used to define an “orchestra” made up of “instruments” downloaded in the bitstream and Structured Audio Score Language (SASL), a rich language with significant more functionalities than MIDI.

A face object  is a digital representation of a human face intended for portraying the facial
expression of a real of imaginary person while it moves e.g. as a consequence of his speaking activity. A face object is animated by a stream of face animation parameters (FAP)  encoded
at low bitrate.  The FAPs control key feature points in a mesh model of the face to produce
animated visemes for the mouth (lips, tongue, teeth), as well as animation of the head and facial features like the eyes.  A face model can also be manipulated at the receiving end.

It is possible to animate a default face model in the receiver with a stream of FAPs or a custom face can be initialized by downloading Face Definition Parametres (FDP)  with specific background images, facial textures and head geometry.

The ‘facial animation object’ can be used to render an animated face. The face object contains a generic face with a neutral expression that can be rendered as is. The shape, texture and expressions of the face are controlled by Facial Definition Parametres (FDP) and/or Facial Animation Parametres (FAP).


Figure 1 – Face Definition Parametres

Upon receiving the animation parameters from the bitstream, the face can be animated: expressions, speech, etc. and FDPs can be sent to change the appearance of the face from something generic to a particular face with its own shape and texture. If so desired, a complete face model can be downloaded via the FDP set. However, face models are not mandated by the standard. It is also possible to use specific configurations of the lips and the mood of the speaker.


Figure 2 – Face Animation Parametres

The Body is an object capable of producing virtual body models and animations in the form of a set of 3D polygonal meshes ready for rendering. Here, too, we have two sets of parameters defined for the body: the Body Definition Parametre (BDP) set, and the Body Animation Parametre (BAP) set. The BDP set defines the set of parametres to transform the default body to a customised body with its surface, dimensions, and (optionally) texture. The BAPs will produce reasonably similar high level results in terms of body posture and animation on different body models.

In the area of synthetic audio two important technologies are available. The first is a Text To Speech (TTS) Interface (TTSI), i.e. a standard way to represent prosodic parameters, such as pitch contour, phoneme duration, and so on. Typically these can be used in a proprietary TTS system to improve the synthesised speech quality and to create, with the synthetic face, a complete audio-visual talking face. The TTS can also be synchronised with the facial expressions of an animated talking head as in Figure 3.


Figure 3 – TTS-driven Face Animation

The second technology provides a rich toolset for creating synthetic sounds and music, called Structured Audio (SA). Newly developed formats to specify synthesis algorithms and their control, any current or future sound-synthesis technique can be used to create and process sound in MPEG-4.

3D Mesh Coding (3DMC) targets the efficient coding of 3D mesh objects, polygonal model that can be represented in BIFS as IndexedFaceSet, a node representing a 3D shape constructed with faces (polygons) from a list of vertices, and Hierarchical 3D Mesh node. It is defined by 1) vertice positions (geometry), 2) face and its sustaining vertices association (connectivity), and optionally by 3) colours, normals and texture coordinates (properties).
3DMC can operate in a basic mode with incremental representations of a single resolution 3D model, and in optional modes:  1) support for computational graceful degradation control; 2) support for non-manifold model; 3) support for error resilience; and 4) quality scalability via hierarchical transmission of levels of detail with implicit support for smooth transition betw
een consecutive levels.

At the Melbourne meeting in October 1999 Euee S. Jang, then with Samsung, took over from Peter to complete FBA and address the important area of 3D mesh compression, in particular efficient encoding of generic 3D model animation framework, later to be called Animation Framework eXtension (AFX) and to become part 16 of MPEG-4. At the Fairfax meeting in March 2002 Mikaël Bourges-Sévenier took over from Euee to continue AFX and develop Part 21 MPEG-J Graphics Framework eXtensions (GFX).

The MPEG-4 Animation Framework eXtension (AFX) — ISO/IEC 14496-16 — contains a set of 3D tools for interactive 3D content operating at the geometry, modeling and biomechanical level and encompassing existing tools previously defined in MPEG-4. The tools available in AFX and related illustrations are summarized in Figure 1.

Tool name Objective Example
Parametric curve and surface representations Delivering smooth shapes with a high level deformation control  synthetic_face
Subdivision Surfaces Simplification and progressive transmission of large scale models Subdivision_Surfaces
MeshGrid Surface Representing generic models preserving volume information, and offering versatile manipulation features  MeshGrid_Surface1MeshGrid_Surface2
Footprint Based Representation Simplification and progressive transmission of object based on footprints (buildings, cartoons, etc)  Footprint-Based_Representation
Depth Image-Based Representation 3D photorealistic display of objects from a set of images  Depth_Image-Based_Representation
Depth Image-Based Representation Version 2 High-quality rendering of image- and point-based objects  Depth_Image-Based_Representation_V2-1Depth_Image-Based_Representation_V2-2
Multi-Texture Provide multiple textures for natural appearance together with view-adaptive real-time weighting Multi-Texture
Morphing space Combining bilinear interpolation of several target shapes with a base shape in order to obtain precise deformations and smooth animation Morphing_space1 Morphing_space2
Solid Modeling Combining simple 3D primitives for a compact and exact analytical representation of manufactured and architectural models Solid Modeling1
Deformers Enabling controlled non rigid displacements  Deformers
Bone-Based Animation Modeling and animation of generic articulated 3D objects Bone-Based_Animation1 Bone-Based_Animation2

Figure 4 AFX tools

At the Hong Kong meeting in January 2005 Mahnjin Han took over from Mikaël especially to continue the AFX activity.

At the Marrakesh meeting in January 2007 Marius Preda took over from Mahnjin. Besides continuing the AFX activity Marius proposed a new area of work called 3D Graphics Compression Model with the goal of specifying an architectural model able to accommodate third-party XML based description of scene graph and graphics primitives, possibly with binarisation tools and with MPEG-4 3D Graphics Compression tools specified in MPEG-4 part 2, 11 and 16.


Figure 5 – 3DG Compression Model

This layers of this architecture are (numbering from the low layer)

  1. Layer 3: any scene graph and graphics primitive representation formalism expressed in XML
  2. Layer 2: the binarised version of the XML data not encoded by the MPEG-4 elementary bitstreams encoders (e.g. scene graph elements), encapsulated in the “meta” atom of the MP4 file
  3. Layer 1: a range of compression tools from MPEG-4 part 11 and 16

141 thoughts on “MPEG-4 Inside – Graphics

  1. Pingback: A Roadmap – Riding the Media Bits

  2. Pingback:

  3. Pingback: online webcam chat

  4. Pingback: more detailed on this page

  5. Pingback: follow this address

  6. Pingback: click at this page

  7. Pingback: find out more here

  8. Pingback: find out more here

  9. Pingback: click to find out more

  10. Pingback: find out more

  11. Pingback: visit web page

  12. Pingback: read further

  13. Pingback: 100% Free Porn Meet Members 60779

  14. Pingback: FREE XXX Dates Users 56837

  15. Pingback: Best Sex Dates Members 11778

  16. Pingback: Best Porn Dating Members 15932

  17. Pingback: 100% Free PORNO Dating Members 27448

  18. Pingback: Best PORNO Dating List 97349

  19. Pingback: 100% Free XXX Dates Members 28999

  20. Pingback: 100% Free PORNO Meet Members 7924

  21. Pingback: 100% Free Sex Dates Users 9291

  22. Pingback: here 7ifYz

  23. Pingback: link to the page ZjGS

  24. Pingback: click to continue

  25. Pingback: link

  26. Pingback: click for details

  27. Pingback: view more

  28. Pingback: link to the page Vkjod

  29. Pingback:

  30. Pingback: check this out v16

  31. Pingback:

  32. Pingback: link to a continuation

  33. Pingback: click to read more

  34. Pingback: click to go

  35. Pingback: find out more P0Kr

  36. Pingback: see more

  37. Pingback: follow this address

  38. Pingback: continue

  39. Pingback: a source Zp2K

  40. Pingback: read more

  41. Pingback: more info lzYK

  42. Pingback: read article

  43. Pingback: a source SLf

  44. Pingback: find out more here Bj20u

  45. Pingback: click to continue

  46. Pingback: visit the page

  47. Pingback: click for details rVXWnS

  48. Pingback:

  49. Pingback: continued here

  50. Pingback: click to go qhs3

  51. Pingback: article source

  52. Pingback: go to the page

  53. Pingback:

  54. Pingback: based on these data

  55. Pingback: Porn Tube

  56. Pingback: more detailed on this page

  57. Pingback: based on these data

  58. Pingback: find out more

  59. Pingback: i provide a link

  60. Pingback: click to read more

  61. Pingback:

  62. Pingback: taken from here 66l16W

  63. Pingback:

  64. Pingback: click to find out more

  65. Pingback: click to find out more

  66. Pingback: read an article

  67. Pingback: a source

  68. Pingback: check this out

  69. Pingback: click to go

  70. Pingback:

  71. Pingback: go to the page JFO13

  72. Pingback:

  73. Pingback: continued here

  74. Pingback: link pkUc3

  75. Pingback: read article

  76. Pingback:

  77. Pingback: link x6fz1

  78. Pingback:

  79. Pingback: view more UOWMk

  80. Pingback: i provide a link

  81. Pingback: click yhwmo

  82. Pingback: here

  83. Pingback: i'll provide a link olP

  84. Pingback: go to the page ZaS

  85. Pingback: here

  86. Pingback:

  87. Pingback: click the following article JIj8

  88. Pingback:

  89. Pingback: Coub видео приколы лучшее 2018

  90. Pingback: 2018-2019

  91. Pingback: 2019

  92. Pingback:

  93. Pingback: a2019-2020

  94. Pingback: facebook

  95. Pingback: facebook1

  96. Pingback:

  97. Pingback:

  98. Pingback: tureckie_serialy_na_russkom_jazyke

  99. Pingback: Ñìîòðåòü âñå ñåðèè ïîäðÿä

  100. Pingback: Video

  101. Pingback: +1+

  102. Pingback: 1 2 3 4 5 6 7 8 9 10

  103. Pingback: movies

  104. Pingback: movies online

  105. Pingback: karan johar

  106. Pingback: Top Movies

  107. Pingback: Movies1

  108. Pingback: 11 10 2019

  109. Pingback:

  110. Pingback: Serial smotret

  111. Pingback: kinokrad

  112. Pingback: kinokrad 2020

  113. Pingback: ñïèñîê íîâûõ ôèëüìîâ

  114. Pingback: Watch TV Shows

  115. Pingback: casino

  116. Pingback: Kinokrad 2019 Kinokrad Hd

  117. Pingback: kinokrad-2019

  118. Pingback: serial 1

  119. Pingback: serial

  120. Pingback:

  121. Pingback: lostfilmtv net

  122. Pingback:

  123. Pingback: bofilm

  124. Pingback: Êîíñóëüòàöèÿ ïñèõîëîãà

  125. Pingback:

  126. Pingback:

  127. Pingback:

  128. Pingback:

  129. Pingback:

  130. Pingback:

  131. Pingback:

  132. Pingback:

  133. Pingback:

  134. Pingback:

  135. Pingback: rick and morty season 3

  136. Pingback: Watch+movies+2020

  137. Pingback: 1plus1tv

  138. Pingback: #1plus1

  139. Pingback: 2020

  140. Pingback: Watch Movies Online

  141. Pingback: Film 2020