Embedded Files
DSPOptimal design is an art form in which the designer selects from a myriad of alternatives to bring the 'optimum' choice for user. In many complex systems the notion of 'optimum' is difficult to define. Indeed, the users themselves wil not agree. So, the 'best' system is simply the one in which the designer evaluates the options and takes the responsibilities!
NoiseGenerator.mp4IPImage processing in the context of imaging science. The rate of discovery in the natural sciences has been accelerated by powerful computer-based imaging methods, which enable scientists to map certain object properties into an 'image space'. These methods also support the goals of medicine, education, art, commerce, and the media. Every digital imaging method involves a sequence of interdependent steps: namely, image-data acquisition, reconstruction and processing, recording and distribution, display/visualization, observation and analysis, and a criterion for image evaluation. When viewed within the context of imaging science, 'optimal' image processing must take account of how the image-data are acquired and the intervening steps leading to how the images are to be evaluated, based on a well-defined goal and criterion.
Multimedia science & art: 1. experimental (extended) media 2. multi (cross/trans) media(matics) 3. new (digital) media 4. post (meta) media
Immersive multimediaThe quest for a truly immersive media experience is almost the history of media technology. We have sought to create an experience just like being there from the earliest days of recorded sound, right up to the present day. Curiously, the desire for richer and more immersive productions has spawned technologies that help us in other areas.FROM CLICKABLE PAGES TO WALKABLE SPACESImmersive media = G2G formats @ E2E eco-systems (ARcore, ARkit, MRms)3D spatial acquisition and immergere reconstruction/reproduction = hyper-realistic media + personalized virtual walk + involving interactivityImaging modalities = omnidirectional / depth-enhanced / point cloud / light field / holographicAR/VR = Mix video/graphical formats + 3D registration + interactivityARframe@FaceTrackingSession = tracking (position, orientation, distances) + scene understanding (plane, hit, ligths) + rendering (viewport)Head-motion Parallax is displacement or difference in the apparent position of an object viewed from different viewing positions or viewing orientations.Motion-to-High-quality Latency is the time it takes between head motion and displaying content at high quality in a head-mounted device.Interactivity = precise motion + minimal latency + tracking + natural UIEcosystems = 3Dgraphics/Animation, VisualFX, CinematicVR, GamingArchitecture = The difference is the size of the viewing zone/volume (sitting/standing vs. some steps). The amount of data that is needed scales proportionately to the actual size of the display.
Immersion impossible?How immersive is enough? to trigger emotions -> change user's focus and perception (attention is driven by contrast and patterns)VR = immersion+presence (3D tracking&mapping ->real-time graphics engine)AR = overlay of content on the real world, but that content is NOT anchored (just aligned) to, or part of it.MR = overlay of synthetic content on the real world that is anchored to and interacts with the real world.
Immersion impossible?How immersive is enough? to trigger emotions -> change user's focus and perception (attention is driven by contrast and patterns)VR = immersion+presence (3D tracking&mapping ->real-time graphics engine)AR = overlay of content on the real world, but that content is NOT anchored (just aligned) to, or part of it.MR = overlay of synthetic content on the real world that is anchored to and interacts with the real world.
Immersive communicationCommunication, like the atmosphere itself, is ubiquitous and essential for humans, and, with the development of new technologies, such as ubiquitous network, big data, 3D printing, virtual reality, and artificial intelligence, it has become almost impossible to live without it. In addition, means of communication have changed immeasurably. New research paradigm incorporates new features and factors of communication and a new theoretical framework named “immersive communication.” Pointing out that communication today has moved beyond the bidirectional mass communication of “the second media age” to ubiquitous, immersive communication in “the third media age,” the author discusses the definition, characteristics, information structure, and models of immersive communication.We feel that communication is becoming more and more ubiquitous and omnipresent; we are more and more closely integrated with the media, and recognize that a new mode of communication is greatly changing our way of life. It seems that we are returning to the original state of human existence, going in a circle from the end to the starting point. We have gone from direct face-to-face information and emotional communication between individuals, to using media as a communication interface, to the invisible and even disappearing interface, and now we return to the original starting point of “direct” communication.
Digital videoThe emergence of digital video in the 1990s was made possible by compression techniques developed in the preceding decades. Even though the earliest videophones and consumer digital video formats were limited to very low-resolution images of 352 × 288 pixels or smaller, the amount of information required to store and transmit moving video was too great for the available transmission channels and storage media. Video coding or compression was an integral part of these early digital applications, and it has remained central to each further development in video technology since 1990.By the early 1990s, many of the key concepts required for efficient video compression had been developed. From the 1970s onwards, industry experts recognised that video compression and digital video had the potential to revolutionise the television industry. Efficient compression would make it possible to transmit many more digital channels in the bandwidth occupied by the older analogue TV channels.The development of industry standards for compression or coding and decompression or decoding set the scene for mass-market digital video in the 1990s, alongside improvements in integrated circuit technology that made it possible to encode and decode video in real time on consumer devices. Video DVDs and digital TV broadcasting rapidly replaced their analogue predecessors in the late 1990s and early 2000s, bringing digital video and video coding into many of the world’s households. The years 2000–2010 saw the increasing use of video over the internet and on mobile devices and a gradual move from Standard Definition, around 720 × 480 pixels, to High Definition, 1920 × 1080 pixels. As of 2024, the quantity and resolution of transmitted video continues to increase, with 4K or Ultra HD increasingly the norm for many video applications.Video coding or video compression bridges the gap between displayable or uncompressed video and practical methods of storing or transmitting data. Digital video is made up of a series of frames or pictures, each of which contains hundreds of thousands or millions of picture elements or pixels. Storing or transmitting video in its original, uncompressed form can require an impractically large amount of storage capacity or bandwidth. A video encoder compresses video into a smaller and more compact form known as coded video, suitable for storage or transmission. A video decoder reverses this process, extracting displayable video from coded video. The complementary pair of an encoder and a decoder is a video codec.Video coding is a key technology for storing and transmitting video. The development of video coding standards since the early 1990s has enabled an increasingly diverse range of applications for digital video. These range from digital TV and digital versatile disc (DVD) video, to today’s use of video for day-to-day communication and leisure activities, using a wide range of consumer devices.An important consideration for practical video codecs is the trade-off between compression, video quality and computation. A video coding standard, such as H.264 or H.265, provides a set of available coding tools, which are specified in the profiles that are defined in each standard. The actual performance of a video codec, even when it conforms to a standard, depends very much on how it is implemented in software or hardware, as well as on the decisions taken by the video encoder. Different applications can and often do require different encoding and/or implementation strategies.A video encoder or decoder can be implemented in software, in hardware or in a mix of the two, by using dedicated hardware for certain computationally intensive functions and programmable software for other functions.
Foundations of video coding [1950–1990]
Video coding standards and formats [1990–2021]
MPEG-I Coded Representation of Immersive Media [2016-2023] ISO/IEC 23090
Foundations of video coding [1950–1990]
- Entropy Coding [1952–1987]
- Frame Differencing [1956–1972]
- Discrete Cosine Transform [1974–1984]
- Development of the Video Codec Model [1977–1985]
- Early Advances in Motion-Compensated Prediction [1989–1995]
Video coding standards and formats [1990–2021]
- H.120, H.261 and MPEG-1: Early video coding standards
- MPEG-2: Digital TV and DVD-Video
- H.263 and MPEG-4 Part 2: Internet and Mobile video
- H.264: Multi-purpose video coding
- H.265/HEVC
- H.266/VVC
- other standards and formats (VC-1, VP8, VP9 and AV1)
MPEG-I Coded Representation of Immersive Media [2016-2023] ISO/IEC 23090
- Part 1 Architectures for immersive media [UnderDevelopment]
- Part 2 Omnidirectional media format
- Part 3 Versatile Video Coding (VVC)
- Part 4 MPEG-I immersive audio [UnderDevelopment]
- Part 5 Visual volumetric video-based coding (V3C) & Video-based point cloud compression (V-PCC)
- Part 6 Immersive media metrics
- Part 7 Immersive media metadata
- Part 8 Network based media processing
- Part 9 Geometry-based point cloud compression
- Part 10 Carriage of visual volumetric video-based coding data
- Part 11 Network-based media processing implementation guidelines
- Part 12 MPEG immersive video
- Part 13 Video decoding interface for immersive media
- Part 14 Scene description
- Part 15 Conformance testing for versatile video coding
- Part 16 Reference software for versatile video coding
- Part 17 Reference software and conformance for omnidirectional media format (OMAF)
- Part 18 Carriage of geometry-based point cloud compression data
- Part 19 Reference Software for V-PCC
- Part 20 Conformance for V-PCC
- Part 21 Reference Software for G-PCC
- Part 22 Conformance for G-PCC
- Part 23 Conformance and reference software for MPEG immersive video
- Part 24 Conformance and reference software for scene description
- Part 25 Conformance and reference software for carriage of visual volumetric video-based coding data
- Part 26 Conformance and reference software for carriage of geometry-based point cloud compression data
- Part 27 Media, renderers, and game engines for render-based systems and applications
- Part 28 Efficient 3D graphics media representation for render-based systems and applications
- Part 29 Video-based dynamic mesh coding (V-DMC)
- Part 30 Low latency, low complexity LiDAR coding
- Part 31 Haptics Coding (haptic feedback, with various modalities, is more and more used in multimedia application)
- Part 32 Carriage of haptics data
- Part 33 Conformance and reference software for haptics coding
Visual data compression in the AI era
Compression with AI: enhance the coding performance of compression algorithms with AI techniques. I will discuss the use of generative AI models, particularly variational autoencoders in lossy image compression, and their resemblance to traditional coding concepts such as transform coding, and wavelet transform.Compression for AI: design compression systems for AI-based recognition and processing (as opposed to human viewing only). I will showcase potential real-world applications of coding for machines and present several recent methods targeting mobile-cloud systems.Compression of AI: efficient compression of AI models to reduce computational costs. I will present pruning and quantization methods to address the challenges of compressing neural network-based codecs.
Page updated
Google Sites
Report abuse