This talk shall focus on contemporary visual computing research trends with critical implications for 6G semantic communications. Semantic communication was initially proposed by Weaver and Shannon 70+ years ago in the early 1950s in which they outlined the classical definition of three levels of communications: the technical problem, the semantic problem, and the effectiveness problem. Until 5G, most researchers and practitioners have been working on the first technical problem. For 6G, semantic communication becomes necessary to handle the overwhelming volume of visual data among all IP traffic. We firmly believe that a paradigm-shifting framework needs to be designed to transport the volumetric visual data under the 6G mobile communication architecture. We show that recent technical advances in contemporary visual computing bear great potential for 6G semantic communication. Among the volumetric visual data, a significant portion of them has been acquired for machine intelligence purposes. Therefore, structured extraction and representation of the semantics from these visual data are desired to facilitate the 6G semantic communication. For contemporary visual computing, the well-structured scene graph generation (SGG) approaches have been demonstrated capable of representing compactly the logical relationship among the subjects and objects detected from the visual data. We shall show that the unique capability of structured SGG can be applied to 6G semantic communication towards future advances in integrating visual computing with 6G.

In this talk, I will present an overview of the current state of generated and manipulated media, such as Deep Fakes, and describe how these methods work, where they are being used and how to detect them. I will also describe the history of manipulated media content and how we got where we are today. An outline of my talk is:

• Historical Overview of Manipulated Media
• The disappearing Russians
• Hollywood and CGI
• Cheapfakes and Deepfakes
• Seeing is not believing
• An overview of the technology
• The threat of manipulated media
• Where is this all going in 5, 10, 20 years?
• Is help coming?

1. AI in Standards: We will review developments in video codec standards like MPEG ECM and AOM’s AVM, emphasizing machine learning advancements for BD-rate gains and challenges in complexity and hardware implementation.
2. End-to-End Neural Network Compression: We will discuss cutting-edge research on new neural network architectures for video compression, focusing on industry responses and anticipated adoption.
3. GenAI Video Compression: We will explore the challenges and opportunities of GenAI videos, like digital humans, which feature characteristics like very low noise, sharp edges, bright colors, and dynamic motion, necessitating different encoder settings and comparing them with other computer-generated content such as screen content, animations, and gaming.
4. Video Quality Assessment: We will highlight collaborations with industry leaders, exploring methods for evaluating grainy videos and trends like high-density, low-power VMAF calculations on Nvidia GPUs.
5. Playback Enhancement: We will discuss player-side synthesis work, including AV1’s Film Grain synthesis, leveraging decoder-side computational power in modern mobile devices to enhance playback and reduce power consumption.
6. Ultra-Low Latency Live: We will share insights on low-delay live streaming using AV1 encoding, its optimization for low bandwidth and low complexity, and potential in real-time 3D applications and immersive video sharing. We will demonstrate that AV1 encoding can be optimized to remain competitive in compression efficiency while achieving very low complexity comparable to OpenH264.
7. Joint CPU+GPU Optimization: In the industry, for VOD, CPU encoders are widely used for encoding the most-viewed videos, while GPUs are utilized for long-tail content. For live streaming, CPUs are preferred for live event streaming, whereas GPUs are deployed for 24/7 live channel operations. We will explore the optimization of existing standards using a combination of GPU and CPU, highlighting how GPUs enhance machine learning processing for more effective content-adaptive encoding while maintaining high throughput.