 ICACT20250139 Slide.01
[Big Slide]
|
Chrome  Click!! |
 |
Hello, this is XIN LI from Hanyang University. I would like to give a presentation on the paper titled "Efficient Atlas Coding Strategy using Cropping for Object-based MPEG Immersive Video“. |
| ICACT20250139 Slide.02
[Big Slide]
|
Chrome Click!! |
 |
With the advancement of virtual reality (VR), augmented reality (AR), and free-viewpoint video, there is an increasing demand for efficient immersive video coding. MIV addresses these needs by providing a standardized framework for encoding and rendering immersive content.
|
| ICACT20250139 Slide.03
[Big Slide]
|
Chrome Click!! |
 |
MPEG Immersive Video, or MIV, is a standard developed by the Moving Picture Experts Group (MPEG) to efficiently represent immersive 3D scenes using multiple video views.
Unlike traditional 2D video coding, MIV enables users to navigate within a scene and change viewpoints, enhancing realism and interactivity.
There is a reference software called TMIV that supports MIV.
TMIV takes multiple source views as input, including geometry, attributes, object component video, and various parameters related to the views.
TMIV labels the input source videos as basic views and additional views.
TMIV prunes pixels in the additional views while considering redundancy, based on the basic views. The unpruned pixels are then used to create a mask, which is aggregated and formed into patches.
Then, the patches are packed to generate an atlas video. Considering video coding efficiency, the atlas video is then quantized and scaled.
The generated atlas video is encoded using HEVC or VVC to produce a bitstream, which is then multiplexed together with other auxiliary bitstreams.
|
| ICACT20250139 Slide.04
[Big Slide]
|
Chrome Click!! |
 |
This shows the atlas video generated by the TMIV encoder. Encoding dozens of multiple-view videos as input to the encoder produces two atlas videos for texture and geometry respectively.
|
| ICACT20250139 Slide.05
[Big Slide]
|
Chrome Click!! |
 |
This is the result of decoding and rendering the atlas video shown on the previous page. It can be decoded and rendered from various viewpoints, and what you are seeing now corresponds to one of the multiple views.
|
| ICACT20250139 Slide.06
[Big Slide]
|
Chrome Click!! |
 |
Now, I would like to introduce Entity-based MIV.
|
| ICACT20250139 Slide.07
[Big Slide]
|
Chrome Click!! |
 |
This is the overall structure of the Entity-based MIV encoder.
It is almost identical to the previously introduced MIV structure.
The key difference is that while the previously introduced MIV could encode attribute and geometry videos along with parameters for views as input.
But the Entity-based MIV encoder must also take object component video as input.
|
| ICACT20250139 Slide.08
[Big Slide]
|
Chrome Click!! |
 |
The image you are looking at visualizes the object component video by coloring each object differently.
Each object has a unique entity number.
The Entity-based MIV encoder separates only the objects within the user-specified range of entity numbers and performs MIV encoding on them.
|
| ICACT20250139 Slide.09
[Big Slide]
|
Chrome Click!! |
 |
The left side of the dotted line shows the atlas video generated by the Entity-based Encoder, while the right side presents the decoded and rendered result using the atlas video on the left.
This demonstrates that users can specify the desired entity and perform encoding, decoding, and rendering exclusively for the selected entity.
|
| ICACT20250139 Slide.10
[Big Slide]
|
Chrome Click!! |
 |
Now, I would like to explain the proposed method.
|
| ICACT20250139 Slide.11
[Big Slide]
|
Chrome Click!! |
 |
During MIV encoding, the generated patches are efficiently packed into the atlas using the MaxRect algorithm.
This method optimizes packing by rotating or flipping patches, ensuring that they occupy the minimum possible area.
Each patch includes the metadata such as the top-left coordinates, width value and height value.
Using this information, the bottom-right coordinates of each patch can be calculated.
The patches are packed efficiently from the top-left corner of the atlas.
Once the patch packing is complete, the largest bottom-right x and y values among the packed patches can be determined.
These x and y values define the cropping range.
|
| ICACT20250139 Slide.12
[Big Slide]
|
Chrome Click!! |
 |
This shows the texture atlas video generated by the TMIV encoder and the atlas video using proposed method.
The encoder generated two atlases, c00 and c01. The anchor refers to the TMIV Encoder, while the proposed refers to the TMIV Encoder with the proposed method applied.
From the left, the first and second images show the atlas generated for the c00 atlas using the anchor and proposed methods, respectively.
In the second image, since patches are packed across the entire atlas area, there is no available cropping region.
From the left, the third and fourth images show the atlas generated for the c01 atlas using the anchor and proposed methods, respectively.
In the third image, the green box represents the cropping region outside the red box, which encloses the packed patches within the atlas.
Therefore, as shown in the fourth image from the left, the proposed method enables cropping of the area marked by the green box.
|
| ICACT20250139 Slide.13
[Big Slide]
|
Chrome Click!! |
 |
This shows the geometry atlas video generated by the TMIV encoder and the atlas video using proposed method.
The encoder generated two atlases, c00 and c01. The anchor refers to the TMIV Encoder, while the proposed refers to the TMIV Encoder with the proposed method applied.
The geometry atlas also has a cropping region corresponding to the green box, excluding the red box that encloses the packed patches in the third image.
Therefore, as shown in the fourth image from the left, the proposed method allows cropping of the area marked by the green box.
|
| ICACT20250139 Slide.14
[Big Slide]
|
Chrome Click!! |
 |
From this point, I will present the experimental results of the Entity-based TMIV with the proposed method applied.
|
| ICACT20250139 Slide.15
[Big Slide]
|
Chrome Click!! |
 |
This shows the atlas video generated by encoding specific objects, specifically Entity 4, 7, 9, and 21.
The atlases above the dotted line were generated by the Entity-based TMIV Encoder, while the atlases below were produced using the Entity-based TMIV Encoder with the proposed method applied.
As observed, a significant portion of the area was cropped in the proposed method.
|
| ICACT20250139 Slide.16
[Big Slide]
|
Chrome Click!! |
 |
This summarizes the quality evaluation results after decoding and rendering the atlases from the previous page.
The rendered images on the left show the decoded and rendered results for Entities 4, 7, 9, and 21.
The chart on the right compares the anchor (red) and the proposed method (green) in terms of WS-PSNR and IV-PSNR. From top to bottom, the results for Entities 4, 7, 9, and 21 are listed.
WS-PSNR (Weighted Spherical PSNR) is a quality metric designed for 360-degree and VR video, accounting for spherical distortion to provide an accurate perceptual quality assessment.
And IV-PSNR (Immersive Video PSNR) is a quality metric for MIV that evaluates reconstructed video accuracy in 3D space by considering depth information and geometric consistency.
As seen in the results, the proposed method demonstrates better coding efficiency compared to the anchor.
|
| ICACT20250139 Slide.17
[Big Slide]
|
Chrome Click!! |
 |
What you are looking at summarizes the results of encoding, decoding, and rendering objects using the Museum dataset, with quality measured in terms of Y-PSNR and IV-PSNR.
Y-PSNR (Luminance PSNR) is a quality metric that evaluates video compression efficiency by measuring signal fidelity based on the Y (luminance) channel in the YUV color space.
Additionally, Y-PSNR and IV-PSNR measured at high bitrates and Y-PSNR and IV-PSNR measured at low bitrates have been separately organized.
The green areas indicate good coding efficiency, while the red areas indicate poor coding efficiency.
Based on the results we have seen, it can be confirmed that the coding efficiency of Entity-based MIV improves when the proposed method is applied.
|
| ICACT20250139 Slide.18
[Big Slide]
|
Chrome Click!! |
 |
Here is the conclusion.
|
| ICACT20250139 Slide.19
[Big Slide]
|
Chrome Click!! |
 |
This study presents an efficient atlas coding strategy that uses cropping algorithms to improve the performance of object-based MIV.
In the proposed method, the cropping algorithm leverages the patch information generated during the packing of patches in the MIV encoding.
Using the position of the upper-left point, along with width and height information, the area of the atlas that requires cropping can be calculated after the patches are packed, excluding the area of the packed patches.
The experimental results show that, overall, the larger the cropped area of the atlas, the greater the end-to-end coding gain.
This is because a larger cropped area of the atlas results in a lower bitrate generated during encoding.
The proposed method can provide an efficient coding strategy for scenarios in which object-by-object coding is necessary.
It is particularly valuable in situations that require object-by-object processing within a scene in application fields such as VR and AR in the metaverse.
|
| ICACT20250139 Slide.20
[Big Slide]
|
Chrome Click!! |
 |
Thank you.
|
