|
Before dive into our proposed fusion method, the first step is to convert multi-view images and radar point cloud data into a shared Bird¡¯s Eye View (BEV) space from the baseline framework. Multi-view RGB images are first passed through an image backbone network to extract features. These features are then processed through a view transform module, which integrates information from multiple camera views and projects them onto a top-down BEV map. Finally, the camera BEV features are extracted from Image BEV encoder. Simultaneously, radar point cloud data is processed using a dual-stream radar backbone, which includes a point-based encoder for capturing local features and a transformer-based encoder for extracting global contextual information. The radar features are then refined using the RCS-aware scattering to handle radar¡¯s sparsity and noise. Finally, the radar BEV features are obtained from radar BEV encoder.
|