 |
Visible images are colorful and have more texture details. Thermal images lose the information. Thermal images can be imaged clearly in dark scenes, because the image quality is independent of the illumination condition. As visible and thermal images have their own advantages and disadvantages in different scenarios, researchers have raised interest in multispectral detection.
In many application scenarios, such as autonomous driving, high accuracy and low latency are required for multispectral detection. At the same time, it is desired to reduce cost, chip area, and power consumption as much as possible. However, to ensure accuracy, multispectral detection commonly uses a two-stream architecture. Compared with the traditional CNN model, it requires more computational resources, more on-chip memory, and higher off-chip bandwidth. To address this problem, using dynamic neural networks is a feasible solution. Compared to static neural networks, dynamic networks selectively activate model components based on the input sample during inference. Studies have shown that dynamic neural networks have greater computational efficiency and representation capability.
It is challenging to process dynamic neural networks efficiently, due to the significant irregularities introduced by the redundancy elimination at runtime. General hardware accelerators are difficult to benefit from dynamic architectures, because they can not efficiently process optimized models, and eliminate redundant computing and memory access.
|
IEEE/ICACT20230038 Slide.03
[Big Slide]
[YouTube]