 ICACT20260085 Slide.01
[Big Slide]
[YouTube] |
Chrome  Click!! |
 |
Hello everyone, my name is Junior Okangondo Loshima.
It is a great pleasure for me to present today our research work entitled “Acoustic-visual fusion for robust day/night detection of vectors.”
This work is situated in the field of intelligent entomological surveillance and proposes a multimodal fusion approach combining acoustic and visual analysis to robustly detect mosquitoes under both daytime and nighttime conditions, while meeting real-time processing requirements and Edge deployment constraints. |
| ICACT20260085 Slide.02
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Mosquito-borne diseases remain a major global public health challenge, particularly in tropical and subtropical regions where environmental conditions favor vector proliferation. Diseases such as malaria, dengue, and Zika continue to impose a significant health and socio-economic burden worldwide. In this context, reliable and continuous surveillance systems are essential to anticipate outbreaks and support effective vector control strategies. However, conventional monitoring approaches often rely on manual inspections or periodic sampling, which are costly, labor-intensive, and difficult to scale. These limitations highlight the need for automated, real-time surveillance solutions capable of operating under diverse environmental conditions, thereby motivating the development of intelligent systems based on advanced sensing and data-driven techniques. |
| ICACT20260085 Slide.03
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Current mosquito detection systems rely primarily on single modalities such as acoustics or vision, each suffering from significant limitations. Acoustic-based approaches are highly sensitive to environmental noise, while visual systems experience severe performance degradation under low-light or nighttime conditions. This reliance on unimodal sensing restricts the robustness and reliability of existing solutions in real-world environments. |
| ICACT20260085 Slide.04
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Our research aims to investigate how acoustic and visual information can be effectively combined to enable robust mosquito detection under both daytime and nighttime conditions. It also examines how such a system can maintain high accuracy while satisfying real-time processing requirements and the computational constraints of edge-based platforms. |
| ICACT20260085 Slide.05
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Our work proposes a multimodal framework that integrates acoustic spectrogram analysis with RGB–infrared visual processing to improve mosquito detection robustness. By exploiting the complementary strengths of both modalities, the proposed system achieves significant accuracy improvements over unimodal approaches while remaining compatible with real-time processing and Edge AI deployment constraints. |
| ICACT20260085 Slide.06
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Acoustic analysis exploits wing-beat frequencies, which constitute highly discriminative signatures for identifying mosquito species as well as distinguishing sex. Unlike purely visual approaches, this modality remains effective even under visual occlusion or complete darkness, making it particularly suitable for low-light and nighttime environments. As a result, acoustic sensing provides essential robustness for vector detection in conditions where vision-based systems are prone to failure. |
| ICACT20260085 Slide.07
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
RGB cameras provide rich spatial and texture information under daytime conditions, enabling accurate visual characterization of mosquitoes and their flight patterns. However, their performance significantly degrades under low-light or nighttime conditions. The integration of infrared imaging enhances robustness by capturing thermal and structural cues that remain stable in the absence of visible light. Consequently, the RGB–IR combination effectively mitigates illumination-related performance degradation and ensures more reliable detection across varying lighting conditions. |
| ICACT20260085 Slide.08
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
In our work, we leverage multimodal fusion to exploit the complementary strengths of acoustic and visual cues in order to enhance detection robustness. By jointly analyzing sound and visual information, our system significantly reduces false negatives, particularly in challenging environments characterized by high noise levels or low-light conditions. This multimodal integration allows us to achieve more reliable mosquito detection across diverse and variable real-world scenarios. |
| ICACT20260085 Slide.09
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
In our literature review, we observe that existing mosquito detection approaches mainly rely on acoustic-only, vision-only, or audiovisual methods, often exhibiting limited robustness in real-world conditions. Many studies employ complex fusion strategies or computationally intensive models, such as Transformers or multi-sensor systems, leading to high latency and poor suitability for edge deployment, while others focus exclusively on visual modalities and neglect the robustness provided by acoustic information under low-light or noisy environments. In contrast, our work introduces a lightweight multimodal architecture that combines an acoustic CNN with an RGB–IR ResNet-based visual branch through intermediate fusion, explicitly addressing synchronization, computational efficiency, and robustness requirements for Edge AI deployment in real-world conditions. |
| ICACT20260085 Slide.10
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
From our analysis of existing work, we identify that most current mosquito detection systems suffer from limited robustness under real-world conditions characterized by environmental noise and lighting variability. In addition, edge compatibility and low-latency requirements are rarely addressed simultaneously, which restricts the practical deployment of these systems in real-time and resource-constrained environments. |
| ICACT20260085 Slide.11
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
In our methodology, we collected 2,400 synchronized audio–video recordings covering four mosquito species under both daytime and nighttime conditions. The experiments were conducted under controlled variations of illumination and ambient noise in order to evaluate the robustness of the proposed system across diverse environmental scenarios. |
| ICACT20260085 Slide.12
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
In our approach, acoustic signals are transformed into Gabor spectrograms in order to capture discriminative time–frequency patterns related to mosquito wing-beat activity. Visual features are extracted independently from RGB and infrared frames, allowing each modality to preserve its specific characteristics before being integrated through multimodal fusion. |
| ICACT20260085 Slide.13
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Our proposed architecture is composed of two parallel processing branches designed to handle acoustic and visual information independently. The acoustic branch relies on a lightweight CNN applied to Gabor spectrograms extracted from audio signals, while the visual branch is based on a modified and lightweight ResNet-18 that processes synchronized RGB and infrared frames. An intermediate feature fusion layer concatenates the learned representations from both modalities, enabling joint multimodal learning. The fused features are then passed through fully connected layers to perform final mosquito detection and classification tasks, including presence detection, species identification, and sex classification, while maintaining low computational complexity suitable for Edge AI deployment. |
| ICACT20260085 Slide.14
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
we adopt an intermediate fusion strategy that concatenates acoustic and visual feature vectors learned independently by each branch. This fused representation enables effective interaction between modalities and is subsequently refined through fully connected layers to perform accurate mosquito detection and classification while preserving computational efficiency. |
| ICACT20260085 Slide.15
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
To enable efficient Edge AI deployment, we apply model pruning and quantization techniques to reduce both memory footprint and inference latency. These optimizations allow the proposed system to satisfy real-time processing requirements while maintaining high detection performance on resource-constrained platforms. |
| ICACT20260085 Slide.16
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Our training results demonstrate that the proposed acoustic–visual multimodal fusion significantly outperforms unimodal approaches. While vision-only and acoustic-only models achieve accuracies of 78.5% and 80.3% respectively, the multimodal fusion model reaches 91.2% accuracy with an F1-score of 91.0%. This corresponds to an improvement of more than 12% compared to unimodal systems, highlighting the effectiveness of combining complementary acoustic and visual information during training. |
| ICACT20260085 Slide.17
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Our real-world evaluation shows that the proposed multimodal fusion model maintains strong performance under challenging environmental conditions. While vision-only and acoustic-only models achieve accuracies of 78.0% and 79.6% respectively, the multimodal approach reaches 88.7% accuracy with an F1-score of 88.3% in low-light and high-noise environments. Moreover, the average inference latency remains compatible with real-time Edge AI operation, confirming the practicality of the proposed system in real deployment scenarios. |
| ICACT20260085 Slide.18
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Our results demonstrate the strong robustness of the proposed multimodal fusion framework and its effective generalization from controlled experimental settings to real-world conditions. Despite significant variations in lighting and ambient noise, the observed performance degradation remains limited, highlighting the complementarity of acoustic and visual modalities. These findings confirm that the proposed architecture is well suited for reliable mosquito detection in realistic deployment scenarios. |
| ICACT20260085 Slide.19
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
In future work, we plan to integrate thermal imaging and 3D point cloud data in order to further enhance robustness under challenging environmental conditions. We also intend to explore federated learning strategies and advanced model compression techniques to improve scalability, privacy preservation, and real-time deployment on resource-constrained Edge AI devices. |
| ICACT20260085 Slide.20
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Our proposed framework enables real-time entomological surveillance in rural and resource-constrained environments. By combining acoustic and visual modalities on Edge AI devices, our approach contributes to early warning systems for vector-borne disease prevention and supports timely public health decision-making. |
| ICACT20260085 Slide.21
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
In this work, we demonstrate the effectiveness of acoustic–visual fusion for robust mosquito detection under both day and night conditions. The proposed system achieves high accuracy and strong robustness while remaining compatible with real-time Edge AI constraints, making it suitable for practical deployment in real-world entomological surveillance scenarios. |
| ICACT20260085 Slide.22
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Thank you for your attention. We would be happy to take any questions or suggestions. |
