 ICACT20260314 Slide.01
[Big Slide]
[YouTube] |
Chrome  Click!! |
 |
Hello everyone.
My name is Ferhat Ozgur Catak from the University of Stavanger.
In this presentation, I will introduce our work titled “Comparative Analysis of Attention Mechanisms for Automatic Modulation Classification in Radio Frequency Signals,” which is joint work with Murat Kuzlu and Umit Cali.
The main goal of this study is to analyze how different attention mechanisms influence both classification accuracy and computational efficiency in automatic modulation classification systems.
This work is motivated by the increasing need for efficient and real-time signal intelligence in next-generation wireless communication systems. |
| ICACT20260314 Slide.02
[Big Slide]
|
Chrome Click!! |
 |
First, I will briefly introduce the motivation and background of automatic modulation classification.
Then, I will explain the proposed CNN–Transformer hybrid architecture and the different attention mechanisms investigated in this study.
After that, I will describe the dataset and experimental setup.
Finally, I will present the performance results, computational efficiency analysis, and key insights, followed by conclusions and future research directions. |
| ICACT20260314 Slide.03
[Big Slide]
|
Chrome Click!! |
 |
Automatic Modulation Classification, or AMC, is a critical component in modern wireless communication systems, particularly for cognitive radio, spectrum monitoring, and interference management.
Traditional feature-based approaches rely on handcrafted statistical features and often suffer from poor generalization under varying channel conditions.
Deep learning methods, especially convolutional neural networks, have significantly improved AMC performance.
However, CNNs mainly capture local patterns and are limited in modeling long-range temporal dependencies present in radio signals.
Attention mechanisms, originally introduced in Transformer architectures, provide an effective way to model such temporal dependencies.
This motivates our study on how different attention designs affect AMC performance and efficiency. |
| ICACT20260314 Slide.04
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
This slide summarizes the main contributions of our work.
First, we propose a CNN–Transformer hybrid architecture tailored for I/Q-based radio signal classification.
Second, we perform a systematic comparison of three attention mechanisms: baseline multi-head attention, causal attention, and sparse attention.
Third, we evaluate not only classification accuracy but also computational efficiency, which is crucial for real-time applications.
Finally, we provide a detailed per-modulation analysis to understand how different modulation types benefit from different attention patterns. |
| ICACT20260314 Slide.05
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
This slide illustrates the overall system overview of the proposed approach.
The input consists of complex-valued in-phase and quadrature, or I/Q, samples, which are converted into a two-channel real-valued representation.
A convolutional neural network is used to extract local spatial features from the input signal.
These features are then processed by Transformer blocks with different attention mechanisms to capture temporal dependencies.
Finally, a global pooling layer and a classifier are used to predict the modulation class. |
| ICACT20260314 Slide.06
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
The CNN–Transformer hybrid architecture consists of three main components.
First, the CNN feature extractor uses one-dimensional convolutional layers with batch normalization and ReLU activation to extract local features from I/Q samples.
Second, the Transformer block applies multi-head attention to model temporal dependencies across the signal sequence.
Residual connections and layer normalization are used to stabilize training.
Finally, a lightweight multilayer perceptron produces the final modulation classification output. |
| ICACT20260314 Slide.07
[Big Slide]
|
Chrome Click!! |
 |
In this study, we investigate three different attention mechanisms.
The baseline multi-head attention allows full bidirectional information flow across all time steps, providing maximum modeling capacity but with quadratic computational complexity.
Causal attention restricts attention to past and current samples only, which makes it suitable for real-time and streaming scenarios.
Sparse attention limits attention to a local temporal window, significantly reducing computational cost while preserving local temporal structure.
These mechanisms represent different trade-offs between accuracy and efficiency. |
| ICACT20260314 Slide.08
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
We evaluate our proposed approach using the RML2016.10a dataset, which is a widely used benchmark for automatic modulation classification.
The dataset contains 220,000 signal samples covering 11 different modulation schemes, including both digital and analog modulations.
Each sample consists of 128 complex-valued I/Q points generated under various signal-to-noise ratio conditions.
In our experiments, we focus on the practical SNR range from minus six to eighteen decibels. |
| ICACT20260314 Slide.09
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
All models are implemented using the PyTorch deep learning framework.
We use the AdamW optimizer with cosine learning rate scheduling and early stopping based on validation accuracy.
Gaussian noise injection is applied as data augmentation to improve robustness.
Model performance is evaluated using accuracy, macro F1-score, inference time, and the number of trainable parameters. |
| ICACT20260314 Slide.10
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
This slide presents the overall performance comparison of the three attention mechanisms.
The baseline attention model achieves the highest classification accuracy of 85.05 percent.
Causal and sparse attention models show slightly lower accuracy but remain above 83 percent.
All models have the same number of parameters, ensuring a fair comparison focused on attention behavior rather than model size. |
| ICACT20260314 Slide.11
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
This figure compares the inference time of different attention mechanisms.
Causal attention achieves an 83 percent reduction in inference time compared to baseline attention.
Sparse attention achieves a 75 percent reduction.
These results demonstrate that attention design has a significant impact on computational efficiency and real-time feasibility. |
| ICACT20260314 Slide.12
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
This slide shows the F1-score for each modulation type.
We observe that simpler modulation schemes, such as CPFSK, GFSK, and PAM4, benefit from sparse attention, indicating that local temporal context is sufficient.
In contrast, more complex modulations, such as QAM16 and QAM64, require global context and perform best with baseline attention.
Analog modulations, particularly WBFM, remain challenging for all attention mechanisms. |
| ICACT20260314 Slide.13
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
This slide illustrates the distribution of F1-scores for different attention mechanisms.
Baseline attention shows a slightly higher median performance.
However, there is significant overlap between the distributions, indicating that causal and sparse attention provide competitive performance at much lower computational cost. |
| ICACT20260314 Slide.14
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
The confusion matrices reveal consistent error patterns across all models.
QAM16 and QAM64 are frequently confused due to their similar constellation structures.
AM-DSB and WBFM also show cross-confusion because of shared spectral characteristics.
Digital modulations such as BPSK and QPSK are classified robustly by all models. |
| ICACT20260314 Slide.15
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Based on these results, we can provide practical deployment guidelines.
Baseline attention should be used when maximum accuracy is required and computational resources are available.
Causal attention is well suited for real-time and low-latency systems.
Sparse attention is ideal for edge or embedded deployments where computational efficiency is critical. |
| ICACT20260314 Slide.16
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
There are several directions for future work.
First, a more detailed analysis across different SNR ranges could provide deeper robustness insights.
Second, evaluating the model under realistic channel conditions such as fading and multipath would improve practical relevance.
Finally, adaptive attention mechanisms that dynamically adjust based on signal characteristics could further optimize performance. |
| ICACT20260314 Slide.17
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
In conclusion, this study presented a comparative analysis of attention mechanisms for automatic modulation classification.
The proposed CNN–Transformer hybrid architecture achieves strong performance on the RML2016.10a dataset.
While baseline attention provides the highest accuracy, causal and sparse attention offer substantial reductions in inference time with only minor performance loss.
These findings provide useful guidance for designing efficient AMC systems in next-generation wireless networks.
Thank you for your attention. |
