 ICACT20250023 Slide.01
[Big Slide]
[YouTube] |
Chrome  Click!! |
 |
Hello everyone,
My name is Junior Okangondo Loshima, and today I will be presenting our research on "Comparative Evaluation of the Performance and Effectiveness of Machine Learning and Deep Learning Algorithms for Anomaly Detection in Android Log Analysis. |
| ICACT20250023 Slide.02
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
The massive adoption of Android devices has led to an increase in cyberattacks, making system log analysis essential for detecting malicious activities. However, these logs are large and unstructured, making manual processing complex.
Objective: This study explores AI-based approaches to automate log analysis and effectively detect anomalies.
|
| ICACT20250023 Slide.03
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Machine learning algorithms are selected for their ability to handle large datasets and identify anomalies by learning patterns from normal data.
Isolation Forest isolates anomalies by partitioning data.
DBSCAN detects anomalies by identifying isolated points in low-density regions.
One-Class SVM trains a model only on normal data, making it effective in detecting deviations.
Conclusion: These algorithms enable fast and effective anomaly detection in Android logs.
|
| ICACT20250023 Slide.04
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Deep learning models are used for their ability to extract complex features and capture temporal dependencies in Android logs.
Autoencoders reduce dimensionality and reconstruct data to identify anomalies.
CNNs extract local features from log sequences to detect unusual patterns.
RNNs (LSTM) capture temporal dependencies, improving anomaly detection in event sequences.
|
| ICACT20250023 Slide.05
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Deep learning models are well-suited for unstructured and large-scale Android logs but require parameter optimization. |
| ICACT20250023 Slide.06
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Existing research shows that machine learning and deep learning approaches complement each other in Android log anomaly detection.
Machine learning models are often faster and require less data.
Deep learning models can exploit complex structures but require extensive training and large datasets.
Despite advancements, challenges remain in feature selection, class balancing, and interpretability of results.
|
| ICACT20250023 Slide.07
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
We conducted a comparative evaluation of machine learning and deep learning models for Android log anomaly detection.
We tested three machine learning models (Isolation Forest, DBSCAN, One-Class SVM).
We compared their performance with three deep learning models (Autoencoder, CNN, RNN-LSTM). |
| ICACT20250023 Slide.08
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
We used key metrics: precision, recall, F1-score, and AUC-ROC.
Our results show that DBSCAN is the most effective algorithm, and we recommend optimizing deep learning models and exploring hybrid approaches
|
| ICACT20250023 Slide.09
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
We collected Android logs from multiple sources to build a representative dataset of system events:
Logcat: The primary Android tool for recording system and application messages.
System and event logs: Information on system errors, performance, and security.
LogHub: A database of logs from instrumented Android devices.
These diverse sources ensure rich and varied data, essential for training models.
|
| ICACT20250023 Slide.10
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Before analyzing logs, cleaning and normalization are required to ensure data consistency:
Removal of redundant or irrelevant data (low-importance logs).
Normalization of date and time formats to align events on a coherent timeline.
Encoding log levels (INFO, DEBUG, ERROR) into numerical values.
Feature extraction such as event frequency and message length.
Proper preprocessing improves data quality and algorithm performance.
|
| ICACT20250023 Slide.11
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
After analysis, we observed that the model detected one hundred ninety-seven anomalies, demonstrating a high anomaly detection capability. However, it suffers from a high false positive rate, leading to low precision despite good anomaly coverage. |
| ICACT20250023 Slide.12
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
After analysis, we observed that DBSCAN detected 3 anomalies, showing excellent generalization ability with high precision. However, its detection capacity remains limited, making it more suitable for scenarios where only normal data is available.
|
| ICACT20250023 Slide.13
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
After analysis, we observed that One-Class SVM detected ninety-four anomalies, demonstrating good generalization ability. However, it suffers from low precision and is more effective in scenarios where only normal data is available for training. |
| ICACT20250023 Slide.14
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
After analysis, we observed that Autoencoder detected eighty-six anomalies, showing good sensitivity to log variations and the ability to identify anomalies in reconstructed data. However, its performance is impacted by class imbalances, which can affect detection accuracy. |
| ICACT20250023 Slide.15
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
After analysis, we observed that CNN detected 0 anomalies, indicating low performance in its current configuration. To improve its effectiveness, it requires more advanced sequence preprocessing to better capture temporal dependencies in log data |
| ICACT20250023 Slide.16
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
After analysis, we observed that RNN detected 0 anomalies, demonstrating low performance in its current configuration. To enhance its effectiveness, it requires more advanced sequence preprocessing to fully exploit temporal dependencies in log data.
DBSCAN is the most effective in terms of precision and handling false positives.Deep learning models require adjustments to improve effectiveness. |
| ICACT20250023 Slide.17
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Machine Learning Approaches
The machine learning models demonstrated good anomaly detection capabilities but were highly dependent on parameter selection to achieve optimal performance.
Isolation Forest detected 197 anomalies but suffered from a high false positive rate, leading to low precision (0.00152).
DBSCAN achieved perfect precision and recall but detected only 3 anomalies, indicating a limited detection range.
One-Class SVM detected 94 anomalies, showing good generalization ability but low precision (0.0319) due to class imbalances.
Deep Learning Approaches
Deep learning models showed promising potential but were limited by the need for large, high-quality datasets and proper hyperparameter tuning.
CNN and RNN (LSTM) failed to detect any anomalies, highlighting the need for better sequence preprocessing and model adaptation.
Autoencoders detected 86 anomalies, demonstrating sensitivity to log variations but were affected by class imbalances, leading to low precision (0.0349).
False Positives and False Negatives
The results indicate a need for further optimization of both machine learning and deep learning models to reduce:
False Positives: Avoid unnecessary alerts that can overwhelm security analysts.
False Negatives: Prevent missing critical anomalies that could indicate cyber threats.
Conclusion: DBSCAN emerged as the best-performing model in terms of precision, but future improvements should focus on hybrid approaches that combine machine learning and deep learning techniques to enhance detection accuracy. |
| ICACT20250023 Slide.18
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
To improve model performance, it is essential to optimize hyperparameters for deep learning models and explore hybrid approaches that combine machine learning and deep learning techniques. Additionally, the integration of Transformer models can help capture complex dependencies in Android logs more effectively. Enhancing log representation with advanced embedding techniques will further refine anomaly detection capabilities.
Conclusion: Implementing these technical and methodological adjustments is crucial to achieving better model accuracy and reliability.
|
| ICACT20250023 Slide.19
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
DBSCAN proved to be the most effective for Android log anomaly detection.
Deep learning models require tuning to be more efficient.
Hybrid approaches and advanced models like Transformers could provide better solutions.
AI integration in Android device cybersecurity is crucial for detecting and preventing cyberattacks.
|
| ICACT20250023 Slide.20
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
These references justify the methodology and algorithm choices presented throughout the study. |
| ICACT20250023 Slide.21
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Thanks for your attention! Now! Floor is open for QnA. Welcome any comment or question
|
