 ICACT20240459 Slide.00
[Big Slide]
|
Chrome  Click!! |
 |
Introduction |
| ICACT20240459 Slide.01
[Big Slide]
|
Chrome Click!! |
 |
Intelligent Anomaly Detection System Based on Ensemble and Deep Learning |
| ICACT20240459 Slide.02
[Big Slide]
|
Chrome Click!! |
 |
Anomaly Detection System
Introduction
Methodology
Dataset Description
Experimental Results
Conclusion
Future Research Direction
References
Q & A
|
| ICACT20240459 Slide.03
[Big Slide]
|
Chrome Click!! |
 |
In the virtual environment, just like in the physical world, individuals with malicious intentions are consistently active around the clock.
Safeguarding computer systems and network resources from unauthorized access, as well as protecting critical information and user credentials, is always top priority.
Attackers typically employ low-footprint strategies to minimize traces and remain undetected for as long as possible.
This approach allows them to achieve their goals with a reduced risk of discovery. |
| ICACT20240459 Slide.04
[Big Slide]
|
Chrome Click!! |
 |
There are many innovative ideas have been put forwarded to develop an intelligent intrusion detection system (IDS). The IDS is classified into two major categories:
|
| ICACT20240459 Slide.05
[Big Slide]
|
Chrome Click!! |
 |
Our first concern to address uneven distribution of samples among classes, where one class significantly outweighs the other, may introduce a bias toward the majority of samples.
|
| ICACT20240459 Slide.06
[Big Slide]
|
Chrome Click!! |
 |
Samples distribution |
| ICACT20240459 Slide.07
[Big Slide]
|
Chrome Click!! |
 |
We addressed the imbalance by augmenting the size of the minority class through the generation of new instances.
This strategy enhances the model’s ability to learn the minority class pattern, it also poses the risk of overfitting.
We constructed the Random Forest classifier (with multiple decision trees) and combines their predictions, offering resilience against overfitting compared to individual decision trees.
|
| ICACT20240459 Slide.08
[Big Slide]
|
Chrome Click!! |
 |
Total training samples: 82,332
Total testing samples: 175,341
|
| ICACT20240459 Slide.09
[Big Slide]
|
Chrome Click!! |
 |
Feature selection:
The identification of relevant attack features is crucial for precise anomaly detection in complex and high-dimensional datasets.
It helps in removing irrelevant features from feature pool, thereby minimizing computational time.
|
| ICACT20240459 Slide.10
[Big Slide]
|
Chrome Click!! |
 |
Experimental results:
The proposed method performances a comprehensive analysis utilizing various metrics, such as accuracy, precision, recall, f1-score, and the receiver operating characteristic curve.
We also computed the confusion matrix, a square matrix where the diagonal elements represent correctly classified instances, and the off-diagonal elements represent misclassified instances.
|
| ICACT20240459 Slide.11
[Big Slide]
|
Chrome Click!! |
 |
The confusion matrix for the UNSW-NB15 dataset, employing a RF and DNN classifiers, reveal the model’s effectiveness in distinguishing between “Normal” and “Attack” categories shown in Table 3 and 4.
|
| ICACT20240459 Slide.12
[Big Slide]
|
Chrome Click!! |
 |
Receiver Operating Characteristic by using Random Forest (RF):
It is a graphical representation that illustrates the performance of a binary classification model at various classification thresholds. It is a widely used tool in statistics and machine learning to assess the trade-off between sensitivity (true positive rate) and specificity (true negative rate) of a classification model. |
| ICACT20240459 Slide.13
[Big Slide]
|
Chrome Click!! |
 |
ROC curve of Deep Neural Network for both binary class and multi class. |
| ICACT20240459 Slide.14
[Big Slide]
|
Chrome Click!! |
 |
The confusion matrix you provided appears to be assessing the performance of a classification model on a dataset with different classes. Each row in the matrix represents the instances of a true class, and each column represents the instances predicted by the model. The numbers in the cells indicate the counts of instances falling into each combination of true class and predicted class. |
| ICACT20240459 Slide.15
[Big Slide]
|
Chrome Click!! |
 |
Confusion matrix of UNSW-NB15 dataset of normal and sub-categories of attack by using DNN classifier. |
| ICACT20240459 Slide.16
[Big Slide]
|
Chrome Click!! |
 |
The Table 7 presents a comparative analysis of the performance
measures between RF and DNN classifiers, assessing their effectiveness in both two-class and multiclass scenarios.
In the two-class classification, RF outperforms DNN across various metrics.
RF achieves a higher accuracy (90.20%) compared to DNN (88.35%), and a superior F1-score (90.45%) compared to DNN (88.63%).
The precision of RF (91.98%) also exceeds that of DNN (90.93%). Moving to the multiclass setting, RF maintains its dominance, exhibiting a higher accuracy (75.17%), F1-score (72.93%), and precision (77.44%) compared to DNN (74.33%, 73.07%, and 77.86%, respectively).
These results emphasize the robust performance of RF in both two-class and multiclass classification scenarios, highlighting its efficacy in accurately classifying instances across various metrics. |
| ICACT20240459 Slide.17
[Big Slide]
|
Chrome Click!! |
 |
We implemented an ADS utilizing RF and DNN classifiers to identify anomaly and normal patterns.
Both classifiers effectively distinguish various intruder patterns, and we explored crucial attack features for precise anomaly detection in complex, high-dimensional datasets.
We employed a feature selection technique to identify pertinent features and minimize computational complexity.
With multiple subcategories of attacks, each with distinct characteristics, RF demonstrated the ability for complex pattern recognition.
|
| ICACT20240459 Slide.18
[Big Slide]
|
Chrome Click!! |
 |
The highly imbalanced UNSW-NB15 dataset prompted us to implement oversampling, carefully designing RF and DNN to prevent overfitting.
We evaluated performance using diverse metrics, including classification accuracy, class label accuracy using the confusion matrix, precision, recall, F1-score, and ROC curve.
The consistent and convincing results obtained from both classifiers underscore the effectiveness and reliability of the proposed method.
|
| ICACT20240459 Slide.19
[Big Slide]
|
Chrome Click!! |
 |
We are going to extend generative adversarial network (GAN) for zero-day attack detection and imbalance class problem too.
|
| ICACT20240459 Slide.20
[Big Slide]
|
Chrome Click!! |
 |
Reference |
| ICACT20240459 Slide.21
[Big Slide]
|
Chrome Click!! |
 |
Questions and Answers |
