 ICACT20230240 Slide.00
[Big Slide]
|
Chrome  Click!! |
 |
Hi everyone, in this paper, we will talk about the problem of Activation functions for deep learning and apply for an application for rare attack detection in wireless local area network (WLAN). This work has been done by our group from Hanoi University of Industry, Vietnam, Vietnam Academy of Science and Technology, Vietnam, School of AI and Advanced Computing, XJTLU Entrepreneur College (Taicang), Xi'an Jiaotong - Liverpool University, Suzhou, Jiangsu Province, P. R. China, Information Technology Institute, Vietnam National University, Hanoi, Vietnam and Contents Convergence Software Research Institute, Kyonggi University, South Korea.
|
| ICACT20230240 Slide.01
[Big Slide]
|
Chrome Click!! |
 |
This is the outline of our talk. Firstly, we will give some introduction about our paper. Secondly, we will present some kind of activation function that can be applied for deep leaning. Thirdly, we will conduct some experimentations and discussion. Finally, we conclude the paper.
|
| ICACT20230240 Slide.02
[Big Slide]
|
Chrome Click!! |
 |
WLAN wireless networks, also known as Wi-Fi local area networks, are the most common type of connection commonly used at the household or work scale and in public places. This is a network using wireless technologies with a coverage range of less than 200m. Due to the popularity as well as the open space of the wireless network (easy scale), it is also the type of network that is vulnerable to intrusion/attack due to many subjective users or lack of knowledge about setting up encryption, and security for connection. Three main groups of attacks are
Flooding (DDOS), Injection, and Impersonation.
|
| ICACT20230240 Slide.03
[Big Slide]
|
Chrome Click!! |
 |
Common types of attacks in WLAN are presented in the table 1. In recent years, to prevent network attacks, we can use some machine learning methods to build a model applied for intrusion detection system. Generally, the conditional methods that can be applied for intrusion detection system are support vector machine, Naïve Bayes, Decision Tree, etc. However, in recent years, the deep learning method has received a lot of attention.
|
| ICACT20230240 Slide.04
[Big Slide]
|
Chrome Click!! |
 |
The choice of activation function has a crucial impact on the performance of the system using a neural network model, and different activation functions may be used in different parts of the model. So, the motivation of our work is to implement an empirical study on activation functions for Intrusion detection data sets.
|
| ICACT20230240 Slide.05
[Big Slide]
|
Chrome Click!! |
 |
Some main activation functions are ReLU, Leaky ReLU (LReLU), ELU (Exponential Linear Unit) , and SegRELU. The aim of activation function in a neural network is to define how the weighted sum of the input is mapped into an output from a node or nodes in a layer of the network.
|
| ICACT20230240 Slide.06
[Big Slide]
|
Chrome Click!! |
 |
The first one is ReLU function. This function proposed by Vinod Nair and Geoffrey E. Hinton and widely use in deep learning models. The properties of ReLU function are continuous, not-bounded and not zero-centered. The ReLU function is presented in the equation 1.
|
| ICACT20230240 Slide.07
[Big Slide]
|
Chrome Click!! |
 |
The second one is Leaky ReLU (LReLU), The properties of the LReLU are continuous and not-bounded. The computation using LReLU is cheap and having a zero-centered activation function. The LReLU function allows a small part of negative units instead of pushing them to zero like the ReLU function does. The equation 2 shows the definition of the LReLU.
|
| ICACT20230240 Slide.08
[Big Slide]
|
Chrome Click!! |
 |
The third one is ELU (Exponential Linear Unit) function. In contrast to ReLUs, ELUs have negative values which allows them to push mean unit activations closer to zero like batch normalization but with lower computational complexity. ELUs performs not only to faster learning, but also to significantly better than ReLUs and LReLUs on networks having more than 5 layers. The ELU function is expressed in the equation 3.
|
| ICACT20230240 Slide.09
[Big Slide]
|
Chrome Click!! |
 |
The last one is SegRELU proposed by Gan et al. SegRELU has some properties: (1) parameter-free setting to ease users’ implementation, (2) light architecture to encourage reproducibility in future deep learning models, and (3) robust weight update at both the positive and negative part of the activation functions to preserve effective feature learning during training process.
|
| ICACT20230240 Slide.10
[Big Slide]
|
Chrome Click!! |
 |
To conduct the experiment, we used a part of the AWID dataset (Aegean WiFi Intrusion Dataset) that was actually collected from attacks in the wireless network, denoted O-AWID. AWID consists of 156 attributes, we select 13 features according to the study of Liu et al.. The characteristic of these data set is presented in the table 2.
|
| ICACT20230240 Slide.11
[Big Slide]
|
Chrome Click!! |
 |
To evaluate the performance, the Accuracy and F1-score is used in our experimentation. The F-score, also called the F1-score, is a measure of a model's accuracy on a dataset. The detail of F1-score is in the slide.
|
| ICACT20230240 Slide.12
[Big Slide]
|
Chrome Click!! |
 |
This slide shows the results obtained by the system for two data set O-AWID1 and O-AWID2. We also show the results for J48 as a reference. In each table, we present the results produced when using 4 kind of activation functions. From these tables, we can see that the results obtained by each type of activation function is different. When using the SegRELU function, we can obtain the good results. It can be explained by the fact that, the characteristic of SegRELU is more adapt than the others when applied for these data sets.
|
| ICACT20230240 Slide.13
[Big Slide]
|
Chrome Click!! |
 |
This slide shows the results obtained by the system for two data set O-AWID3 and O-AWID4. Similarly, the results obtained by each type of activation function is different. Using the SegRELU function, we obtained the good results.
|
| ICACT20230240 Slide.14
[Big Slide]
|
Chrome Click!! |
 |
In this paper, we make a brief survey about the main activation functions used for deep learning up to date. The activation function plays an important role in model based on neural network, especially, deep learning. The experimentation results obtained show that each activation function contribute different impact for CNN model. In the future, the new activation function needs to be further studied. When we deal with real applications, the use of different kinds of activation functions should be tried and choosing the best one.
|
| ICACT20230240 Slide.15
[Big Slide]
|
Chrome Click!! |
 |
Thank you for your attention!
|
