ICACT20240037 Slide.20
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Thanks for your attention! Welcome any comment or question.
|
|
ICACT20240037 Slide.19
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
As a result of comprehensive analysis of the experimental results for the two datasets, some of the SNNs using ACL have no loss of accuracy compared to all-to-all connection.
And in some cases, it provides an optimal solution that reduces the number of connections while achieving better accuracy.
Ultimately, we confirmed that applying ACL proposed in this paper can simplify and optimize the HW design without the loss of accuracy.
In the future, we plan to utilize the 2D features of images for the connections in order to use fewer wires.
And we can consider adjacent connections in different topologies.
|
|
ICACT20240037 Slide.18
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The last slide is conclusions and future works.
|
|
ICACT20240037 Slide.17
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The experimental results showed that the accuracy when using adjacent-connection is slightly higher than that when using all-to-all connection.
This phenomenon may appear somewhat unusual, but it may come from the insufficient performance of the learning framework.
As can be seen in this figure, The framework employs a heuristic that does not search the entire solution space, but rather focuses on finding local optimum.
Therefore, reducing the solution space by using ACL can yield better results, while further reduction ultimately led to worse results.
If there was an ideal algorithm, all-to-all connection would always provide higher accuracy..
|
|
ICACT20240037 Slide.16
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
This figure presents the results of the experiment conducted on the Fashion-MNIST dataset.
When compared to all-to-all connection, we can see that the accuracy is barely reduced until the number of connections is reduced by 200.
From the results, in the case of Fashion-MNIST, the optimal number of adjacent-connection is 200 while maintaining accuracy.
|
|
ICACT20240037 Slide.15
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
This figure presents the results of the experiment conducted on the MNIST dataset.
This figure shows a slight improvement in accuracy when using ACL instead of FCL in the structure of the network. And the improved accuracy is maintained from 300 to 50 connections.
And we confirmed that when the number of connections is less than 100, the accuracy decreases as the number of connections decreases.
Considering this, using ACL with 100 connections is optimal for the MNIST dataset .
|
|
ICACT20240037 Slide.14
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The backpropagation through time based learning algorithm was used to train the SNN models.
To implement ACL, we used a custom mask function to control synaptic connections.
|
|
ICACT20240037 Slide.13
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The network structure used for classification of the two datasets consists of input, hidden, and output layers.
The integrate and fire neuron model was used as the neuron model of SNN.
|
|
ICACT20240037 Slide.12
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
For the experiments, MNIST and Fashion-MNIST datasets were used.
We performed SNN simulations using the snntorch library.
|
|
ICACT20240037 Slide.11
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Next, I will tell you about the experiments.
|
|
ICACT20240037 Slide.10
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
This approach is different from conventional pruning techniques.
Pruning is a method of lightening the model by training it not to use some of the connections, although they are implemented.
Therefore, pruning is not a technique that affects implementation.
Ultimately, the implementation of all-to-all connections is necessary because the required connections may change with each training session.
|
|
ICACT20240037 Slide.09
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
This slide explains the strength and weakness of the ACL.
First, as a strength, ACL can reduce the difficulty of manual design by reducing complexity depending on the number of wires,
and as synaptic connections are reduced, the area of operation logic and memory can be proportionally reduced.
Finally, HW designs can be simplified and optimized without the accuracy drop.
A weakness is that it cannot model the information that unconnected synapses can store, which can lead to poor performance.
|
|
ICACT20240037 Slide.08
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
To reduce the number of wires, we proposed an adjacently-connected layer (ACL) where only some neurons are connected.
As shown in this figure, each neuron in the current layer is connected to neurons determined to be adjacent in the next layer. And the number of these connecting neurons is limited.
Reducing the number of connections in the ACL can potentially impact inference accuracy, so we will investigate this effect in experiments slides.
|
|
ICACT20240037 Slide.07
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The following slides describe the approach proposed in our paper.
|
|
ICACT20240037 Slide.06
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
There are several challenges in implementing Fully-Connected Layers (FCL) when implementing SNNs. Firstly, FCL requires a substantial number of wires, leading to increased complexity, expanded area, longer wire delays, and reduced scalability when implemented on a chip. A second challenge arises from the manual implementation required for SNNs, especially in low-power scenarios that require asynchronous or analog circuit design. This manual design process requires significant time and effort to implement wire-intensive FCLs in SNNs.
|
|
ICACT20240037 Slide.05
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
For AI in embedded systems, an SNN that operates at low power is required.
Circuits used to implement hardware that allows SNNs to operate at low power include analog circuits and asynchronous digital circuits.
However, these circuits cannot be automated at HDL(Hardware Description Language)-level and are complex circuits that must be implemented manually.
Thus, in this paper, we will propose a methodology to alleviate this difficulty.
|
|
ICACT20240037 Slide.04
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Therefore, SNN can be an attractive method for implementing AI in embedded systems with these limitations.
The implementation of Spiking neural networks (SNNs) offer a distinct advantage over other AI technologies, primarily due to their low-power operation.
Neuromorphic chips that implement spiking neural networks (SNNs) mostly operate at low power, providing a distinct advantage over other AI technologies.
So, Spiking neural networks (SNNs) are a highly promising AI technology for embedded systems, owing to their energy-efficient properties.
However, the manual implementation of SNNs encounters practical challenges because of the all-to-all connections in large networks.
I will explain these challenges in detail.
|
|
ICACT20240037 Slide.03
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Embedded systems are specialized computing systems used a wide range of applications, such as the examples in this figure.
The recent advancements in AI have spurred a growing demand for AI capabilities in embedded systems.
Unlike existing computers, embedded systems have strong constraints on power consumption, area, and real-time processing.
Therefore, it is important to optimize both hardware and software to meet the requirements.
|
|
ICACT20240037 Slide.02
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
First, the introduction slides explain the background of this paper.
|
|
ICACT20240037 Slide.01
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The contents of this presentation are Introduction, Proposed Approach, Experiments, and Conclusions & Future Works.
|
|
ICACT20240037 Slide.00
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Hi everyone, my name is Hyeonguk Jang. This presentation is about Optimizing Implementation of SNN for Embedded Systems. Start the presentation from now own.
|
