ICACT20230043 Slide.17
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Thanks for your attention! Now! Floor is open for QnA. Welcome any comment or question.
|
|
ICACT20230043 Slide.16
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Here is thr references to related work
|
|
ICACT20230043 Slide.15
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In this paper, we propose a scheduling memory access architecture based on clos interconnect network, and analyze the influence of HBM bulit in interconnect structure on bandwidth. A experiment is done to test its performance. The experimental results show that proposed scheme can be implemented and effectively reduce the influence of HBM internal interconnection on bandwidth. We believe the scheduling memory access architecture will be improved in the future, so that the bandwidth of HBM can be used more effectively in parallel algorithm calculation.
|
|
ICACT20230043 Slide.14
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
We realize parallel BFS algorithm on VCU128 development board by emplying HBM and memory access scheduling module. The Table III shows the resource consumption of HBM interconnect module on FPGA,
The Table IV shows the effective bandwidth of HBM built in interconnect module, the effective bandwidth of HBM clos interconnect module and HBM Connect on FPGA. Compared HBM Connect,
the effective bandwidth of the clos interconnect is lower, but
it uses less resources and it has scalability, which could be
expanded to use more HBMs easily due to the orthogonal link
selection. The experimental results show that the effective bandwidth of the HBM clos interconnect is more than twice that of the HBM built in interconnect, which is in line with expectations.
|
|
ICACT20230043 Slide.13
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The output level unit is similar to the input level unit, consisting of data distributor, rrarbiter and FIFO.As shown in the Fig, The OM allocates the received data to corresponding ports. After IM and CM selection, data that in the range of OM would be sent to the corresponding OM output ports. Therefore, it is very convenient to allocate the received data to the corresponding OM output port to complete the task.
|
|
ICACT20230043 Slide.12
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
As shown in the Fig, the intermediate level unit is composed of two parts. The first part is the data allocator, which selects data and sends datas to the corresponding data selector according to the time subslot. The second part is the data selector, which transmits the data received from the data distributor to the corresponding CM output port. CM is uesd to send the data received from IM to the corresponding OM unit according to the different time, which does not lead to data collision.
|
|
ICACT20230043 Slide.11
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
As shown in the Fig, each input level unit is composed of five parts.FIFO,DEMUX,RRARbiter FIFO, MUX. The first part is FIFO, this part is used to store the transmitted data in a burst to avoid the loss or invalidity of transmitted data; the second part is the data distributors(demux) connected to the input ports,through which the input data is sent to the corresponding FIFOs; the third part is the rrarbiters connecting between data distributor and FIFO, which is used to avoid the data collision conflict; the fourth part is FIFO corresponding to the output level unit; the fifth part is the mux connected to the output port of FIFO and the output interface of the input level unit,respectively.
|
|
ICACT20230043 Slide.10
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In order to avoid collision during data transmission in
Clos network, an orthogonal link selection is adopted. To
prevent data collision during transmission, IMs cannot send data through the same CM to the same OM at the same time. So we fixed the paths from IM to CM and from CM to OM. The port of each IM is sent to its destination OM via a different CM; CM would send data received from IM to the corresponding OM, so we just need to make sure that different IMs send data to different OM through the same CM at the same time.
|
|
ICACT20230043 Slide.09
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
There are several multi-level network topologies, including
Omega, Clos, Benes, and butterfly networks. Among them,
we choose Clos network because of the scalability and fixed switching delay of Clos networks, and the orderiness of data
transmission. The three-stage Clos switching network is composed of input, intermediate and output stage. There is only one link
between modules at each level. The input stage consists of K
N × M crossbars. The intermediate stage is composed of M crossbars of K ×K. The output stage consists of K M × N crossbars. That M represents the number of input ports in each module and N represents the number of output ports in each module,K represents the number of input and output ports of the intermediate stage.
|
|
ICACT20230043 Slide.08
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The contributions of this paper can be summarized as
follows:
We propose an efficient and scalable HBM interconnection architecture designed based on clos network.
We use RTL based solutions to achieve customized crossbar, and
HBM access scheduling optimization.
We evaluate the customized crossbar system on the VCU128
and analyze its performance.
|
|
ICACT20230043 Slide.07
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In recent years, several articles have proposed their own solutions to the above HBM problems, such as HBM Connect and Memory Access Optimizer IP core. HBM Connect uses a selective structure based on the butterfly switching network which could achieve higher effective bandwidth due to this characteristics. The butterfly network reduces its hop distance at the later stages of the custom crossbar. But it is difficult to use in the case of multiple HBMs. For MAO IP
core, its paper does not introduce its specific structure, so
it would not be discussed here.
|
|
ICACT20230043 Slide.06
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
HBM AXI Switch Connections enable global addressing from any AXI port to any portion of the HBM, but the shared connections limit the maximum throughput laterally to 50% of the full bandwidth.For East/West transactions within an AXI Switch instance there is no lateral performance loss.For East/West transactions which leave an AXI Switch instance there will be a lateral throughput performance loss.A Write sequence with near 100% MC throughput results in the largest drop in throughput due to the extra switching cycle. As a result, smaller blocks have a larger percentage loss than larger
blocks. Therefore, in order to optimize HBM access scheduling.
we propose a custom crossbar to replace HBM’s Built in
crossbar. Resolving the problem of the limited number of
lateral connections and try to make data arrive at the same
time to improve the uniformity.
|
|
ICACT20230043 Slide.05
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Here is HBM AXI Switch Connections,Two lateral connections are provided between sets of 4 masters x 4 slaves within the switch,
with one lateral connected to M0 and M1, and the other connected to M2 and M3. The shared connections limit the maximum throughput laterally to 50% of the full bandwidth, but enables global addressing from any AXI port to any portion of the HBM.
|
|
ICACT20230043 Slide.04
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
For HBM2,it contains two HBM stacks, each stack has eight channels,one channel is divided into two pseudo channels,so each HBM stack has 16 pseudo channels totally. For 4 Hi-stack each PCH provides 2Gbit storage capacity, there are 8 memory controllers
in each stack, each of which controls the accesses towards two
adjacent PCHs. The switch network is composed of 8 4×4
mini-switches,each of which connects two adjacent MCs and
exposes 4 AXI interfaces to the FPGA. Every two adjacent mini-switches are connected with a bus to provide global addressing, such that a memory access towards an arbitrary PC can be issued from any AXI interface.
|
|
ICACT20230043 Slide.03
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
High bandwidth memory is a CPU/GPU memory chip,which is many DDR chips stacked together by 3D die-stacking and TSV technology,and packaged together with GPU to achieve a large-capacity, highwidth DDR composite array. The HBM stack is not physically integrated with the CPU or GPU, but is tightly and quickly connected through the intermediary layer. The HBM has almost the same characteristics as the chip integrated RAM, therefore, has higher speed and bandwidth.
HBM is package-level memory based on a wide input/output parallel interface.
|
|
ICACT20230043 Slide.02
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The growing demand for computational power has recently reached new heights with the widespread adoption of deep learning and big data technologies. However the external memory access of FPGA is much slower than that of GPUs. The resulting processor memory gap has been a major challenge for more than 20 years. HBM is one of the memory architectures for high throughput applications.HBM has higher speed and bandwidth than DRAM, and HBM is package-level memory based on a wide input/output parallel interface.
|
|
ICACT20230043 Slide.01
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Hi everybody, my name is Shuang Xue. For ICACT 2023, I have cooperated with two partners from University of Science and Technology of China.Huawen Liang and Qizhe Wu. Our presentation is about a solution for memory access optimization, that our title is Scheduling Memory Access Optimization for HBM Based on CLOS.
|
