 |
In convolutional layers, the output pixel correlates with part of the input pixels. So we can get the output pixel directly without computing all the intermediate features. We propose the cross-layer scheduling for dynamic neural networks, so that the off-chip memory access of intermediate layers can be reduced and CTC can be improved.
This figure illustrates the cross-layer scheduling of two convolutional layers, and the kernel sizes are 1 times 1 and 3 times 3. The dashed lines represent the pruned channels. Data has to be tiled due to limited on-chip memory, and T_h, T_w, T_c denote the tile shape of height, width, and channels, respectively. If the data is tiled on C dimension, then extra partial sums will be produced, resulting in 4 times memory overhead (the features are 8-bit quantized and partial sums are 32-bit). In order to benefit the DMA transactions with continuous data, we cache the entire row in the on-chip SRAM. Therefore, it is more appropriate to tile on H dimension.
|
IEEE/ICACT20230038 Slide.08
[Big Slide]
[YouTube]