Now 5048 visitors
Today:0 Yesterday:0
Total: 147036 212S 50P 4R
2026-10-20, Week 43
Member Login
TACT Journal Homepage
Call for Paper
Committee
Find My Paper
Author Homepage
Statistics & History
Welcome Message
Paper Procedure
FAQ
Paper Archives
Outstanding Papers
Program & Proceedings
Presentation Platform
Hotel & Travel Info
Photo Gallery
Scheduler Login
Seminar
Archives Login
Sponsors


















Presentation Assistant Go to [Q&A Session]
Paper Number
Paper Title
Keyword
ICACT20260320 Slide.01        [Big Slide]      [YouTube] Chrome Click!!
Hello everyone，my name is Zhixiang Zhao. Today I’m very happy to present our work at ICACT 2026. The title of our paper is “GCCache: A Non-blocking and Low-latency General-purpose Context Cache Design for RDMA NICs”. This work was jointly conducted by, Guo Zhichuan, Song Mangu, and Ling Zhiyuan, and all of us are from Institute of Acoustics, Chinese Academy of Sciences.
ICACT20260320 Slide.02        [Big Slide]      [YouTube] Chrome Click!!
My presentation consists of four parts: Background and Motivation, Method Design, Experiments, and Conclusion.
ICACT20260320 Slide.03        [Big Slide]      [YouTube] Chrome Click!!
Lets start with the background and motivation.
ICACT20260320 Slide.04        [Big Slide]      [YouTube] Chrome Click!!
RDMA has become a critical infrastructure in todays AI clusters, data center networks, and high-performance computing environments, thanks to its ultra-low latency and high throughput capabilities. As shown in this figure, this is a typical RDMA communication architecture between a requester and a responder. Due to the hardware offloading nature of RNICs, the on-chip resources are quite limited and cannot accommodate a large number of connections. Under normal operation, the RNIC retrieves context entries ,such as QPC, CQC, and MPT ,directly from the on-chip cache for fast, low-latency access. It only falls back to the slower host memory when a cache miss occurs. This means the performance of the RNIC context cache is critical to overall system efficiency.
ICACT20260320 Slide.05        [Big Slide]      [YouTube] Chrome Click!!
Now lets look at the challenges in current RNIC cache designs. The first challenge is Head of Line Blocking. Most existing RNIC caches follow a simple principle: first request, first response. But this creates a problem. As illustrated in figure a, when the first request from QP1 misses the cache, it has to fetch the context from the host memory. During this time, even if the subsequent request, QP2, could hit the cache, it still has to wait until QP1 gets its response. This is classic head of line blocking, and it significantly hurts performance. What we want instead is shown in figure b, the cache should not block the entire pipeline just because one request has a cache miss. Ideally, QP2 should be able to proceed immediately if it hits the cache, without waiting for QP1.
ICACT20260320 Slide.06        [Big Slide]      [YouTube] Chrome Click!!
The second challenge is Redundant Host Accesses. In real workloads, consecutive requests received by the cache may actually be identical. For example, multiple packets from the same memory region may arrive in a short burst. As shown in figure a, if the first request from MR1 misses the cache, it triggers a fetch from the host memory. But before the response comes back, several more MR1 requests may arrive. In current designs, each of these requests will also trigger a separate host access, even though theyre all asking for the same context. This results in significant PCIe bandwidth waste, as highlighted in red here. What we want is shown in figure b: when repeated requests miss the cache, they should only trigger one host access, and all of them should share the same response.
ICACT20260320 Slide.07        [Big Slide]      [YouTube] Chrome Click!!
The third challenge is High Latency. When the RNIC data processing pipeline queries the cache, it must wait for the response before proceeding. So high cache access latency directly degrades overall system performance. To give you a concrete example: at a clock frequency of 200 MHz, if the RNIC needs to forward 1024-byte packets at 100 Gbps line rate, the cache must return a response within just 16 clock cycles. Thats an extremely tight timing budget, and any delay can become a bottleneck. The fourth challenge is Lack of Compatibility. As we saw earlier, an RNIC manages several different types of contexts , QPC, CQC, SRQC, MPT, MTT, and so on. In traditional designs, engineers often build specialized caches for each context type separately. This approach works, but it requires a long development cycle and makes the system harder to maintain and extend. What we really need is a unified, general-purpose cache architecture that can handle all these context types efficiently.
ICACT20260320 Slide.08        [Big Slide]      [YouTube] Chrome Click!!
In the next section, I will present our method design.
ICACT20260320 Slide.09        [Big Slide]      [YouTube] Chrome Click!!
Now let me introduce the overall architecture of GCCache. As shown in this figure, GCCache consists of three main subsystems, each highlighted in a different color. On the left in green is the Scheduler Subsystem. It handles incoming requests from multiple RNIC pipelines and dispatches responses back. The key components here include the Request Handler, Response Handler, and Target Handler with a PID Pool for tracking requests. In the middle in yellow is the Cache Subsystem , the core of our design. It contains the Cache Core with TAG, LRU, and status RAM, along with the data RAM. Several processing modules handle cache hits, replacements, and state transitions through a central finite state machine. On the right in blue is the ORP Subsystem ,which stands for Outstanding Request Processing. This subsystem manages cache misses. It includes the Context Adapter, which interfaces with the RNIC DMA engine via AXI, and the Outstanding Manager with its Outstanding Table to track in-flight requests.
ICACT20260320 Slide.10        [Big Slide]      [YouTube] Chrome Click!!
Let me now explain the Scheduler Subsystem in detail. This subsystem manages request and response scheduling, and it contains three modules. First, the REQ HANDLER. It receives requests from multiple RNIC pipelines, buffers them and parses each request. After arbitration, requests are forwarded to the Target Handler. Second, the TARGET HANDLER. It maintains a pool of Process IDs. When a new request arrives, a PID is popped from the pool and assigned to that request. When the request completes, the PID is returned to the pool. Third, the RESP HANDLER. It receives responses from the Cache Subsystem and distributes them back to the correct pipeline based on the pipeline ID.
ICACT20260320 Slide.11        [Big Slide]      [YouTube] Chrome Click!!
Now lets look at the Cache Subsystem, which acts as the central decision-making unit of GCCache. On the left, you can see the three core functions. First, it determines hits by rapidly validating tag matches. Second, it updates states. Third, it triggers the ORP Subsystem when a cache miss occurs. On the right, you can see the logic flow. When a request arrives with address and data, it enters the Core State Machine, which coordinates three parallel processing modules: Hit Processing performs tag comparison and data retrieval. State Processing handles coherence updates and status checks. Replacement applies eviction policies and selects cache lines for replacement. The key design choice here is that these three modules operate in parallel. The Core State Machine synchronizes their results and decides the appropriate action. If the hit module indicates a cache hit, the state module updates the cache status, and hit information is output directly , achieving very low latency. If a cache miss occurs, the miss information is sent to the ORP Subsystem to fetch data from the host. This parallel architecture directly addresses Challenge 3 , the high latency requirement , by ensuring that cache lookups complete within just a few clock cycles.
ICACT20260320 Slide.12        [Big Slide]      [YouTube] Chrome Click!!
Now lets examine the ORP Subsystem. It is core role to handle cache misses asynchronously without blocking the main cache pipeline. As shown in the flow diagram on the right, when a cache miss occurs, it enters the ORP Subsystem and flows through two modules. First, the Outstanding Manager. It maintains an outstanding table that tracks all pending miss requests, recording information such as the PID and request order. This table is essential for managing multiple in-flight DMA requests and ensuring responses are correctly matched to their original requests. Second, the Context Adapter. It interfaces directly with the RNICs DMA engine via the AXI interface. When the Outstanding Manager forwards a miss request, the Context Adapter initiates a DMA read to fetch the required context from host memory. When the response returns, it flows back through the Outstanding Manager, which retrieves the corresponding miss information and loops it back to the Scheduler Subsystem for cache update. If we need to support a different context type, switching from QPC to MTT, we only need to modify the Context Adapter module. All other modules remain unchanged. This directly addresses Challenge 4 , the diverse context types in RDMA systems. This asynchronous design also helps address Challenge 2 by efficiently batching and managing host memory accesses.
ICACT20260320 Slide.13        [Big Slide]      [YouTube] Chrome Click!!
This slide highlights one of the most important design principles in GCCache , the decoupling of hit processing and miss processing. First, Decoupling. Hit processing and miss processing are architecturally separated. This separation ensures that the complex, variable-latency miss handling logic does not interfere with the fast path of hit processing. Second, Fixed Latency. The Cache Subsystem operates deterministically. As you can see in the diagram on the right, the main pipeline , from incoming requests through tag check, data access, to hit response , maintains a predictable, fixed cycle count. This is critical for meeting the strict timing requirements of RNIC pipelines. Third, No Blocking. When a miss occurs, it triggers the ORP Unit, but the main pipeline is not blocked. The cache continues to accept and process new requests immediately. If those new requests hit the cache, they complete without waiting for the previous miss to be resolved, directly addressing Challenge 1.
ICACT20260320 Slide.14        [Big Slide]      [YouTube] Chrome Click!!
This slide presents an important optimization in GCCache , the deduplication mechanism for handling repeated cache misses. Let me walk you through the scenario shown in the diagram on the right. At time T0, Request A with PID 10 arrives and misses on address 0xA000. This creates an entry in the Outstanding Request Table, and a DMA request is sent to the host. Shortly after, at time T1, Request B with PID 11 arrives , and its also requesting the same address 0xA000. This is a repeated miss. The system checks the Outstanding Request Table for existing entries before issuing a new DMA request. When a match is found , it links the new request to the pending one via the Next PID pointer. The DMA request for PID 11 is blocked and saved , no redundant host access is generated.
ICACT20260320 Slide.15        [Big Slide]      [YouTube] Chrome Click!!
When the response finally returns from the host, the Outstanding Manager will process both PID 10 and PID 11 using the linked list, forwarding them back to the Scheduler Subsystem. As shown in the flow diagram on the left, the process has four steps: Step 1: Response Arrival. The system detects an incoming response packet from the host via the DMA engine. Step 2: Select Entry by PID. Using the PID stored in the AXI ID field, the Outstanding Manager identifies the corresponding entry in the outstanding table. Step 3: Scheduler . The system first sends the original request to the Scheduler Subsystem. Then, it traverses the next PID chain to find all linked duplicate requests. Step 4: Cleanup. All linked requests are served sequentially, and the valid bits are cleared to free the table entries. The key mechanism here is the linked-list structure. It allows us to handle all dependent requests efficiently in a single batch and ensuring all duplicate requests are served in order from just one host response.
ICACT20260320 Slide.16        [Big Slide]      [YouTube] Chrome Click!!
Now that we have covered the architecture and key design mechanisms of GCCache, lets move on to Part 3: Experiments.
ICACT20260320 Slide.17        [Big Slide]      [YouTube] Chrome Click!!
Commercial RNICs are integrated circuits with fixed cache architectures, and existing hardware-implemented designs do not disclose their implementation details. Therefore, we developed an RNIC with proprietary intellectual property that integrates GCCache. The design was simulated using Cocotb and then implemented on the U200 platform. On the right, you can see our four evaluation metrics: Resource Utilization , Cache Latency , Throughput and Compatibility.
ICACT20260320 Slide.18        [Big Slide]      [YouTube] Chrome Click!!
Lets start with the resource utilization results. First, Let me briefly explain our cache configuration. The MTT cache is configured as a 4 way set associative cache with 512 sets and 8 cache lines per set , this larger configuration is because MTT entries are frequently accessed during memory translation. The other context caches , are configured as 4 way 512 set caches with 1 cache line per set, which is sufficient for their access patterns. Now, looking at the resource consumption numbers. Compared to existing designs, our architecture achieves significantly lower resource utilization. It leaves room for future scalability , we can potentially increase cache capacity or add more cache instances without resource constraints.
ICACT20260320 Slide.19        [Big Slide]      [YouTube] Chrome Click!!
Next, lets look at the cache latency comparison. GCCache achieves a fixed latency of only 5 clock cycles , and this is consistent across all context types. In contrast, the baseline design shows latencies exceeding 10 clock cycles for most context types.
ICACT20260320 Slide.20        [Big Slide]      [YouTube] Chrome Click!!
Now let us examine the throughput results, which demonstrate the real-world performance impact of GCCache. The figure on the right shows the throughput of our RNIC under a single Queue Pair with varying message sizes. First, our performance is comparable to commercial NICs. This validates that GCCache does not introduce any performance penalty. Second, we achieve significantly better results compared to Xilinx ERNIC. The ERNIC struggles to reach high throughput even with large messages, while our design saturates the link much earlier. Third, when the message size exceeds 1KB, our throughput approaches 100Gbps . This is directly enabled by the low latency of GCCache. This result confirms that GCCache successfully eliminates the context caching bottleneck for high-speed RDMA operations.
ICACT20260320 Slide.21        [Big Slide]      [YouTube] Chrome Click!!
Finally, let me present a functional comparison between GCCache and existing designs. This table summarizes the key differentiating features. We evaluate along three dimensions: First, Non-blocking support. GCCache provides full non-blocking support through our deduplication and outstanding manager mechanisms. SRNIC and ScalaRNIC do not support non-blocking access at all. csRNA only partially addresses this. ERNIC technically avoids blocking, but only by storing all context on-chip, which severely limits scalability to just a few hundred QPs. Second, Full Context support. GCCache supports caching for all five context types. In contrast, all prior works support only a subset of these context types. This means they cannot provide a complete solution for real-world RDMA workloads that require all these metadata structures. Third, Hardware Implementation. GCCache has been fully implemented in hardware on a Xilinx U200 FPGA. ScalaRNIC and csRNA were only evaluated in simulation. As you can see, GCCache is the only design that achieves all three checkmarks: full non-blocking support, complete context type coverage, and validated hardware implementation. This makes it a comprehensive and practical solution for high-performance RDMA NICs.
ICACT20260320 Slide.22        [Big Slide]      [YouTube] Chrome Click!!
Now, let me conclude this presentation.
ICACT20260320 Slide.23        [Big Slide]      [YouTube] Chrome Click!!
Let me conclude by summarizing the contributions of this work.  In this paper, we addressed the critical challenges of context cache in RDMA NICs.  We proposed and implemented GCCache, a non-blocking, low-latency, general-purpose cache architecture. Let me highlight our four key contributions:  First, Non-blocking Architecture. GCCache is the first architecture to decouple hit and miss processing paths. By separating these two flows and using an outstanding request manager, we completely eliminate head-of-line blocking, ensuring continuous operation.  Second, Bandwidth Efficiency. We introduced an deduplication mechanism that detects and merges duplicate cache misses for the same context. This significantly reduces redundant PCIe transactions and saves precious memory bandwidth.  Third, High Performance. Through careful hardware pipelining and parallel processing, we achieved an ultra-low cache response latency. This enables our RNIC to sustain 100 Gbps line-rate throughput, matching the performance of commercial RNICs.  Fourth, Scalability. GCCache provides a unified caching solution for all RDMA context types.  In summary, GCCache provides a comprehensive, practical, and high-performance solution to the context caching problem in modern RNICs.
ICACT20260320 Slide.24        [Big Slide]      [YouTube] Chrome Click!!
Thank you very much for your attention!