 |
This slide shows the end-to-end architecture of our adaptive retrieval system, split into an offline indexing phase and an online adaptive inference phase.
Starting with the offline phase, we ingest cyber threat intelligence corpora such as NVD and MITRE ATT&CK and build two complementary indices. The first is a sparse TF-IDF index that captures exact lexical matches, which are especially important for vulnerability IDs and precise technical terms. The second is a dense semantic index built using MiniLM embeddings and FAISS, which helps retrieve conceptually relevant passages even when wording differs.
During online inference, the process begins with a user¡¯s threat intelligence query. Instead of retrieving a fixed number of documents, we start with a small retrieval depth, or top-k, using this hybrid retriever.
The retrieved context is passed to the LLM to generate a summary. We then immediately evaluate that generated summary using the Grounded Entailed Claim Score, or GECS, which measures how well each claim in the summary is supported by the retrieved evidence.
GECS acts as a feedback signal to the adaptive controller. If factual grounding improves meaningfully when we increase retrieval depth, the controller allows another retrieval step. If the factual gains saturate or the latency budget is exceeded, retrieval stops.
The key point here is that the LLM itself is unchanged. The intelligence lives in the controller: retrieval depth becomes a dynamic, per-query decision rather than a static configuration, allowing the system to balance factual accuracy and cost in real time.
|
IEEE/ICACT20260248 Slide.09
[Big Slide]