 |
This slide illustrates how retrieval depth affects factual consistency, measured using GECS, across different types of CTI data.
On the x-axis, we vary the retrieval depth, or top-k, and on the y-axis we plot the mean GECS score. What we observe is a clear pattern of diminishing returns.
For structured data like NVD vulnerability descriptions, factual gains increase quickly with small amounts of retrieved context, but they saturate very early. Beyond that point, additional retrieval does not improve factual grounding and can even introduce noise.
For semi-structured narrative data like MITRE ATT&CK, the gains are weaker and more variable. These descriptions are more conceptual and less fact-dense, so adding more context does not necessarily translate into stronger entailment.
This behavior highlights why a fixed retrieval depth is suboptimal. A single top-k value either over-retrieves for simple, structured queries or underperforms for more complex ones.
Our adaptive approach leverages this saturation behavior explicitly. By monitoring marginal improvements in GECS, the system learns when additional retrieval is no longer justified and stops early, preserving factual quality without unnecessary computation.
|
IEEE/ICACT20260248 Slide.18
[Big Slide]
[YouTube]