Controlled Evaluation of Metacognitive-Gated Retrieval: Domain-Dependent Payoff Boundaries and a Transferable Diagnostic Framework

Abstract: Metacognitive-gated dynamic retrieval is widely used for hallucination mitigation in large language models, yet it rests on three untested assumptions: self-reported confidence correlates with correctness (H1), a tuned threshold enables decision-making (H2), and layering gating on grounding is additive (H3). On four small-to-mid open-weight instruction models, with full retrieval as the baseline, we run exhaustive threshold replay over synthetic and decontaminated real SEC corpora, together with a same-trigger-rate random control and an oracle upper bound. Self-reported confidence yields AUCs of only 0.341–0.500 for predicting correctness, and no interior threshold dominates both endpoints; switching to semantic entropy raises within-stratum AUC to 0.886/0.750, yet end-to-end performance never beats always-retrieve. Formally, a gate beats always-retrieve iff the precision of its confident answers exceeds retrieval accuracy (Theorem 1), while the self-report channel carries only 0.000–0.069 bits of knowledge information—a readout bottleneck (Theorem 2). This failure is domain-dependent: on open-domain factoid QA the same confidence is discriminative (AUC 0.64–0.69), and gating reliably beats RAG under imperfect retrieval. We distill a domain-agnostic diagnosis-and-deployment framework, and report two methodological defects: referent ambiguity of abstention confidence and cross-run irreproducibility. All confidence-derived metrics should be reported with k/n.