Hunter-Seeker Docs For agents This page as markdown

The two loops

Hunter-Seeker runs two loops. They are different mechanisms with different failure modes, and conflating them is how a system ends up grading its own homework.

Loop A — self-governing. Is this prediction good enough to act on, for this row, right now? Loop B — self-improving. Did acting on it actually change anything?

Nothing here is new vocabulary. Every mechanism below has an established name in the literature, and the table at the bottom gives each one a citation. If you know the term, you already know what the mechanism does; what is unusual is only that they are wired together behind one MCP surface.

Loop A — self-governing

hs_rank_topk  ──►  hs_model_quality  ──►  hs_score_entity  ──►  hs_verify_verdict
   fit + rank         is it trustworthy?      band this row        prove it to a third party
      │                      │                     │                        │
      │                      │                     │                        └─ status: valid
      │                      │                     └─ act / escalate / refuse + max_autonomy
      │                      └─ top_decile_lift · calibration_error · validation · leak_guard
      └─ honest-empty below the lift bar: a terminal result, and the run is refunded

The loop is closed at the top, not the bottom. hs_rank_topk either clears the published lift bar or returns a structured refusal; below the bar there is no ranking to interrogate and no scorecard to score against, so the rest of the loop never starts. See Refusal is a result.

hs_score_entity is where the per-row decision happens. It returns a band, and the band is engine-authored — nothing above the engine may set or move it. It also returns max_autonomy, a ceiling for that one decision.

The band does not say whether the outcome is good. act means the engine will vouch that this row is likely; whether likely-to-churn is bad and likely-to-convert is good is a judgement about your domain, and the engine does not make it. That fact is on the Verdict as verdict.outcome.polarity, and reading it is the caller's job. [test: packages/mcp/test/agent-decides-valence.test.ts]

The autonomy sentence, stated exactly

The database schema that defines this says it in one line, and it is worth quoting rather than paraphrasing: autonomy never exceeds executed-only-on-a-trigger. [test: packages/mcp/test/surfaces-parity.test.ts::B7: the root AGENTS.md is short, command-first, and links rather than inlines]

There are two autonomy settings, and neither is "full autonomy":

approve_each — the default. Every action waits for a human.
preapproved:<class> — a standing grant for one named class of action, chosen by the operator. Under such a grant the loop does close on its own: the relay enqueues without a human waitpoint, and no person is in the moment of the decision.

A blanket grant is refused: the operator has to choose a class. So "a human is always in the loop" would be false, and "it runs fully autonomously" would also be false. What is true is narrower: it acts without a human only for a class of action a human named in advance, only on a trigger, and never above the max_autonomy ceiling the engine put on that specific decision.

Loop B — self-improving

hs_explain_levers ──► hs_attest_action ──► hs_report_outcome ──► hs_action_evidence ──► hs_drift_status
   what to change        you did it,          what happened          did it work?          has the pattern
                         and how far          (append-only)          (null below floor)     itself moved?
                                                                            │                     │
                                                                            │                     └─ refit_of ──► new run
                                                                            └─ live: null until 30 per cell
                                                                               small_n until 100

Four properties make this loop honest rather than flattering:

hs_report_outcome is append-only and never retrains. A fact store that revises history cannot be evidence about it. Reporting an outcome does not change any model.
hs_action_evidence refuses below a floor. live is null until each cell has 30 rows and flagged small_n below 100. A difference computed on four entities is not evidence, and returning one would be worse than returning nothing. [live: S3]
It compares within the same pattern. Attestation-compliant versus not, for entities the same model flagged — not "acted" versus "the rest of the world".
hs_drift_status reports labels and directions, never a new number. abandon requires two consecutive honest-empties, so a single bad refresh cannot retire a pattern.

Loop A and Loop B are never merged. hs_model_quality answers "is this model any good on held- out data"; hs_action_evidence answers "did acting on it change outcomes". Merging them would let the model's own fit statistics stand in for evidence that acting works — the specific failure this separation exists to prevent. What that separation does not defend against is selection bias: see The randomized control arm.

The vocabulary, with citations

Every row is an established term. If a row has no citation, that is a bug in this page.

Hunter-Seeker mechanism	Established name	Primary source
`refuse` / `escalate` bands; honest-empty	Selective prediction / classification with a reject option	Chow, On optimum recognition error and reject tradeoff (1970) · Geifman & El-Yaniv, Selective Classification for Deep Neural Networks (2017)
`escalate` → a human decides	Learning to defer	Madras, Pitassi & Zemel, Predict Responsibly (2018)
`score` as a probability	Calibration	Guo, Pleiss, Sun & Weinberger, On Calibration of Modern Neural Networks (2017)
`calibration_error`	Expected Calibration Error (ECE)	Naeini, Cooper & Hauskrecht, Obtaining Well Calibrated Probabilities (2015)
`validation` (out-of-time / holdout)	Out-of-time validation; model risk management	Federal Reserve SR 26-2, Revised Guidance on Model Risk Management (2026) — supersedes SR 11-7 (2011)
`top_decile_lift`, `lift_curve`	Lift and cumulative gains	Provost & Fawcett, Data Science for Business (2013)
`reading` (`trace@1`, `windowed@1`)	Point-in-time correctness	Feast, Point-in-time joins (documentation, checked 2026-09-03)
`leak_guard`	Target leakage	Kaufman, Rosset & Perlich, Leakage in Data Mining (KDD 2011)
`max_autonomy` (L0–L4)	Levels of automation	SAE J3016, Taxonomy and Definitions for Driving Automation Systems
`principal_reasons`	Adverse action reasons	ECOA Regulation B, 12 CFR §1002.9
`hs_action_evidence` (acted vs not)	Uplift modelling	Gutierrez & Gérardy, Causal Inference and Uplift Modelling: A Review (2017)
Judging an action from logged decisions	Off-policy / counterfactual evaluation	Bottou et al., Counterfactual Reasoning and Learning Systems (JMLR 2013)
The control arm this page recommends	Randomized experiment; switchback design	Bojinov, Simchi-Levi & Zhao, Design and Analysis of Switchback Experiments (2020)
Adjusting for who got acted on	Propensity score	Rosenbaum & Rubin, The Central Role of the Propensity Score (1983)

Next: Refusal is a result · The randomized control arm · How it's different · Status: served vs planned