# The two loops Hunter-Seeker runs two loops. They are different mechanisms with different failure modes, and conflating them is how a system ends up grading its own homework. **Loop A — self-governing.** Is this prediction good enough to act on, for *this* row, right now? **Loop B — self-improving.** Did acting on it actually change anything? Nothing here is new vocabulary. Every mechanism below has an established name in the literature, and the table at the bottom gives each one a citation. If you know the term, you already know what the mechanism does; what is unusual is only that they are wired together behind one MCP surface. --- ## Loop A — self-governing ``` hs_rank_topk ──► hs_model_quality ──► hs_score_entity ──► hs_verify_verdict fit + rank is it trustworthy? band this row prove it to a third party │ │ │ │ │ │ │ └─ status: valid │ │ └─ act / escalate / refuse + max_autonomy │ └─ top_decile_lift · calibration_error · validation · leak_guard └─ honest-empty below the lift bar: a terminal result, and the run is refunded ``` The loop is closed at the top, not the bottom. `hs_rank_topk` either clears the published lift bar or returns a structured refusal; below the bar there is no ranking to interrogate and no scorecard to score against, so the rest of the loop never starts. See [Refusal is a result](/docs/refusal-is-a-result). `hs_score_entity` is where the per-row decision happens. It returns a band, and the band is **engine-authored** — nothing above the engine may set or move it. It also returns `max_autonomy`, a ceiling for that one decision. **The band does not say whether the outcome is good.** `act` means the engine will vouch that this row is *likely*; whether likely-to-churn is bad and likely-to-convert is good is a judgement about your domain, and the engine does not make it. That fact is on the Verdict as `verdict.outcome.polarity`, and reading it is the caller's job. [test: packages/mcp/test/agent-decides-valence.test.ts] ### The autonomy sentence, stated exactly The database schema that defines this says it in one line, and it is worth quoting rather than paraphrasing: **autonomy never exceeds executed-only-on-a-trigger.** [test: packages/mcp/test/surfaces-parity.test.ts::B7: the root AGENTS.md is short, command-first, and links rather than inlines] There are two autonomy settings, and neither is "full autonomy": - **`approve_each`** — the default. Every action waits for a human. - **`preapproved:`** — a **standing grant for one named class of action**, chosen by the operator. Under such a grant the loop **does** close on its own: the relay enqueues without a human waitpoint, and no person is in the moment of the decision. A blanket grant is refused: the operator has to choose a class. So "a human is always in the loop" would be false, and "it runs fully autonomously" would also be false. What is true is narrower: *it acts without a human only for a class of action a human named in advance, only on a trigger, and never above the `max_autonomy` ceiling the engine put on that specific decision.* --- ## Loop B — self-improving ``` hs_explain_levers ──► hs_attest_action ──► hs_report_outcome ──► hs_action_evidence ──► hs_drift_status what to change you did it, what happened did it work? has the pattern and how far (append-only) (null below floor) itself moved? │ │ │ └─ refit_of ──► new run └─ live: null until 30 per cell small_n until 100 ``` Four properties make this loop honest rather than flattering: 1. **`hs_report_outcome` is append-only and never retrains.** A fact store that revises history cannot be evidence about it. Reporting an outcome does not change any model. 2. **`hs_action_evidence` refuses below a floor.** `live` is `null` until each cell has 30 rows and flagged `small_n` below 100. A difference computed on four entities is not evidence, and returning one would be worse than returning nothing. [live: S3] 3. **It compares within the same pattern.** Attestation-compliant versus not, for entities the same model flagged — not "acted" versus "the rest of the world". 4. **`hs_drift_status` reports labels and directions, never a new number.** `abandon` requires two consecutive honest-empties, so a single bad refresh cannot retire a pattern. **Loop A and Loop B are never merged.** `hs_model_quality` answers "is this model any good on held- out data"; `hs_action_evidence` answers "did acting on it change outcomes". Merging them would let the model's own fit statistics stand in for evidence that acting works — the specific failure this separation exists to prevent. What that separation does **not** defend against is selection bias: see [The randomized control arm](/docs/control-arm). --- ## The vocabulary, with citations Every row is an established term. If a row has no citation, that is a bug in this page. | Hunter-Seeker mechanism | Established name | Primary source | |---|---|---| | `refuse` / `escalate` bands; honest-empty | **Selective prediction / classification with a reject option** | [Chow, *On optimum recognition error and reject tradeoff* (1970)](https://ieeexplore.ieee.org/document/1054406) · [Geifman & El-Yaniv, *Selective Classification for Deep Neural Networks* (2017)](https://arxiv.org/abs/1705.08500) | | `escalate` → a human decides | **Learning to defer** | [Madras, Pitassi & Zemel, *Predict Responsibly* (2018)](https://arxiv.org/abs/1711.06664) | | `score` as a probability | **Calibration** | [Guo, Pleiss, Sun & Weinberger, *On Calibration of Modern Neural Networks* (2017)](https://arxiv.org/abs/1706.04599) | | `calibration_error` | **Expected Calibration Error (ECE)** | [Naeini, Cooper & Hauskrecht, *Obtaining Well Calibrated Probabilities* (2015)](https://ojs.aaai.org/index.php/AAAI/article/view/9602) | | `validation` (out-of-time / holdout) | **Out-of-time validation; model risk management** | [Federal Reserve SR 26-2, *Revised Guidance on Model Risk Management* (2026)](https://www.federalreserve.gov/supervisionreg/srletters/sr2602.htm) — supersedes SR 11-7 (2011) | | `top_decile_lift`, `lift_curve` | **Lift and cumulative gains** | [Provost & Fawcett, *Data Science for Business* (2013)](https://data-science-for-biz.com/) | | `reading` (`trace@1`, `windowed@1`) | **Point-in-time correctness** | [Feast, *Point-in-time joins* (documentation, checked 2026-09-03)](https://docs.feast.dev/getting-started/concepts/point-in-time-joins) | | `leak_guard` | **Target leakage** | [Kaufman, Rosset & Perlich, *Leakage in Data Mining* (KDD 2011)](https://dl.acm.org/doi/10.1145/2020408.2020496) | | `max_autonomy` (L0–L4) | **Levels of automation** | [SAE J3016, *Taxonomy and Definitions for Driving Automation Systems*](https://www.sae.org/standards/content/j3016_202104/) | | `principal_reasons` | **Adverse action reasons** | [ECOA Regulation B, 12 CFR §1002.9](https://www.consumerfinance.gov/rules-policy/regulations/1002/9/) | | `hs_action_evidence` (acted vs not) | **Uplift modelling** | [Gutierrez & Gérardy, *Causal Inference and Uplift Modelling: A Review* (2017)](https://proceedings.mlr.press/v67/gutierrez17a.html) | | Judging an action from logged decisions | **Off-policy / counterfactual evaluation** | [Bottou et al., *Counterfactual Reasoning and Learning Systems* (JMLR 2013)](https://arxiv.org/abs/1209.2355) | | The control arm this page recommends | **Randomized experiment; switchback design** | [Bojinov, Simchi-Levi & Zhao, *Design and Analysis of Switchback Experiments* (2020)](https://arxiv.org/abs/2009.00148) | | Adjusting for who got acted on | **Propensity score** | [Rosenbaum & Rubin, *The Central Role of the Propensity Score* (1983)](https://academic.oup.com/biomet/article/70/1/41/240879) | --- *Next:* [Refusal is a result](/docs/refusal-is-a-result) · [The randomized control arm](/docs/control-arm) · [How it's different](/docs/how-its-different) · [Status: served vs planned](/docs/status)