Skip to Content
Numinance
Concepts

Reliability and recovery

How independent verification, bounded repair, canary releases, rollback and escalation limit the cost of a wrong change.

View as .mdUpdated
Under development
Verification, repair and recovery run in the proprietary runtime. Playbook and verifier contracts are public-spec.

Failure is expected. The design makes it small, finite and visible.

Independent verification

The agent that wrote a change does not decide that it is good. Verifiers run under the reviewer identity, in a workspace the author does not control, against the toolchain declared for the repository. A LanguageToolchain names the verifiers, for example unit tests, lint and type checking, and the playbook says which must pass for a step.

A verifier that cannot run says so. In the generated Python profile, unit tests execute candidate code and are reported unavailable in the live verifier provider until its nested sandbox exists. Unavailable is not the same as passed.

Verification and repair

When a check fails, what exactly can happen next, and what can never happen?

Illustrative · simulated
Scenario
max_repair_attempts
  1. 1● current

    Author

    author-agent

  2. 2pending

    Deterministic checks

    tests, lint, typecheck

  3. 3pending

    Independent review

    reviewer-agent

  4. 4pending

    Rehearsal

    disposable environment

  5. 5pending

    Canary

    release-pipeline

  6. 6pending

    Release

    release-pipeline only

Step 1 of 8state: In progress

Author implements the change

Who acts
author-agent
Evidence recorded
candidate and diff recorded by the runner
Repair attempts
attempt 0/2

What can happen next

  • Check fails: bounded repair, then re-check
  • Attempts or budget used up: pause and escalate; never widen the grant
  • Reviewer disagrees: escalate to a human
  • Canary regresses: rollback under the release identity

◆ Never, at any step

Repair cannot widen the grant, change protected paths, or approve itself. The author never reviews or deploys its own change; a human or policy accepts, not the model.

Step 1 of 8. Author implements the change. Acting identity: author-agent. Repair attempts 0 of 2.

● current stage✓ passed✕ failed or escalated◆ fixed rule
Text description

Illustrative and simulated: no change is running. The generated Playbook sets max_repair_attempts: 2; the field accepts 0 to 10 and the control here shows 0 to 3.

  1. The author-agent implements. Deterministic checks (tests, lint, typecheck) run next.
  2. A reviewer-agent, a separate identity from the author, reviews independently and returns findings.
  3. The change is rehearsed in a disposable environment, then canaried and released by the release-pipeline, the only identity that deploys.
  4. If a check fails, the change goes back to the author for a bounded repair, with the findings reference as the only new input. After max_repair_attempts it is escalated to a human.
  5. If a budget is used up, the change pauses and escalates; it never gets a wider grant. If the reviewer disagrees, it escalates. If the canary regresses, it rolls back.

Invariant in every case: repair cannot widen the grant, change protected paths, or approve itself.

Bounded repair

When verification fails, the change can enter repair: the failure is given back to the author agent for another attempt. This loop is finite. A playbook sets max_repair_attempts, and the generated bugfix-bounded playbook uses 2. There is no unbounded recursion, no "keep trying until green", and each attempt is a recorded state transition. When attempts are exhausted the change moves to CHANGE_STATE_ESCALATED.

This is bounded repair, not open-ended autonomy. The system performs a small number of governed retries, checks them independently, and hands over to a human.

Canary, observation and rollback

If the autonomy level allows release, the release pipeline uses the playbook's canary window. During CHANGE_STATE_OBSERVING signals are compared against the thresholds. A regression leads to CHANGE_STATE_RECOVERING, and a successful recovery ends at CHANGE_STATE_ROLLED_BACK. Rollback is performed by the release-pipeline identity under its own grant, so the recovery path has authority designed for it rather than borrowed from the author.

Escalation

Escalation is a normal outcome, not an error. It happens when repair is exhausted, a gate denies or requires approval, context is insufficient, or the breaker trips. The change waits for a human signal and carries its evidence with it, so the person picking it up sees what was tried.

Failure modes

FailureResponse
Verifier failsBounded repair, then escalation.
Verifier unavailableReported as unavailable. The step does not count as passed.
Evidence cannot be sealedThe failure trips the breaker.
Release regressesRecover and roll back.
Breaker tripsPause, then resume on reset.

Next steps

On this page