Dependency upgrade
A designed walkthrough of a low-risk dependency upgrade change, from triage to a pull request, tied to the real Playbook and AutonomyPolicy examples.
This is a storyboard of the Numinance engine, which is proprietary and not generally available. The configuration documents shown are real and validate with the open CLI. There is no runnable engine command in this guide.
When to use this
Read this guide to see what you would configure, and what you would approve, if you let an autonomous engineer open dependency bumps for you. It assumes you have completed Onboard a repository and read Choose an autonomy level.
The configuration that drives it
The Playbook is the public example from the spec. It has two steps, each with its own capability set and declared risk.
apiVersion: numinance.dev/v1
kind: Playbook
metadata: {name: dependency-upgrade, scope: SCOPE_ORG}
spec:
change_type: dependency-upgrade
steps:
- {name: bump, capability_set: scm-write, risk: RISK_CLASS_LOW, evidence: [diff, lockfile-digest], verifiers: [unit-tests, lint]}
- {name: open-pr, capability_set: scm-pr, risk: RISK_CLASS_MEDIUM, evidence: [pr-ref], rehearse: false}
max_repair_attempts: 2
qualification: {min_runs: 20, max_rework_rate: 1000, max_regression_rate: 0, max_rollback_rate: 50, zero_security_escapes: true}Read the fields as obligations:
capability_setnames the set a step's grant is derived from. The playbook shapes a grant and never widens it.evidencelists what the step must record. Missing declared evidence turns success intoPARTIAL.verifiersrun on the step's output, independently of the agent that produced it.max_repair_attempts: 2bounds the repair loop. The proto allows 0 to 10 and defaults to 2.qualificationis what must be shown before this playbook's autonomy can be promoted. Rates are basis points out of 10000, somax_rework_rate: 1000is a tenth of runs.
The org AutonomyPolicy example pairs with it:
rules:
- {match: {playbook: dependency-upgrade, risk: RISK_CLASS_LOW}, level: AUTONOMY_LEVEL_MERGE}Read the examples together
A change's risk class is the maximum over its playbook steps, and the resolved risk of a step is at least the highest action risk of the RPCs its capability set grants. The example playbook declares open-pr as RISK_CLASS_MEDIUM, so a change that runs it resolves to MEDIUM and the LOW-only rule above does not match; the default AUTONOMY_LEVEL_PR applies. To earn the MERGE rule you would need a playbook whose steps and granted actions all resolve to LOW. Check this before you rely on a rule.
The storyboard
The lifecycle states below are the fixed ChangeState machine. Stage names in your StageRegistry map onto them.
Change lifecycle
What does a durable change do from objective to evidence, and where does it pause?
- Understand / Classify
- ◇ Plan gate
- Implement
- Verify
- ┄ Repair (bounded)
- Rehearse
- ◇ Release gate
- Merge
- Release
- Observe
- ┄ Recover
- Evidence seal
Step 1 of 10 · Understand / Classify
Context is resolved from authorised sources and the Change is classified.
- Lifecycle state
- Opened, Understood, Classified
- Who acts
- engine (read-only grant)
- Temporal holds (references only)
- change_id, state, context bundle artifact_ref + sha256
- Customer artifact storage holds
- Repository index, resolved context bundle, risk classification
- Evidence record emitted
- Context resolved: sources used and digests
Happy path, step 1 of 10: Understand / Classify. Context is resolved from authorised sources and the Change is classified.
Text description
A Change is a durable state machine, not one long agent session. The agent implements one work unit at a time; planning, gates, verification, release and recovery are performed by Numinance with fresh authority per stage. Orchestration payloads carry references and digests only; source, diffs, prompts, logs and findings bodies live in customer artifact storage.
Stages in order: Understand / Classify, Plan gate, Implement, Verify, Repair (bounded), Rehearse, Release gate, Merge, Release, Observe, Recover, Evidence seal. Verification failure re-enters implementation with the findings reference as the only new input, at most max_repair_attempts times (default 2 in the generated Playbook), then the Change is escalated. Plan and release gates can allow, deny or require approval; approvals come from a human other than the author.
Happy path. Every gate allows and verification passes first time.
- Understand / Classify: Context is resolved from authorised sources and the Change is classified. Acts: engine (read-only grant). Evidence: Context resolved: sources used and digests.
- Plan gate: Policy allows the plan under the initial grant. Acts: engine plans; policy decides. Evidence: Policy decision with reasons, plan digest frozen before any write grant.
- Implement: One work unit runs in a sandbox under a work-unit grant. Acts: author-agent (work-unit grant). Evidence: Author run completed: candidate digest, grant id, budget used.
- Verify: Deterministic checks pass; the reviewer adds findings, never a verdict. Acts: reviewer-agent (read-only) and deterministic verifiers. Evidence: VerdictIssued: layers, findings reference.
- Rehearse: A disposable environment rehearses migrations and deployment; it is destroyed at its TTL. Acts: engine with environment provider; reviewer identity. Evidence: Rehearsal evidence: timings and plan digests.
- Release gate: Policy consumes the verdict and the evidence list and allows the release. Acts: policy decides; release-pipeline identity only after allow. Evidence: Gate decision referencing the verdict, never raw findings.
- Merge: The release pipeline merges. At PR level the Change closes here; higher levels continue. Acts: release-pipeline. Evidence: Merge recorded with operation id; PR-level autonomy ends here as Closed.
- Release: Only the release pipeline deploys, and only if the autonomy level allows. Acts: release-pipeline (the only deployer). Evidence: Deployment recorded under the release identity.
- Observe: The canary window is evaluated against the declared SLOs. Acts: engine evaluates canary window. Evidence: Canary result against declared SLOs.
- Evidence seal: The runner seals the evidence sequence and the Change closes. Acts: runner identity (evidence writer). Evidence: Sequence sealed with an outcome record.
Verification fails, then repair. A failing check feeds one bounded repair attempt, then verification passes.
- Understand / Classify: Context is resolved and the Change is classified. Acts: engine (read-only grant). Evidence: Context resolved: sources used and digests.
- Plan gate: Policy allows the plan. Acts: engine plans; policy decides. Evidence: Policy decision with reasons, plan digest frozen before any write grant.
- Implement: The author-agent produces a first candidate. Acts: author-agent (work-unit grant). Evidence: Author run completed: candidate digest, grant id, budget used.
- Verify: A deterministic check fails. The findings are stored and referenced by digest. Acts: reviewer-agent (read-only) and deterministic verifiers. Evidence: VerdictIssued: layers, findings reference.
- Repair (bounded): Attempt 1 of 2 (the default cap). The findings reference is the only new input. Acts: engine re-enters Implementing; author-agent. Evidence: Repair attempt recorded with its number and findings digest.
- Implement: The author-agent runs again under a fresh grant. Acts: author-agent (work-unit grant). Evidence: Author run completed: candidate digest, grant id, budget used.
- Verify: All verification layers pass this time. Acts: reviewer-agent (read-only) and deterministic verifiers. Evidence: VerdictIssued: layers, findings reference.
- Rehearse: A disposable environment rehearses migrations and deployment; it is destroyed at its TTL. Acts: engine with environment provider; reviewer identity. Evidence: Rehearsal evidence: timings and plan digests.
- Release gate: Policy consumes the verdict and the evidence list and allows the release. Acts: policy decides; release-pipeline identity only after allow. Evidence: Gate decision referencing the verdict, never raw findings.
- Merge: The release pipeline merges. At PR level the Change closes here; higher levels continue. Acts: release-pipeline. Evidence: Merge recorded with operation id; PR-level autonomy ends here as Closed.
- Release: Only the release pipeline deploys, and only if the autonomy level allows. Acts: release-pipeline (the only deployer). Evidence: Deployment recorded under the release identity.
- Observe: The canary window is evaluated against the declared SLOs. Acts: engine evaluates canary window. Evidence: Canary result against declared SLOs.
- Evidence seal: The runner seals the evidence sequence and the Change closes. Acts: runner identity (evidence writer). Evidence: Sequence sealed with an outcome record.
Repair exhausted, escalated. Two bounded repairs fail. The Change escalates to a human instead of trying forever.
- Understand / Classify: Context is resolved and the Change is classified. Acts: engine (read-only grant). Evidence: Context resolved: sources used and digests.
- Plan gate: Policy allows the plan. Acts: engine plans; policy decides. Evidence: Policy decision with reasons, plan digest frozen before any write grant.
- Implement: First candidate. Acts: author-agent (work-unit grant). Evidence: Author run completed: candidate digest, grant id, budget used.
- Verify: Verification fails. Acts: reviewer-agent (read-only) and deterministic verifiers. Evidence: VerdictIssued: layers, findings reference.
- Repair (bounded): Attempt 1 of 2. Acts: engine re-enters Implementing; author-agent. Evidence: Repair attempt recorded with its number and findings digest.
- Implement: Second candidate. Acts: author-agent (work-unit grant). Evidence: Author run completed: candidate digest, grant id, budget used.
- Verify: Verification fails again. Acts: reviewer-agent (read-only) and deterministic verifiers. Evidence: VerdictIssued: layers, findings reference.
- Repair (bounded): Attempt 2 of 2. Acts: engine re-enters Implementing; author-agent. Evidence: Repair attempt recorded with its number and findings digest.
- Implement: Third candidate. Acts: author-agent (work-unit grant). Evidence: Author run completed: candidate digest, grant id, budget used.
- Verify: Verification fails a third time. The cap is reached. Acts: engine; waits for a human signal. Evidence: Escalation recorded with the findings digest; no grant is widened.
Approval required. Policy answers require approval. The workflow waits for a human other than the author.
- Understand / Classify: Context is resolved and the Change is classified as higher risk. Acts: engine (read-only grant). Evidence: Context resolved: sources used and digests.
- Plan gate: Policy answers require approval. The workflow now waits on a signal with a timer (default 7 days). Acts: policy; human approver pending. Evidence: Policy decision: require approval, approver group, author excluded.
- Plan gate: A human approver grants. The signal is rejected if the approver equals the author. Acts: human approver. Evidence: Approval record with approver principal and decision id.
- Implement: Work proceeds under the approved grant. Acts: author-agent (work-unit grant). Evidence: Author run completed: candidate digest, grant id, budget used.
- Verify: Verification passes. Acts: reviewer-agent (read-only) and deterministic verifiers. Evidence: VerdictIssued: layers, findings reference.
- Rehearse: A disposable environment rehearses migrations and deployment; it is destroyed at its TTL. Acts: engine with environment provider; reviewer identity. Evidence: Rehearsal evidence: timings and plan digests.
- Release gate: Policy consumes the verdict and the evidence list and allows the release. Acts: policy decides; release-pipeline identity only after allow. Evidence: Gate decision referencing the verdict, never raw findings.
- Merge: The release pipeline merges. At PR level the Change closes here; higher levels continue. Acts: release-pipeline. Evidence: Merge recorded with operation id; PR-level autonomy ends here as Closed.
- Release: Only the release pipeline deploys, and only if the autonomy level allows. Acts: release-pipeline (the only deployer). Evidence: Deployment recorded under the release identity.
- Observe: The canary window is evaluated against the declared SLOs. Acts: engine evaluates canary window. Evidence: Canary result against declared SLOs.
- Evidence seal: The runner seals the evidence sequence and the Change closes. Acts: runner identity (evidence writer). Evidence: Sequence sealed with an outcome record.
Denied by policy. The plan gate denies. The denial is recorded and the Change is abandoned with no write grant ever issued.
- Understand / Classify: Context is resolved and the Change is classified. Acts: engine (read-only grant). Evidence: Context resolved: sources used and digests.
- Plan gate: Policy denies the plan. Denials are evidence too. Acts: policy decision point. Evidence: Denial with rule ids as reasons and the policy decision id.
- Evidence seal: The runner seals the sequence. No code was written. Acts: runner identity (evidence writer). Evidence: Sequence sealed with an outcome record.
Opened and understood
A dependency alert or ticket is accepted at intake and the change is opened. The context resolver then builds the picture of the repository: the code graph, repository docs and the approved knowledge bundle. Retrieved material is treated as untrusted guidance and cannot create authority.
Evidence emitted: the intake reference and the context bundle digest.
Classified
The playbook (dependency-upgrade) and the risk class are fixed, and the initial grant is derived from profile, playbook step, risk and the resolved autonomy level. This is where the AutonomyPolicy and its floors apply. The agent cannot change the outcome.
Planned and plan gate
The plan lists work units, a risk per step and the evidence each requires. Policy evaluates the plan before any side effect. In the generated stage registry this is the GATE_KIND_PLAN stage; it is decided by policy, not by a person, and the denial or approval is recorded.
Implementing: the bump step
The author-agent runs in a sandbox under the scm-write grant. It edits the manifest and lockfile on a branch. It does not hold merge or deploy capability, and it cannot edit .numinance/policy/.
Evidence emitted: diff and lockfile-digest.
Verifying and bounded repair
The unit-tests and lint verifiers run against the candidate. These checks are separate from the author. If one fails, the change enters the repair state, where the author may try again, at most max_repair_attempts times. After the last attempt, the change escalates to a human rather than looping.
Release gate and the pull request
The release gate evaluates policy on all evidence. At the pull-request level the open-pr step opens a PR under the scm-pr grant, and a human maintainer reviews it. The reviewer-agent and author-agent cannot approve it. At a level that allows merging, policy permits the merge only if no matching floor caps it.
Evidence emitted: pr-ref. The change closes and the evidence is sealed.
An illustrative stage log
change dependency-upgrade opened
context resolved graph, docs, knowledge bundle (digests recorded)
classify risk=MEDIUM level=PR (default; no rule matched)
plan-gate allow decision recorded
bump implementing grant=scm-write
verify unit-tests FAIL -> repair attempt 1 of 2
verify unit-tests ok, lint ok
open-pr pull request opened evidence: pr-ref
release awaiting human approver_role=maintainerWhat a human approves
- The org policy that sets the level and floors, by reviewing a change to
.numinance/policy/. - The pull request itself, through the human gate in the stage registry.
- Any promotion of this playbook's autonomy after its qualification record exists.
Failure and escalation paths
| Situation | Designed outcome |
|---|---|
| Verifier fails after the last repair attempt | The change escalates and waits for a human signal. It does not retry further. |
| Declared evidence missing | The step is PARTIAL, not success. |
| Plan gate denies | No side effect occurs; the denial is recorded. |
| Budget exhausted | The run pauses and escalates. |
| Breaker trips | The change is paused at the next checkpoint. See Release and incident response. |
Rehearsal is off for this playbook (rehearse: false): a dependency bump touches no migration or infrastructure, so there is no disposable environment step.
Verify what you can today
You can validate the Playbook and AutonomyPolicy examples with the open CLI.
revision: "files"
ok file "org-ladder.yaml" AutonomyPolicy "org-ladder"go run ./cli/numi validate --files schemas/examples/PlaybookNext steps
Qualify autonomy
Set per-playbook qualification thresholds, build a replay EvalSet, and understand how a promotion is proposed from evidence, accepted by a human, and never applied automatically.
Bugfix with verification
A designed walkthrough of a bug fix that ends in a pull request, showing independent verification, bounded repair, escalation and the evidence each stage emits.