Team Architecture
A deep dive into how Team is built. Skim it for the mental model, or read it in full to contribute. The high-level pipeline overview lives on the home page.
Source of truth: the artifacts in
docs/plans/<id>/*.mdand the in-session TodoWrite ledger.
Contents
- 1. Design philosophy
- 2. Artifact layout & frontmatter
- 3. Pipeline (QRSPI)
- 4. Agent roster
- 5. Phase-table orchestrator
- 6. Skills
- 7. Hooks
- 8. Behavioral evals
- 9. State management
- 10. Nested sub-agents
1. Design philosophy
Agents are decoupled microservices. Each agent consumes a predecessor
artifact on disk, does work, and produces its own artifact under
docs/plans/<id>/. The orchestrator is the main Claude Code session: it
walks a linear phase table, dispatches the right specialist for each phase,
seeds and updates a TodoWrite ledger, and runs the gates.
Principles:
- Files on disk are the durable record. Every artifact under
docs/plans/<id>/carries YAML frontmatter that describes its phase and revision metadata. Phase progression is inferred by scanning artifacts. - TodoWrite is the live coordination ledger. It is session-scoped.
Re-invoking any
/team-*command rebuilds the ledger by scanning artifacts on entry. - Registry is a phase-tagged inventory.
skills/team/registry.jsonlists the 13 specialist agents and the QRSPI phase each serves. The orchestrator dispatches through the phase table inskills/team/SKILL.md, not through the registry. - File artifacts survive compaction. Agents communicate through
files in
docs/plans/<id>/. These survive context-window compaction, can be re-read by any agent in any session, and live in git history. - Research isolation. The researcher and file-finder never receive
the user’s original task description. Enforcement is two-layer.
Structural: the orchestrator only passes the
questions.mdpath to the research agents. Procedural: the research agents’ system prompts forbid readingtask.md. - No mid-run human gates. The design (~200-line alignment doc) is
gated by an adversarial design review. A fresh-context subagent audits
it, and the orchestrator records its verdict to
design-review-<n>.md. The artifacts are thus self-describing. The Structure (~2-page vertical-slice breakdown) and the Plan are not gated. They advance autonomously. The human’s checkpoint is the PR review at the end. - Hooks enforce discipline mechanically. LLMs forget instructions ~20% of the time. Hooks are deterministic.
Trust boundary. The single human checkpoint, the end-of-run PR
review, is a code checkpoint, and it happens after the run has
already executed. Local Bash execution (implementer, verifier, hooks),
branch creation, pushes to remotes, and any multi-repo worktree effects
all occur before a human sees the PR. The PR review contains the diff,
the design’s recorded assumptions, and the deferred ## Review notes. It
does not contain the run’s local side effects. The pipeline also
assumes that repository content the researcher reads is trusted: a repo
that accepts untrusted contributions feeds untrusted text into agent
context. Merging remains human-only: nothing in the pipeline marks a PR
ready for review or merges it.
2. Artifact layout & frontmatter
Runtime canon: the schema below is carried for agents by
skills/artifact-frontmatter/SKILL.md. This section is the doc-surface copy. The executableID_RE/PHASE_FILESdefinitions live inhooks/session-start-recover.mjs.
All phase artifacts live in docs/plans/<id>/, where <id> is one of:
- Ticket-prefixed:
<TICKET>-<kebab-topic>(e.g.,ENG-1234-add-rate-limiting) - Date-prefixed:
<YYYY-MM-DD>-<kebab-topic>(e.g.,2026-05-01-add-rate-limiting)
Every artifact opens with YAML frontmatter. Common fields:
---
topic: <kebab-case>
date: 2026-04-30
phase: design # task | questions | prd | repos | research | design | structure | plan
---
Per-phase additions:
| Phase | Extra frontmatter |
|---|---|
| task | ticketId: <id> (or null) |
| questions | (none) |
| prd | (none. Not gated. The questioner writes it conditionally.) |
| repos | (none. Written conditionally in multi-repo mode.) |
| research | (none) |
| design | revision: 0 |
| structure | (none. Not gated. It advances to PLAN once it exists.) |
| plan | (none. Derived mechanically from the structure.) |
Design-review record (design-review-<n>.md): each review round the
orchestrator writes docs/plans/<id>/design-review-<n>.md (<n> 1-based
per round) with frontmatter topic, date, phase: design-review, and
verdict: <APPROVE|REQUEST CHANGES|COMMENT>. The body carries the
reviewer’s findings verbatim. The verdict field is the deterministic
read for hooks and resume detection. A design has passed review when the
highest-<n> file carries APPROVE or COMMENT.
Review loop (REQUEST CHANGES): the design-author re-drafts with the
reviewer’s findings verbatim. The orchestrator increments
revision: <n+1> in the new draft’s frontmatter. The cap is 5. At the
cap, the run halts terminally.
Phase inference (orchestrator + hooks):
| Latest artifact present | Current phase |
|---|---|
worktree exists for <id>, no task.md yet |
WORKTREE (next up) |
task.md + questions.md only |
RESEARCH (next up) |
research.md |
DESIGN (next up) |
design.md alone (no passing design review) |
DESIGN (review next) |
design.md + passing design-review-<n>.md |
STRUCTURE (next up) |
structure.md |
PLAN (next up) |
plan.md + ≥1 commit on <id> since merge-base |
IMPLEMENT |
plan.md (no commit on <id> yet) |
PLAN (next up) |
| topic branch has slice commits + verifier passed | PR (next up) |
| PR opened or commit shipped | SHIPPED |
Worktree presence: git worktree list --porcelain | grep -q <id>.
IMPLEMENT is confirmed only once there is
≥1 commit on <id> since merge-base with the default branch, so
git log <merge-base>..<id> is non-empty. A present plan.md with no
commit means the run is still pre-IMPLEMENT.
One non-phase sibling output exists: docs/plans/<id>/screenshots/ (PNGs
plus manifest.md), written by ux-reviewer during IMPLEMENT for UI-touching
changes and consumed by team-pr. Discovery keys only on the six PHASE_FILES
names, so this directory is invisible to hooks and skills, and IMPLEMENT still
declares no phase artifact. User-facing setup (the one-time GitHub sign-in
that enables inline upload) is documented in the README’s
“Screenshots in PRs” section.
3. Pipeline (QRSPI)
The pipeline has eight phases. The orchestrator walks them in order. Each phase needs the prior phase’s artifact on disk.
WORKTREE → QUESTION → RESEARCH → DESIGN → STRUCTURE → PLAN → IMPLEMENT → PR → SHIPPED
Phase 1: Worktree
Action: orchestrator-emit (the leading phase) Predecessor: none. The description arrives in ``.
Before QUESTION, the orchestrator creates the home worktree on branch
<id> off origin/HEAD using Claude Code’s native worktree support, and
authors docs/plans/<id>/ inside it. Because the artifact directory
is born in the worktree, no copy is ever needed and the home checkout’s
git status stays clean for the whole run. The orchestrator computes the
worktree’s absolute path once and threads it into every downstream
dispatch (the main session does not cd).
Single-repo (default): repos.md is absent. One home worktree at
<repo>/.claude/worktrees/<id>.
Multi-repo: the questioner writes repos.md autonomously at
QUESTION, one phase after WORKTREE. The design-author writes it at DESIGN
when the questioner missed the multi-repo signals. Only the home worktree
is created here. Secondary worktrees are created
after the design review, one per listed repo. Each repo path must
first pass the containment check: its realpath must be a direct child of
the home repo’s parent directory.
git -C <repo-path> worktree add .claude/worktrees/<id> -b <id> origin/HEAD
At that point the orchestrator writes a ## Worktrees section to
repos.md. It back-records the home worktree path plus each secondary
path. Any later /team-* invocation can thus rediscover all paths from
one file. Only the home repo’s worktree carries docs/plans/<id>/. Other
repos’ worktrees do not duplicate the artifacts. See
skills/worktree-isolation/SKILL.md for full topology.
Fallback: if home-worktree creation fails (shallow clone, certain CI
systems, permissions), the orchestrator reports it and falls back to
in-place for the entire run: docs/plans/<id>/ lives at the home-repo
root and the threaded path is the home-repo root.
Phase 2: Question
Agent: questioner
Predecessor: worktree prepared (+ description in $ARGUMENTS)
Artifacts: docs/plans/<id>/{task,questions}.md
Decomposes the user’s intent into two artifacts. Only the questioner ever
sees the user’s description. There is no separate brief.md. Neutral
codebase context lives in a “Codebase context” section at the top of
questions.md.
Phase 3: Research
Agents: file-finder and researcher (parallel, isolated)
Predecessor: questions.md (orchestrator passes only the
questions.md path to the research agents)
Artifact: docs/plans/<id>/research.md
Orchestrator waits for both agents to return, then writes the combined research artifact with the necessary frontmatter.
Phase 4: Design
Agent: design-author (resolves its own open questions, recording
each as an auditable assumption) Predecessor: research.md
Artifact: docs/plans/<id>/design.md Gate: REVIEW. The
orchestrator dispatches a fresh-context, read-only Explore subagent
with the ## Review brief from skills/eng-design-doc-review/SKILL.md.
The subagent holds no Write or Edit tools, so the reviewer cannot touch
the artifacts it judges. The orchestrator records the verdict to
design-review-<n>.md. APPROVE and COMMENT advance. On REQUEST CHANGES
the agent re-drafts with the findings verbatim and increments revision.
At cap 5 the run halts terminally.
Phase 5: Structure
Agent: structure-planner
Predecessor: design.md + passing design-review-<n>.md
Artifact: docs/plans/<id>/structure.md
Gate: NONE (autonomous). Once structure.md exists the pipeline
advances to PLAN.
Phase 6: Plan
Agent: planner
Predecessor: structure.md
Artifact: docs/plans/<id>/plan.md
No gate. The plan is mechanically derived from the structure.
Phase 7: Implement
Sub-pipeline:
- Test-first.
test-architectwrites failing acceptance tests. - Mechanical gate. The orchestrator runs the suite. All tests must fail with assertion errors, not crashes.
- Slice execution.
implementerworks through the plan one slice at a time, committing each atomically. - Code review. 5 reviewers in parallel:
code-reviewer,security-reviewer,technical-writer,ux-reviewer,verifier. - Aggregate gate. The orchestrator sorts every finding into a
severity tier: Blocking, Major, or Minor-and-below. See
skills/review-severity-tiers/SKILL.md. While any Blocking or Major finding remains, it dispatches the implementer to fix the typed failure class. It then re-runs all 5 reviewers automatically. It never consults the user. This is the no-consult rule. The cap is 5 rounds. At the cap, the run halts terminally. Once Blocking and Major are clean, any remaining Minor-and-below findings are recorded in the PR body’s## Review notesfor the human’s PR review.
The orchestrator tracks the round count by appending “Review round N” items to the TodoWrite ledger.
Recovery after a terminal halt (either cap): a human addresses the
unresolved findings by hand (editing the design or the code) and
re-invokes the same /team-* command bare. The aggregate round counter
is session-scoped through TodoWrite and starts fresh on re-invocation.
The design revision counter persists in design.md frontmatter. Lower
it by hand to restore the revision budget.
Phase 8: PR
Action: orchestrator-emit Predecessor: aggregate gate passed
Update CHANGELOG.md (filter for user-facing commits since last release),
push the branch, and open a draft PR automatically with
gh pr create --draft. The PR phase never waits for approval. Then
surface the tracking ticket, if task.md carries ticketId. When a
capture manifest exists (docs/plans/<id>/screenshots/, see the
artifact-layout note in section 2), the PR body also gets a
## Screenshots section populated by uploading the PNGs through GitHub’s
user-attachments pipeline. The worktree stays in place after the PR
opens. Teardown is deferred until the PR merges or the user asks, so the
branch remains available for iteration. Completion points at the
standalone /pr-watch utility for watching the PR once it is ready for
review.
4. Agent roster
13 specialist agents, organized by phase:
| Phase | Agents |
|---|---|
| QUESTION | questioner |
| RESEARCH | file-finder, researcher (parallel) |
| DESIGN | design-author |
| STRUCTURE | structure-planner |
| PLAN | planner |
| IMPLEMENT | test-architect, implementer, code-reviewer, security-reviewer, |
technical-writer, ux-reviewer, verifier (last 5 parallel) |
Each agent’s QRSPI phase is recorded in skills/team/registry.json.
Agent frontmatter uses only Claude Code’s supported fields.
The dev hook .claude/hooks/check-registry-sync.mjs validates that
the inventory in registry.json and the files under agents/ agree by
name.
Checks and balances
The roster splits into two roles with deliberately non-overlapping powers. Producers write artifacts and code. Reviewers cast verdicts. No agent holds both powers, and neither role can complete a review cycle alone.
| Role | Agents | Tools | permissionMode |
Power |
|---|---|---|---|---|
| Producer | questioner, design-author, structure-planner, planner, test-architect, implementer |
include Write/Edit |
acceptEdits |
Changes the tree. Casts no verdict. |
| Reviewer | code-reviewer, security-reviewer, technical-writer, ux-reviewer, verifier |
read-only | plan |
Blocks the line. Changes nothing. |
This is enforced by frontmatter, not by prompt text. A reviewer that could
edit could fix the defect it found and then approve its own fix, which collapses
the generator and the evaluator into one role. Read-only tool grants plus
permissionMode: plan make that impossible at the harness layer.
tests/protocol.test.ts pins both halves as an L2 tripwire, so a new reviewer
that ships with Write fails CI.
researcher and file-finder are also read-only, for a different reason:
research isolation (see Phase 3). Read-only does not by
itself mean reviewer.
Three more balances bound the checks themselves:
- The veto is bounded. The review loop is capped at 5 rounds, then halts to a
human with the unresolved findings. See
skills/review-severity-tiers/SKILL.md. - The check has a check. The optional skeptic pass is default-keep. An inconclusive refutation leaves the finding standing, so the pass removes false positives only.
- The orchestrator cannot punt. The no-consult rule forbids escalation of a Blocking or Major finding to the user mid-run.
Model tiering
The principle:
complex work runs on the most capable available model. Bounded judgment runs on sonnet. Mechanical checks run on haiku.
The most capable model is fable (Fable 5). It was temporarily suspended
for all customers under a U.S. government export-control directive (see
Anthropic’s notice).
During the suspension, complex work ran on opus (Opus 4.8), Fable’s
documented fallback target. Access has been restored, and the
complex-work agents now run on fable.
fable(complex work):researcher,design-author,structure-planner,planner,test-architect,implementer, andcode-reviewer.opus(security review):security-reviewerstays onopuspermanently, not as a fallback. Fable 5’s cybersecurity safety classifiers flag security-review content. In a non-interactive subagent context, a flagged request ends the turn with a refusal instead of a fallback.sonnet(bounded single-pass judgment):questioner,ux-reviewer,technical-writer.haiku(mechanical checks):file-finder,verifier.
Notes:
- What the
fableagents require of plugin users: Claude Code ≥ v2.1.170 and Fable access. Fable 5 is not available under zero data retention (30-day retention is necessary), and on Bedrock/Vertex/FoundryANTHROPIC_DEFAULT_FABLE_MODELmust be pinned. Users without access must override toopus. See the override note below.opusremains the documented fallback tier for everyfableagent. - 1M context window comes for free at the fable and opus tiers. Fable
5’s context window is 1M by default, and Opus 4.8 always runs with the
1M window on the Anthropic API. Max, Team, and Enterprise plans include
the 1M upgrade with the subscription, and Pro degrades gracefully to
200K. These agents thus need no
[1m]suffix. The sonnet agents stay at 200K, because their bounded single-pass work is nowhere near the ceiling. Haiku does not support 1M. -
Users can override any agent’s model with
CLAUDE_CODE_SUBAGENT_MODEL(applies to all subagents), or copy an agent file into.claude/agents/with a differentmodel:. - Four agents (
researcher,implementer,code-reviewer,security-reviewer) additionally hold theAgenttool and may spawn read-only nested sub-agents. See 10. Nested sub-agents.
Effort tiering
Effort tiering mirrors the model tiers: low (mechanical), medium and
high (judgment), and xhigh (strategic artifact authors). The xhigh
tier holds design-author and structure-planner, whose artifacts set
the direction everything downstream inherits. Methodology skills carry no
effort. They inherit it from the loading agent.
5. Phase-table orchestrator
The orchestrator (the main Claude Code session) drives /team by
walking the phase table in skills/team/SKILL.md. Pseudocode:
setup:
resolve $ARGUMENTS (issue URL → gh issue view; ticket → use as prefix).
derive <id> = "<TICKET>-<topic>" or "<YYYY-MM-DD>-<topic>".
create docs/plans/<id>/ if needed.
seed TodoWrite ledger with one item per phase.
on resume: scan docs/plans/<id>/, fast-forward the ledger.
loop:
1. Inspect TodoWrite. If all phases completed → exit.
2. Identify the in_progress phase. Look up agent(s) and predecessor
artifact path(s) in the phase table.
3. Make sure that predecessors exist on disk. For STRUCTURE that includes
a `design-review-<n>.md` with a passing verdict. If one is missing, the
run is desynced. Suggest the bare /team-* command again. Its three-tier
discovery resolves docs/plans/<id>/ without an explicit arg.
4. Dispatch the agent(s) — pass them the artifact directory
`docs/plans/<id>/`.
5. Write returned artifacts to docs/plans/<id>/<name>.md with the
YAML frontmatter the agent specifies.
6. Run the gate for this phase (HUMAN | MECHANICAL | ORCHESTRATOR-EMIT
| AGGREGATE).
7. Mark current TodoWrite item complete and the next one in_progress.
8. Goto loop.
6. Skills
Skills live under skills/. There are two flavors:
Entry-point skills (slash commands)
Every entry-point skill carries an argument-hint field in its
frontmatter (Claude Code skills frontmatter)
that documents the expected $ARGUMENTS shape.
Each downstream skill (team-research and beyond) treats $ARGUMENTS as
an artifact directory, typically the path printed by the previous phase’s
completion message. For the 8 directory-consuming skills
(team-research, team-design, team-structure, team-plan,
team-worktree, team-implement, team-pr, eng-design-doc-review)
the docs/plans/<id>/ argument is optional. Each skill resolves the
directory through a three-tier chain: explicit $ARGUMENTS →
newest-mtime convention discovery → AskUserQuestion. The middle tier
filters by ID_RE and PHASE_FILES, ported from
hooks/session-start-recover.mjs as a POSIX ERE translation, and by the
skill’s necessary predecessor artifact. The orchestrator or entry-point
skill calls AskUserQuestion itself, never a subagent. Subagents never
pause for user input. Standalone modes still exist. A partial skill
invoked with no resolvable directory, or with a free-form description,
bootstraps the missing upstream artifacts inline rather than hard-error.
Discovery duplication: design rationale. Each archetype-A skill
embeds the three-tier resolver as a single self-contained bash block
rather than calling a shared script. Agent threads reset their cwd
between Bash calls, so the block cannot rely on any shared shell state
and must stand alone in one invocation. The ~6 load-bearing lines
(ID_RE, PHASE_FILES, root literal, and predecessor filter) are thus
duplicated verbatim across the 8 directory-consuming skills by deliberate
decision. No shared runtime helper was added, and the
check-discovery-consistency.sh gate enforces byte-identity, so the
copies cannot drift. A shared discover-topic.sh could also dedup the
two hooks’ findActiveTopic. That is a recorded future consolidation.
The # NOTE: ... future: shared discover-topic.sh comment in each block
points at it.
Methodology skills (loaded by agents, not directly invoked)
Methodology skills carry no argument-hint. Agents load them through one
of two mechanisms. The first is a skills: YAML list in the agent’s
frontmatter. For example, agents/design-author.md declares
skills: [product-thinking, progress-tracking, authoring-designs,
writing-prose]. Those four names are three countable skills under the
load limit in the Design guidelines below: authoring-designs is
design-author’s own extracted procedure skill, which that limit exempts.
The second is an inline prose load instruction in the agent body. For
example, the implementer body’s Code quality section loads
solid-principles inline.
Because they are reference material rather than user actions, methodology
skills set user-invocable: false in their frontmatter. This keeps them
out of the / slash-command menu, because a /qrspi-workflow command is
meaningless to a user. They stay fully loadable by their two mechanisms
above. Neither the skills: preload nor a by-path load is affected by
the field, which governs only menu visibility. The model can still
auto-load a methodology skill when relevant, so
disable-model-invocation is deliberately not set. When you add a
new methodology skill, set user-invocable: false. When you add a new
entry-point skill, leave it unset, so it registers as a slash command.
The trigger-phrase convention keys on the user-invocable field —
not on argument-hint, which docs/skills.md uses to sort skill
flavor. A skill that does not set user-invocable: false must state,
in its description, at least one double-quoted natural-language phrase
(one that does not start with /) plus its own literal /<name>,
normally as a final Trigger on "…", "…", or "/<name>". sentence. A
side-effecting or irreversible entry-point skill — one that commits,
pushes, opens a PR, moves a ticket, merges, deploys, or deletes — MUST
replace the plain carrier with shipit-style explicit-intent guard wording
(“Invoke ONLY on explicit … intent — … never infer …”), still carrying
the quoted phrases and the slash name, and should set
disable-model-invocation where the host honors it. Such a skill stays
listed as a command, but its routing-map line in AGENTS.md states the
explicit intent, so the map never invites it on a plain request. A deterministic test
in tests/architecture.test.ts enforces the phrase invariant with no
opt-out; the slash-name check is prefix-safe, so /team-research cannot
satisfy the /team requirement. The guard wording is NOT
machine-checked — it is the author’s and reviewer’s responsibility, and
its absence on a side-effecting skill is a review-blocking defect.
Among methodology skills, code-review is the only one kept
user-invocable. It is a building block: the code-reviewer,
security-reviewer, ux-reviewer, and technical-writer agents load it
as working methodology. It is also a meaningful standalone user
action, “review this diff”. The field thus stays unset. The distinction
is the primary surface: a skill earns a slash command when a user would
plausibly run it directly, even if agents also compose it.
That standalone path behaves differently from an ordinary entry point,
which simply runs its own procedure. A methodology skill invoked directly
still owes its own rules: the main session shares conversation history
with whatever wrote the code, so it is not a valid reviewer. code-review
therefore dispatches the code-reviewer agent and relays the verdict
rather than reviewing inline.
(This is separate from the entry-point skills, which are user-invocable by
definition. Some of those, e.g. team-worktree and team-pr, are also
referenced by path from team/SKILL.md, but those are procedural
cross-links in the orchestrator’s prose, not a parent loading the skill as
a building block. code-review is the only skill loaded as composed
methodology that is also a user command.)
For the full per-skill reference (all 53 skills, their arguments, consumers, and behaviors), see skills.md.
Design guidelines
-
Methodology skill load limit: Soft limit of 3 methodology skills per agent invocation. A skill averages roughly 190 lines, so 3 skills add roughly 570 lines (~10K tokens, about 5% of 200K context). A fourth skill signals that the agent’s responsibility can be too broad. This is a design convention, not a hard constraint. An agent’s own extracted procedure skill does not count toward the soft limit: it replaces former inline body content 1:1, so it adds no net context.
-
Extraction threshold: capability against fragment. Content that is an independently useful capability earns its own skill regardless of consumer count. Such content is coherent and self-contained, and the model or a future consumer can plausibly load it on demand. Claude Code preloads only each skill’s name and description. The body loads just-in-time when invoked. An unused small skill thus costs almost nothing. Content embedded into a consumer instead forecloses just-in-time loading for everyone else. Content that is only meaningful inside one consumer’s procedure, a procedure fragment, stays inline in that consumer.
7. Hooks
Runtime hooks (hooks/, distributed with the plugin):
| Hook | Event | Purpose |
|---|---|---|
pre-bash-guard.mjs |
PreToolUse(Bash) | Block dangerous shell commands |
pre-compact-anchor.mjs |
PreCompact | Scan docs/plans/ |
session-start-recover.mjs |
SessionStart | Scan docs/plans/ |
post-write-validate.mjs |
PostToolUse(Write|Edit) | Structural validation of plugin component files |
Both pre-compact-anchor.mjs and session-start-recover.mjs work the
same way. They list docs/plans/*/ directories. They pick the most
recent artifact directory by the mtime of any contained artifact. They
infer the current phase from artifact presence and frontmatter. They then
emit a short context message that names the phase, <id>, and the
suggested next /team-* command. Both are stateless, exit 0 on any
error, and return within the 5000ms hook budget.
Development hook (.claude/hooks/, not distributed):
| Hook | Event | Purpose |
|---|---|---|
check-registry-sync.mjs |
PostToolUse(Write|Edit) | Verify the agents/ directory and registry.json agree by agent name |
Development scripts (.claude/scripts/, not distributed) house dev-only
acceptance tooling run by plugin developers. check-discovery-consistency.sh
is the committed consistency gate for the input-discovery feature: it asserts
every archetype-A skill carries the discovery block, the load-bearing fragments
(ID_RE, PHASE_FILES, docs/plans/ root) stay byte-identical
to canon, and the research-isolation invariant holds.
8. Behavioral evals
The behavioral regression harness defends pipeline agents against silent
behavior drift across model upgrades. The implementation is TypeScript +
Bun: harness code in tests/, fixtures/rubrics/results in evals/.
Plugin-developer tooling, not distributed with the plugin. Three tiers:
- Gate (free):
bun test. Static schema validation on every fixture and rubric plus unit tests for the harness helpers. No model calls, noEVALS_ANTHROPIC_API_KEY. Runs in CI on every PR. - E2E (paid):
bun run test:evals(needsEVALS_ANTHROPIC_API_KEY). Spawnsclaude -p --output-format stream-jsonagainst a fixture, parses the NDJSON transcript, persists a per-case result JSON with timing axes (firstResponseMs,maxInterTurnMs). - LLM-judge (paid): deterministic-first regex / ground-truth checks
run cheap and gate the LLM call. Haiku for narrow rubrics, Sonnet for
nuanced ones. Untrusted agent output is wrapped in
<<<UNTRUSTED_OUTPUT>>>delimiters before reaching the judge.
EvalCollector writes incrementally. It finds the previous run on the
same branch and tier. It prints regressions and budget regressions. A
budget regression is ≥2× growth in tool calls or turns without a verdict
change. See
evals/README.md
for fixture schema, rubric format, env-var knobs, and the rerun-on-base
blame protocol.
CI wiring. Two GitHub Actions workflows in .github/workflows/:
harness-checks.yml runs the offline harness validation on every PR (no
secrets, ~7s). periodic-evals.yml runs the live-agent regression check
on a weekly cron (Monday 06:00 UTC) with EVALS_ANTHROPIC_API_KEY. That
key is an environment secret scoped to the evals GitHub
Environment. It needs a one-time Settings → Environments setup: create
the evals environment and attach the secret. You can also set necessary
reviewers or a main-only branch restriction. If the PR-eval workflow
(pr-evals.yml) is active, the evals environment must not restrict
deployment branches to main. PR head/merge refs are not main, so a
main-only policy blocks the secret on PRs, and every gated eval fails at
the deploy gate. Remove it through Settings → Environments → evals →
Deployment branches → “No restriction”, or
gh api -X DELETE repos/bostonaholic/team/environments/evals/deployment-branch-policies/<id>.
The job declares environment: evals and fails closed if the
environment is absent: the secret simply resolves to empty, so no token
spend leaks. pull_request_target is hard-banned for this and any
secret-consuming / claude-spawning job, because it runs in base-repo
context with secrets available, a base-repo-context exfiltration vector.
The ban is enforced by a static tripwire in tests/static-gate.test.ts.
Live jobs that run on pull_request events additionally gate on
PR-author trust: only OWNER/MEMBER/COLLABORATOR authors may spend tokens,
so untrusted PRs (forks, Dependabot, first-time contributors) never
trigger paid execution.
These three tiers are the paid frontier of Team’s broader six-layer testing model. See Testing for where every check belongs.
9. State management
Primary state: the artifacts in docs/plans/<id>/*.md. Each
artifact’s YAML frontmatter is the source of truth for “did this phase
finish?” and “did the design review pass?”
Live coordination: TodoWrite (session-scoped). The orchestrator
seeds the ledger at the start of /team, marks each item in_progress
when dispatching, and completed when the artifact lands. Any /team-*
command rebuilds the ledger by scanning artifacts on entry, so an
interrupted run can be resumed by re-invoking any of them bare: discovery
auto-resolves the artifact directory (an explicit docs/plans/<id>/ is still
accepted).
Design-review records: docs/plans/<id>/design-review-<n>.md
(frontmatter verdict: <APPROVE|REQUEST CHANGES|COMMENT>) records each
review round durably. The artifacts are self-describing.
Compaction defense: the PreCompact hook scans docs/plans/<id>/
directories for the active topic and injects a 4-line anchor (phase,
<id>, suggested next /team-* command). The SessionStart hook does
the same for new sessions.
Artifact persistence: during a run, files in docs/plans/<id>/ live
on disk in the worktree’s working tree and
the pipeline never commits them. That discipline is what lets the
commit-based IMPLEMENT signal (≥1 commit on <id> since the
default-branch merge-base) distinguish “implementation has begun” from
“artifacts merely exist”. Any commit on <id> is thus a code or slice
commit, never an artifact commit. A finished plan directory can land
later in its feature PR, which is outside the run. Persistence across
sessions, compaction events, and context resets comes from the survival
of the worktree’s files, not from git history. They remain the durable
communication protocol between agents and the source of truth for “did
phase N finish?”
10. Nested sub-agents
Claude Code v2.1.172 lets sub-agents spawn their own sub-agents (up to
5 levels deep). The plugin uses this capability in exactly two patterns,
both governed by skills/nested-agents/SKILL.md:
- Context-economy scouts (
researcher,implementer): read-onlyExplore/team:file-finderhelpers that absorb bulk reading the parent would otherwise hold in its own context without ever referencing it again. The researcher’s scouts inherit the research isolation invariant: scout prompts are built only from verbatimquestions.mdtext andrepos.mdpaths. - Skeptic verification (
code-reviewer,security-reviewer): each hard-gate finding (Blocking, CRITICAL, or HIGH) goes to a freshgeneral-purposesub-agent as a neutral, falsifiable claim, with instructions to refute it. The claim never carries the reviewer’s verdict or severity. Default-keep: a finding is dropped only on an evidence-backed refutation the reviewer verifies itself. A false hard-gate finding costs an entire review round (implementer re-dispatch- all 5 reviewers re-run), so the pass pays for itself.
Policy:
- The
Agenttool is granted to exactly four agents:researcher,implementer,code-reviewer,security-reviewer. The allowlist is pinned bytests/nested-agents.test.ts, so any other agent gaining the tool must be a deliberate decision that updates the tripwire. - Nested helpers are read-only, never write under
docs/plans/, and never pause for user input (seeskills/nested-agents/SKILL.md). - Depth budget: pipeline agents sit at depth 2 of 5 and may spawn at most one more level.
- Version-gated. Nesting requires Claude Code >= 2.1.172. The
universal gate is
Agent-tool presence. The platform withholds the tool from sub-agents below that floor. An agent that lacksAgentthus works inline. So does any read-only agent likeresearcherthat has noBash. Agents that also holdBashadditionally run the bundled deterministic checkskills/nested-agents/supports-nesting.mjs "$(claude --version)". Its pure comparison core is unit-tested at L1.tests/nested-agents.test.tspins the skill contract and the version floor. The check is fail-closed: an older release, unrecognizable version output, or an environment where it cannot run all resolve to “unsupported,” which routes the agent to its inline path. - Optimization, never a dependency. Every nested-dispatch section is optional with a mandatory inline fallback. On Claude Code versions without nesting, the plugin degrades to exactly its previous behavior. Nesting is invisible to the orchestrator: no phase-table, gate, or artifact contract depends on it.
Future work (ship-later)
Recorded here so the next capability step is deliberate, not improvised. Prerequisite for all items: nesting survives 2-3 Claude Code releases without breaking changes, parallel dispatch semantics for nested children are confirmed, and the depth cap is stable.
- Verify-coordinator (strongest candidate): an IMPLEMENT-phase
coordinator that owns the 5-reviewer fan-out, the aggregate gate, and
the typed failure retry loop. It returns a compact terminal verdict for
the orchestrator to render:
{verdict: PASS | CONDITIONAL | ESCALATE, rounds, findings[]}. This keeps up to 25 reviewer reports out of the orchestrator’s long-lived context. Needs: a terminal-verdict envelope protocol, because the no-consult rule means no mid-loop user interaction exists to forward. It also needs per-round state artifacts (docs/plans/<id>/review/round-<n>.md) to replace live TodoWrite round visibility and to survive coordinator death. - Research-coordinator: only if multi-repo research outgrows the current 2-agent fan-out (e.g. 2×N agents across N repos).
- Parallel sub-implementers: only ever slice-level (never step-level), one worktree per slice, coordinator cherry-picks sequentially to preserve linear history. Requires measured wall-clock pain on large plans.