Spec-to-Production Pipeline

This trace separates the intended solo-dev policy from the transitions actually implemented by repository workflows. Zero human PR reviews through XL does not mean zero intake authorization, zero required checks, or automatic release-tag creation.

Source baseline: 2dde3774281e75b083984ea970d5d08723cc7179, audited 2026-10-06. Live observations below are dated snapshots, not consumer defaults. Workflow filenames refer to the source repository; downstream installers may rename them or omit source-only release jobs.

Re-audited 2026-10-24 against 5d1464ee (#3927). Table rows below reflect that revision; dated measurement sections keep their original baseline. See Lifecycle audit 2026-10-24 for current stage status and blockers.

State transitions and owners

Stage	Trigger and owner	Output and next transition	Blocking conditions
Intake	Issue opened; issue-triage.lock.yml executes issue-triage.md	Type/priority labels and one triage comment; incomplete PRD/spec evidence requests synthesis	Poor quality adds needs-info; duplicate/type conflicts need normalization
Field sync	Open/reopen/label changes; issue-field-sync.yml	Mirrors labels into native Type/Priority fields	Token/org-field API access; field sync is not implementation authorization
Dependency routing	Issue/PR/comment changes; dependency-relationship-routing.yml	Parses body/comments; issues get blocked, PRs dependency-blocked	Open issue dependencies or unmerged PR dependencies
Spec fallback	synthesize-spec label, successful Issue Triage Agent completion, or manual dispatch; issue-to-spec-synthesis.yml	Creates draft PRD/spec on spec/issue-N-slug, adds documentation/spec, copies a sprint label if present, clears intake labels; an existing branch that conflicts with generated content is preserved and the issue gets one comment plus synthesis-conflict	Draft remains draft; synthesis does not approve the issue, assign implementation, or mark the PR ready
Approve supplied spec	Qualified /approve issue comment; issue-approve.yml	Checks metadata/dependencies/spec, adds approved/copilot-agent, attempts assignment, then reevaluates linked PRs	Missing type/priority/spec, blocker labels, unresolved dependencies, permission or agent availability
Approved-spec handoff	Qualified exact /approve-spec <spec-pr> <head-sha> source-issue comment; approved-spec-handoff.yml with .github/base-coat/scripts/spec-handoff.cjs	Resolver reports spec-draft, blocked, authorization-invalid, assigned or handoff-pending with a stable source/directive/spec identity; the default read-only mode makes no assignment write, and canary/general dispatch agent-assignment-worker.yml only through the gates described below	Spec PR not merged before the directive, approved head changed, issue Spec URL not commit-pinned to the merged spec, spec/PRD not reviewed or implementation-ready, blobs changed on the default branch, Delivery intent not exactly implementation, hold labels, dependencies (including dependency comments), source edits after the directive, or unreadable GitHub evidence
Implement	Assigned Copilot coding agent or explicitly authorized local delivery execution	Ready implementation PR with Closes #N, valid intake contract, spec and validation evidence	The coding agent must be enabled for the repository (#3826); agent-assignment-worker.yml serializes assignment per issue and verifies assignee read-back; reconciling failed assignments is tracked in #3592
Label/readiness	PR open/sync/reopen/ready; pr-size-labeler.yml; metadata changes also drive pr-flow-hygiene.yml	Deterministic size and same-repo sprint labeling; hygiene reports readiness/ownership issues	Hygiene does not turn draft spec PRs into ready implementation PRs
Intake metadata	PR open/edit/sync/reopen/ready; pr-intake-metadata.yml	Authoring-time check of the PR body Design fields that eligibility enforces later	Missing or invalid Change scope (individual or batch), unit inventory, or size fields; read-only, never writes to the PR
Conflict routing	Every 30 minutes, push to main, manual; pr-conflict-routing.yml	Labels conflicted PRs needs-conflict-fix and posts one trusted instruction comment per head	Routing requests a fix; it does not rebase or resolve conflicts itself
Validate	PR head checks; CI, Validate BaseCoat, PR Validation, Agent Merge and spec/harness gates	Current-head results; required-workflow completion routes executor reevaluation	Failed/missing checks, execution approval holds, stale head, invalid intake
Eligibility	PR events, explicit dispatch, required-workflow completion; pr-auto-merge-executor.yml	Reads trusted policy, checks issue/spec/check evidence, publishes eligibility and revalidates snapshot	Draft, delivery hold, unresolved dependency, missing authorization, XXL approval, stale snapshot, unresolved review threads (#3795); metadata drift reports pending rather than failure
Merge queue	Eligible enqueue request; native GitHub queue	Serialized exact integration-tree validation, then squash merge	Queue checks failing or missing; queue command incompatibility; new base/head
Post-merge	Merged PR event or hourly backfill; post-merge-release-chain.yml	Dispatches evidence-only release gate and source-only packaging; writes marker comment	Dispatch/API failure; comment proves dispatch, not completed production deployment
Release preparation	Authorized release execution; optional release-train.yml report	Merge committed version/changelog preparation and create intended vX.Y.Z tag	No automatic tag creation in release train or post-merge chain; tag must match committed version
Tag fan-out	Tag push; release.yml, package-basecoat.yml, publish-to-production.yml	Internal release/assets, validated packages, sanitized public mirror	Version mismatch, label coverage, validation, token preflight, protected environment
Production verification	Successful publish plus docs-production.yml	Public tag/payload and documentation dispatch evidence	Internal release alone is not proof of mirror/docs delivery
Closeout	Implementation PR merge with closing keywords; explicit delivery verification	GitHub closes linked issue at merge; operator records release/tag/production evidence	Issue closure does not imply production completion; public cleanup workflow is separate manual maintenance

Synthesis validation update (#3660): source sections are extracted outside fenced blocks and HTML comments. Canonical Problem/Scope/Why headings take priority over Description and Intake Contract/RCA/Design aliases. The fallback preserves the complete source instead of truncating at eight lines; unclosed fences and HTML comments fail explicitly. Both generated artifacts must pass markdownlint-cli@0.39.0 with .markdownlint.json fetched at the pinned main revision before any branch is created. Invalid Markdown (including unsupported bare URLs or inline markdownlint directives from source text) blocks publication without exposing raw source text in diagnostics. Orphaned-branch recovery requires both stored artifacts to match generated content; conflicting edits are preserved, not overwritten. These checks do not approve incomplete specifications or promote their draft PRs.

Approved-spec handoff contract (#3593): the synthesized spec PR must merge first. Only a fresh qualified /approve-spec naming the merged PR and its final head binds the source issue, whose single - Spec: URL must be the commit-pinned blob of that spec. Later hold-label changes or body edits make the directive stale. Assignment writes stay disabled until a trusted single-issue canary is configured.

Handoff wiring (#3830): approved-spec-handoff.yml routes each comment mentioning /approve-spec, merged same-repository spec PRs, or dispatch to the source issue, serializes per issue, reconciles the newest qualified directive on every run (the lock keeps only one pending run), and upserts one bot-owned basecoat-spec-handoff status comment. A merged PR is a spec candidate only when every changed path, including both paths of a rename, is under docs/ and its visible body does not declare Delivery intent: implementation. Any changed path outside docs/ makes it an implementation PR, even if it also edits a spec; any explicit implementation declaration has the same effect, including when the field is duplicated. These implementation PRs do not produce awaiting-directive or request /approve-spec. BASECOAT_SPEC_HANDOFF_MODE defaults to read-only; canary writes only when the resolved record equals the BASECOAT_SPEC_HANDOFF_CANARY JSON tuple (issue, directiveId, specPr, approvedHead, mergeCommit, specBlob, prdBlob); general also needs persisted canary assignee read-back. Only an administrator can change these variables. Writes go through agent-assignment-worker.yml, which re-resolves the directive at each boundary, reuses its epoch and budget, records gate denials as blocked without revoking the directive, and refreshes the status after assignment read-back. Live use is blocked until the coding agent is enabled (#3826) and a genuine canary is configured.

Labels, directives and tags

For variable locations, canary field values, activation and rollback, see the delivery automation settings guide.

Signal	Meaning and authority
bug, enhancement, documentation, chore, security, question	Canonical type; triage and approval metadata
priority:critical/high/medium/low	Planning priority, not approval or bypass
needs-triage, needs-info, needs-prd, synthesize-spec	Intake states; synthesis clears the latter three after producing or finding a spec PR
synthesis-conflict	Terminal synthesis state: existing spec branch conflicts with generated content; synthesis skips the issue until the branch is reconciled and the label removed
duplicate, invalid, wontfix	Block issue approval; classification is not authorization
blocked, dependency-blocked	Dependency routing state; authoritative dependency checks also read relationships
approved, copilot-agent	Accepted issue directive and assignment intent; verify persisted assignee separately
documentation, spec on a synthesis PR	Draft artifact classification, not implementation approval
size:XS/S/M/L/XL/XXL	Changed-line boundaries: 20/100/300/800/2000/greater than 2000; generated lockfile churn excluded; XXL requires qualified approval
wave:*, sprint:*, wave-*, sprint-*, wave/sprint	Release grouping; labeler supplies sprint:YYYY-Www to same-repo PRs lacking coverage
skip-release-label-gate, dependencies	Supported release-label exemptions; the PR, merge-group and tag release coverage gates share one classifier (scripts/merge-group-release-labels.cjs), so exempt PRs count as covered (release fails only above 10% uncovered)
delivery-hold	Explicit pause; executor disables/refuses auto-merge until removed
skip-prd-spec-check	Spec-gate exception only; does not satisfy executor authorization
remediation, escalated, automation	Watchdog escalation tracking; does not repair or approve automatically
watchdog:ignore, sla:exempt	Suppress watchdog alerts, not required delivery checks
intent-control-plane, ship-it, spec-2-prod, risk-*	Governed intent artifacts; not substitutes for source approval
vX.Y.Z	Release version tag; must identify a commit already carrying the matching version.json

Approval contract update (2026-10-07, #3591): post exact /approve on the source issue or a PR that closes it, with a real HTTP(S) - Spec: URL on the issue. All three approval/delivery routes now share current write/maintain/admin authority, spec validation, metadata, and live dependency checks. Case and surrounding whitespace are accepted; quoted, incidental, or suffixed commands are not. An approved label is not authority. Other audit snapshots in this guide retain their dated baseline.

/spec-2-prod is handled by ship-it-intent-dispatch.yml, not by issue-approve.yml. It requires an already approved open source issue, matching spec, and qualified approval evidence. It does not apply approved by itself. Governed intent generation and a delivery control loop are distinct from the issue-assignment workflow; generating phase issues is not proof they executed.

PR-side forwarding records the original qualified human directive, actor, and comment URL. Delivery re-fetches that comment and current permissions and verifies the PR still closes the issue; the forwarding bot cannot approve. For migration from incidental commands, relative specs, or older bot-only acknowledgements, see the shared approval contract.

The PR release-label gate accepts the two exemptions above. Every tag publisher (release.yml, the package-basecoat.yml release job and the publish-to-production.yml publish job) runs scripts/release-label-gate.sh before publishing, so a failed coverage decision stops all of them; the gate uses the same classifier, fails closed and creates no refs.

Required checks versus policy intent

Solo-dev policy lists six main checks: lint-and-validate, test, validate-commit-messages, validate-unix, validate-windows, and release-label-gate. Cloud-agent policy adds Agent merge guardrails. The executor emits BaseCoat merge eligibility; intended onboarding also requires that context with Actions integration ID 15368.

Live source rules on 2026-10-24 require eight contexts: the seven above plus validate-workflow-syntax (#3576). They still omit BaseCoat merge eligibility, which the executor reports only on PR heads, not merge-group commits, so it is enforced before enqueue rather than by the queue. A green native ruleset is therefore still not equivalent to the full documented policy. The source queue is enabled, serialized with one build/merge, ALLGREEN, a one-minute minimum wait and 60-minute check timeout. The policy pack's merge_queue_posture: deferred describes a default, not this live installation.

Copilot feedback is advisory under solo-dev. Reviewer routing and review requests are not proof of a required human review. GitHub action_required instead means execution approval: trusted same-repo/current-head sweeps may approve those runs; fork runs stay held. Production currently has branch protection but no reviewer approval rule.

Cadence and recovery

Job	Configured cadence / trigger	What recovery actually does
Issue triage	Opened issue, manual dispatch	Labels/comments; no automatic implementation approval
Spec synthesis	Label, triage completion, manual	Selects explicit issue or oldest open unapproved needs-prd issue, skipping closed, synthesis-conflict, and merged-PR-closed issues; completion is not necessarily one-to-one with intake
Dependency routing	Events; Monday 09:00 UTC	Recomputes blockers; shared-ref cancellation can supersede evaluations
Issue metadata hygiene	Monday 07:00 UTC, manual	Metadata remediation, not agent execution
Auto-approval sweep	Same-repo PR events, selected producer completions, manual; every 15 minutes	Rechecks held run membership/head and approves execution; scheduled timing is not guaranteed
Merge eligibility	Required-check completion, PR metadata/head changes, dispatch	One-shot evaluation, no old 15-minute polling loop
Scheduled eligibility reconciliation	Auto-approver schedule	Disabled by solo-dev reconcile_merge_eligibility: false; its review-driven algorithm is not a universal green-PR recovery loop
Human-review reconciliation	Review submission; every 10 minutes	Relevant human approval/exception evidence; not recovery for every zero-review PR
PR hygiene	Readiness/metadata events; Monday 13:00 UTC	Reports draft drift at 14 days and ready inactivity at 7 days
Post-merge backfill	Hourly at minute 05, manual	Finds missing release-chain marker; an existing dispatch marker is not deployment completion evidence
Stuck-state watchdog (automation-stuck-state-watchdog.yml)	Hourly at minute 15, manual	Escalates at configured 24h issue progress, 12h ready-to-merge, 2h merge-to-release evidence thresholds
Release train	Monday 06:00 UTC, manual	Scheduled dry-run summary; live dispatch creates a candidate issue, never a release tag

Watchdog thresholds come from automation-stage-slas.json; they override input fallbacks. Issue progress uses updated_at, so comments/label churn can reset the apparent clock. Release-chain detection looks for marker evidence, not verified tag/publish completion. These are escalation heuristics, not measured stage SLAs or automatic repairs.

The post-merge release-gate dispatch sets dry_run=true and promotion_stage=validate. Packaging is dispatched independently; this is an evidence recorder, not an enforced production promotion barrier.

Measured timing and gaps

The 48-hour window ended 2026-10-06 14:41 UTC: 5,761 unique runs, 98 sampled job records. Equal 24-hour run volumes were 607 then 5,154; 1,190 cancelled. There were 432 repeated workflow/SHA/event groups with 896 extra candidates, not 896 proven redundant builds. Historical attempts are not separately counted. Workflow elapsed time is not pure execution; use executed steps for job timing.

Successful Linux test medians were 7.9 then 8.6 minutes; Windows medians 16.0, 17.3, then 15.1 minutes, with a small late sample. Sampled successful runner wait was approximately 4-6 seconds; failed acquisition is excluded.

Concrete handoffs on 2026-10-06:

• Issue #3574 opened at 14:56:44 UTC; synthesis labels at 15:06:07; draft #3586 opened at 15:08:51: 12m07s intake-to-draft, not intake-to-implementation.
• Twelve #3586 runs were held from 15:11:54-55. They were still held around 15:41; producer-driven recovery ran at 15:47 and a later snapshot showed zero holds. The timing brackets recovery; it does not establish the exact approval instant. This demonstrates a gap after a sweep, not a persistent token failure.

• 3571 opened at 14:17:48 and merged at 14:54:11: 36m23s PR lifetime.

  Tag release run 37483002945 began at 14:55:17 and ended successfully at 14:56:50; publish 37483002860 ended at 14:56:54; docs dispatch 37483233351 succeeded at 14:57:09. This is one sample, not an end-to-end SLA.
• Repeated v4.6.2 tag-triggered runs generated duplicate changelog PRs #3572 and #3581. Both remained ready, green, and without auto-merge or eligibility status at the audit snapshot. Missing reevaluation is observed; the exact lost-event cause is not established.

Priority gaps and tracking (all closed by 2026-10-24; current gaps are listed in Lifecycle audit 2026-10-24):

• 3573: unquoted model-refresh heredoc breaks full workflow lint.

• 3574: --delete-branch makes merge enablement fail with the native queue.

• 3575: core Windows wrapper can mask nested script nonzero exits.

• 3576: syntax enforcement gap; also reconcile intended eligibility enforcement.

• 3577: metadata/completion fan-out; retain legitimate exact-tree queue checks.

• 3578: packaging revalidation and immutable target/evidence alignment.

• 3579: irrelevant housekeeping allocations and late-held-run recovery.

• 3580: stage timing, attempts, failures, queue and deployment observability.

• 3587: this trace, directive asymmetry, draft-to-implementation handoff,

  disabled reconciliation, release-label mismatch, and tag ownership.

Do not call work delivered solely because an issue closed, a PR merged, a dry-run gate passed, or a dispatch comment exists. Verify intended version/tag, release assets, public publish, and required docs/smoke evidence. The public tag can have a different commit SHA because publication sanitizes the source tree.

Lifecycle audit 2026-10-24

Baseline: 5d1464ee, live repository state on 2026-10-24 (#3927). The 2026-10-06 gaps above are closed. The lifecycle is now limited by stalls between stages, not by check failures: every stage after spec synthesis is waiting on a gate that has no active owner or no escalation.

Board: spec-to-production-lifecycle.excalidraw (open at aka.ms/excalidraw).

flowchart LR
    A[Issue opened] --> B[Triage agent]
    B -->|needs-prd| C[Spec synthesis draft PR]
    C -->|human review and merge| D["/approve-spec or /approve"]
    D --> E[Assignment worker]
    E --> F[Coding agent PR]
    F --> G[Checks and intake metadata]
    G --> H[Eligibility executor]
    H --> I[Merge queue]
    I --> J[Post-merge chain]
    J -->|manual tag| K[Release and publish]
    K --> L[Docs and production evidence]
    B:::broken
    C:::stalled
    E:::blocked
    J:::stalled
    classDef broken stroke:#D13438,stroke-width:3px
    classDef blocked stroke:#D13438,stroke-dasharray:5 3
    classDef stalled stroke:#F7630C,stroke-dasharray:5 3

Solid red border: failing. Dashed red: blocked by configuration. Dashed orange: work accumulates with no escalation.

Stage status

Stage	Status on 2026-10-24	Evidence	Tracking
Triage	Failing	Issue Triage Agent fails every run since 04:39 UTC; gpt-5-mini no longer resolves	#3914, #3911
Spec synthesis	Idle	Of the last 30 runs, 28 skipped and 2 cancelled; none produced a spec draft	#3914
Spec review	Stalled	Six synthesized spec drafts (#3868, #3877, #3881, #3883, #3886, #3890) are 11 or more days old	#3926
Approval and handoff	Inert	BASECOAT_SPEC_HANDOFF_MODE unset, so approved-spec-handoff.yml runs read-only (4 success, 26 skipped of the last 30)	delivery-settings guide
Assignment	Blocked	Coding agent not enabled; all 5 open approved issues are unassigned	#3826, #3592
Checks and queue	Slow	Required checks take about 15 minutes; one serialized queue entry takes 18 to 25 minutes	#3923
Post-merge to release	Stalled	61 commits on main since v4.6.5 (2026-10-11); release train runs only as a dry run and never tags	#3594, #3663, #3843
Publish and docs	Healthy when tagged	v4.6.2 to v4.6.5: release, publish and docs dispatch each finished within 2 minutes of the tag	none
Stuck-state watchdog	Blind	30 of 30 runs succeeded with no escalations while the stalls above accumulated	#3926

Observed lead times

Sample	Issue opened to merge	PR open to merge	Merge to production
#3830 via #3849	17h01m	15h30m	Unreleased
#3859 via #3878	11d07h59m	11d01h07m	Unreleased

Samples are single observations, not SLAs. Lead time to production is currently unbounded because no tag is created after validated merges.

Why the watchdog misses these stalls

• Issue-to-PR stage: it queries only label:approved label:copilot-agent, skips issues without a verified coding-agent assignment before checking age, and ages the rest from the assignment's verifiedAt. Approved issues that were never assigned are not measured at all.
• Ready-to-merge stage: it skips draft PRs, so synthesized spec drafts are never measured.
• Merge-to-release stage: it checks for the post-merge-release-chain:v1 marker, which proves that a dispatch happened, not that a tag or publish exists.

Resolution order

1. Restore intake: fix the model pin (#3914).
2. Unblock implementation: enable the coding agent (#3826), then configure a handoff canary per the delivery settings guide.
3. Make stalls visible: add approval, spec-draft, and release-lag checks to the watchdog (#3926).
4. Bound release lag: governed automatic tag ownership (#3594), with #3663 and #3843 as prerequisites.
5. Shorten checks and queue time (#3923).