You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
red-team: address second Copilot review pass (5 new comments)
Resolves inline comments from review round 2:
1. Added explicit fenced Usage block under ## Outline; §1 now
references it by name ("print the fenced Usage block from §Outline")
instead of the ambiguous "print the usage block above" instruction.
2. Simplified the >5-lens flow: removed the "non-interactive detection"
branch entirely. Interactivity is now determined solely by whether
--yes was passed — CI / batch runs MUST pass --yes; otherwise the
run prompts. Added a clarifying parenthetical and updated §8 row.
3. Aligned dry-run session-ID example with the canonical format.
Both now use RT-<feature-id>-<YYYY-MM-DD>[-<NN>] with the -<NN>
as optional.
4. Converted UK spelling to US to match the rest of templates/commands/:
"artefact" -> "artifact", "behaviour" -> "behavior".
5. Picked a single threshold for overwhelming-findings: >=25
HIGH+CRITICAL combined. Dropped the inconsistent ">50 HIGH"
example; the row now states one actionable heuristic.
No behavior changes to the protocol; all fixes are template-hygiene
and contract-clarity.
Copy file name to clipboardExpand all lines: templates/commands/red-team.md
+14-9Lines changed: 14 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -54,7 +54,11 @@ Goal: Run an adversarial review of a functional spec using project-configured le
54
54
55
55
**Schema references**: a concrete schema for the lens catalog (`.specify/red-team-lenses.yml`) and the findings report format is **TODO — to be added as sibling template files in a follow-up PR if this core approach lands**. Until then, the minimal required lens-catalog shape is inline in §2 preconditions and the findings-report sections are enumerated in §6.5 below.
- `--dry-run` (flag): report which lenses would run and why, without dispatching adversary agents.
67
71
- `--session-suffix <NN>` (flag with value): override the session ID's trailing ordinal when multiple sessions occur on the same day.
68
72
69
-
If `$ARGUMENTS` is empty OR the target spec path is missing, print the usage block above and STOP. Do NOT try to infer the target from context.
73
+
If `$ARGUMENTS` is empty OR the target spec path is missing, print the fenced Usage block from §Outline above and STOP. Do NOT try to infer the target from context.
70
74
71
75
## 2. Preconditions check
72
76
@@ -132,13 +136,14 @@ Resolve each name against the catalog. Unknown names produce a warning and are d
132
136
- The proposed top-5 default with, for each, a one-line rationale (which triggers it covers + severity_weight).
133
137
- The dropped lenses with the reason they ranked below.
134
138
- **If `--yes` was passed**: auto-accept the proposed default. Set `selection_method: auto` in the session record with a note that --yes was used. Skip to §5.
135
-
- **If `--yes` was NOT passed AND the invocation is non-interactive**: FAIL FAST. Print: `ERROR: more than 5 lenses match; interactive confirmation required. Re-run with --yes to accept the proposed default, or --lenses to specify explicitly.` and STOP.
136
-
- **If interactive**: ask the maintainer to respond:
139
+
- **Otherwise** (no `--yes`): prompt the maintainer to respond:
137
140
- "accept" / "yes" → use proposed default; `selection_method: proposed-and-confirmed`.
138
141
- "swap A for B" → swap a default lens with a dropped lens; `selection_method: swapped`.
139
142
- "expand to N" (N > 5) → run more than 5 lenses (maintainer opts into the cost); `selection_method: expanded`.
140
143
- Anything else → re-prompt with the three options above.
141
144
145
+
*(CI / batch runs MUST pass `--yes` to auto-accept the proposed default; running without `--yes` in a non-interactive context will stall waiting for input. This keeps the behavior simple: interactivity is determined by whether `--yes` was passed, not by detecting the terminal.)*
146
+
142
147
Write the final `selected_lenses` list. Validate 3 ≤ `len(selected_lenses)` ≤ 5 (unless `selection_method == expanded`). If below 3, warn the maintainer that lens diversity is weak — offer to abort.
**Rationale**: SpecKit working records in `specs/<feature-id>/` capture a point-in-time decision state. They serve as the audit trail of "what was decided at time T." Rewriting them destroys that audit trail. If the correct fix for a red team finding would require editing one of these files, the correct resolution category is:
@@ -248,7 +253,7 @@ Walk the maintainer through each finding. For each finding in the table (group b
248
253
249
254
## 8. Failure-mode handling
250
255
251
-
| Condition | Behaviour |
256
+
| Condition | Behavior |
252
257
|---|---|
253
258
| Target spec missing | Fail fast with `ERROR: target spec not found at <path>`. No session created. |
254
259
| Lens catalog missing | Fail fast with the minimal-required-shape error printed in §2.2 above (no external doc references). |
@@ -258,9 +263,9 @@ Walk the maintainer through each finding. For each finding in the table (group b
258
263
| Constitution lacks trigger criteria | Warn and proceed in bootstrap mode using the six default categories. UNLESS `--lenses` was passed (bypass). |
259
264
| Target spec matches zero triggers AND no `--lenses` | Print info message and STOP. Not an error. |
260
265
| No lens in catalog covers matched triggers | Fail fast — asks maintainer to extend catalog or pass --lenses. |
261
-
| >5 matches in non-interactive without `--yes` | Fail fast. Maintainer must re-run with `--yes` or `--lenses`. |
266
+
| >5 matches without `--yes` | Prompt the maintainer for accept / swap / expand. CI / batch runs MUST pass `--yes` to auto-accept the proposed default; otherwise the run will stall waiting for input. |
262
267
| Individual adversary agent fails (timeout, parse error, empty response) | Record failure in session metadata with lens name + reason. Continue with other lenses. Flag for re-run via `--lenses`. Do NOT abort the session. |
263
-
| Overwhelming findings (e.g., >50 HIGHafter aggregation) | After §6 completes, if total findings exceed a threshold (default: ≥25 HIGH+CRITICAL), warn the maintainer the spec may not be ready for red team and offer an abort path. Abort records session state for later resumption. |
268
+
| Overwhelming findings (≥25 HIGH+CRITICAL combined after aggregation) | After §6 completes, if the combined count of HIGH and CRITICAL findings meets or exceeds **25**, warn the maintainer the spec may not be ready for red team and offer an abort path. Abort records session state for later resumption. |
264
269
| Spec was updated since prior red team | On session start, check for prior findings report in the feature dir. If present and older than a material-change threshold (heuristic: target spec has new FRs or section count changed), warn the maintainer and ask whether to proceed or abort. |
265
270
| Session interrupted mid-resolution | Report file is saved atomically after every resolution update. On re-invocation with same session ID, offer to resume from last-resolved finding. |
0 commit comments