Simplify PR template and add write-pr-description skill#1084
Simplify PR template and add write-pr-description skill#1084craigmarker wants to merge 4 commits intomainfrom
Conversation
The 7-section PR template had low compliance — fields got skipped or filled with "n/a." Since we squash & merge, the PR body becomes the commit message on main, so the template should be designed for that reality. This replaces the template with two high-value sections (Summary and Test plan), updates CONTRIBUTING.md to reflect the new philosophy, and adds a write-pr-description Claude skill that walks developers through producing a complete PR description. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The PR title becomes the commit subject line after squash & merge, so the skill should guide authors on writing good titles too. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
9 eval cases built from real commits across three tiers (small, medium, large). Ran 3 representative evals through the skill-creator pipeline — overall 83% pass rate (15/18). Key findings: the skill handles large architectural changes well (100%) but leaks implementation details on medium changes and misses user-visible symptoms on bug fixes. These results give a concrete baseline for iterating on the skill. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The workspace contains generated outputs and grading results from eval runs — useful locally but not appropriate for the repo. The eval definitions (evals.json) remain checked in. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
@craigmarker I noticed that the commit titles will go from: I thought we were following the Conventional Commit convention before but it looks like we're moving away from that. Looking at our commit history, we were somewhat doing this but wanted to check if this is something we wanted to follow. |
I’m open to following the type prefixing (feat/fix/docs/etc). One concern I have about the type prefixing approach is that when we’re squashing and merging, a single PR of multiple commits becomes a single git history entry. It’s easy for several commits to satisfy multiple types. If we supported rebase & merge, I wouldn’t be concerned about this problem. What do you think? |
Ah I see. In my last company, we actively tried to keep our PRs as small as possible to have it more focused and easier to review so the type prefixing made more sense since PRs covered only one of those. If we're not enforcing anything like that, it makes sense to not follow it. |
Summary
The 7-section PR template had low compliance — fields got skipped or
filled with "n/a." Since we squash & merge, the PR body becomes the
commit message on main, so the template should be designed for that
reality.
This replaces the template with two high-value sections (Summary and
Test plan), updates CONTRIBUTING.md to reflect the new philosophy, and
adds a write-pull-request-description Claude skill that walks developers
through producing a complete PR description.
Before / After Examples
These were generated by running the skill against real commits from our history.
Bug fix (3 files) — original commit message:
What the skill produces:
Large feature (48 files) — original commit message:
What the skill produces:
Test plan
.claude/skills/write-pull-request-description/evals/