feat(ci): add nightly canary and stable release workflows

[ashish921998]

Summary

• packages/web/package.json: add "private": true so changeset publish never tries to publish the Next.js dashboard to npm
• .changeset/config.json: remove @aoagents/ao-web from the linked group, add it to ignore alongside ao-integration-tests, and add a snapshot.prereleaseTemplate for deterministic 0.x.y-nightly-<sha> version strings
• .github/workflows/release.yml: stable @latest publishing via changesets/action@v1 — opens/updates a "Version Packages" PR when changesets are pending; publishes on merge
• .github/workflows/canary.yml: triggered by workflow_run on CI success on main; includes stale-run guard (skips if SHA isn't current tip of main) and snapshot gap fix (creates a temp changeset when none exist so the snapshot step always has something to version); posts the install command as a comment on the merged PR
• README.md: canary install callout after the stable npm install line
• CONTRIBUTING.md: "Testing your changes" section after Development Setup

Test plan

• Verify pnpm changeset publish skips @aoagents/ao-web (private package)
• Merge a change to main with no pending changesets — canary workflow should auto-create temp changeset and publish @nightly
• Merge a "Version Packages" PR — release workflow should publish @latest
• Verify canary bot comments on merged PRs with the exact install command
• Run npm install -g @aoagents/ao@nightly after a canary publish to confirm it works

Closes #152

🤖 Generated with Claude Code

[github-actions]

Test Coverage Report

No TypeScript source files changed in this PR.

[greptile-apps]

Greptile Summary

This PR wires up nightly canary and stable release CI pipelines using Changesets, marks @aoagents/ao-web as private so it is never accidentally published, and adds developer docs for testing canary builds.

• P1 (canary.yml lines 98–104): The gh api comment body inherits YAML block-scalar indentation, placing 10 spaces before the ``` fence markers. CommonMark only recognises a fenced code block with 0–3 spaces of leading indentation, so the code block will render as literal text in the PR comment.

Confidence Score: 4/5

Safe to merge after fixing the comment-body indentation so the code fence renders correctly in PR comments.

One P1 finding: the YAML block-scalar indentation in the gh api call will cause the fenced code block in canary PR comments to render as plain text. The rest of the changes (release workflow, changeset config, private flag, docs) are correct and well-structured.

.github/workflows/canary.yml — comment body indentation (lines 98–104)

Important Files Changed

Filename	Overview
.github/workflows/canary.yml	New nightly canary workflow; stale-run guard and snapshot gap fix are sound, but the PR comment body has indentation that will break the fenced code block rendering (P1), and there is a minor dead short_sha output.
.github/workflows/release.yml	Standard changesets/action@v1 stable release workflow; permissions, fetch-depth, and build filter look correct.
.changeset/config.json	Adds snapshot prereleaseTemplate, removes @aoagents/ao-web from linked group, and adds it to ignore list; all changes are consistent with the private-package goal.
packages/web/package.json	Adds "private": true to prevent accidental npm publishing of the Next.js dashboard.
CONTRIBUTING.md	Adds canary testing instructions; example version is illustrative and acceptable.
README.md	Adds canary install callout after the stable install line; straightforward documentation change.

Sequence Diagram

sequenceDiagram
    participant Dev as Developer
    participant GH as GitHub (main push)
    participant CI as CI Workflow
    participant Canary as Canary Workflow
    participant Release as Release Workflow
    participant NPM as npm Registry
    participant PR as Merged PR Comment

    Dev->>GH: Push / merge PR to main
    GH->>CI: Trigger CI (lint, typecheck, test)
    GH->>Release: Trigger release.yml (push to main)
    Release->>Release: pnpm install + build
    Release->>Release: changesets/action@v1
    alt Changesets pending
        Release->>GH: Open/update Version Packages PR
    else Version Packages PR merged
        Release->>NPM: pnpm changeset publish at latest
    end

    CI-->>Canary: workflow_run completed (success)
    Canary->>GH: Check if SHA == tip of main
    alt SHA is stale
        Canary->>Canary: Skip (exit early)
    else SHA is current
        Canary->>Canary: pnpm install + build
        alt No changesets exist
            Canary->>Canary: Create canary-temp.md
        end
        Canary->>Canary: pnpm changeset version --snapshot nightly
        Canary->>NPM: pnpm changeset publish --tag nightly
        Canary->>PR: Comment with install command
    end

Loading

_{Reviews (1): Last reviewed commit: "feat(ci): add nightly canary and stable ..." | Re-trigger Greptile}

[ashish921998]

Fixed in the latest push. Three issues addressed:

1. [P1] Code fence indentation — switched to a heredoc so the backtick lines sit at column 0 and render as a proper code block in GitHub markdown.
2. [P2] Dead short_sha output — removed the two lines.
3. [P2] Case-sensitive README grep — changed to grep -vi so readme.md is also filtered out.

Also fixed a separate Codex-found [P1]: removed @aoagents/ao-web from the changeset ignore list — @aoagents/ao-cli depends on it, which caused a ValidationError on every changeset command. The private: true flag in packages/web/package.json is sufficient to prevent publishing.

[i-trytoohard]

PR Review: Canary + Stable Release Workflows

Overall this is a solid PR — the architecture is correct (changesets for stable, snapshots for canary, private flag for web). The Greptile findings were already addressed in commits 2-4. A few things to consider:

---

🔴 Worth Addressing

1. release.yml and canary.yml both trigger on every push to main

release.yml triggers directly on push; canary.yml triggers via workflow_run on CI completing. Both do pnpm -r --filter '!@aoagents/ao-web' build redundantly. If no changesets exist, release is a no-op — fine. But consider adding a path filter to release.yml so it only runs when .changeset/ files change, or at minimum document the interaction. Not a blocker, but wastes CI minutes.

2. id-token: write permission declared but unused

Both workflows declare id-token: write but neither uses OIDC federation. Remove it — least-privilege.

---

🟡 Suggestions

3. CANARY_VERSION extraction is hardcoded

CANARY_VERSION=$(node -p "require('./packages/ao/package.json').version")

This hardcodes the path. If the package is ever renamed/restructured, it silently breaks. Consider pnpm changeset status --output=json or at least add a comment noting the dependency.

4. PR lookup for commenting may miss squash-merged PRs

workflow_run.head_sha is the merge commit on main, not the PR head SHA. The commits/{sha}/pulls API should map merge commits to their PRs, but worth verifying with a real squash-merge. If it doesn't work, you'd need to parse the commit message for (#NNN).

5. HUSKY=0 set after pnpm install

If the install script ever triggers a husky hook (unlikely in CI but possible), it would fail. Set it before install:

- run: echo "HUSKY=0" >> $GITHUB_ENV
- run: pnpm install --frozen-lockfile

---

📝 Nits

• CONTRIBUTING.md example version 0.2.5-nightly-abc1234 will go stale. Consider x.y.z-nightly-<sha> as placeholder.
• No cleanup of canary-temp.md — fine in CI (fresh checkout), but worth a comment.

---

Verdict

Approve with nits. The id-token removal and HUSKY ordering are trivial fixes. The release/canary redundancy and path filter can be a follow-up. Ship it.

[ashish921998]

Thanks for the thorough review. Addressed in the latest commit:

Fixed:

• Removed id-token: write from both workflows — you're right, neither uses OIDC/provenance
• Moved HUSKY=0 before pnpm install in both workflows
• Added comment on the hardcoded CANARY_VERSION path noting the dependency
• Updated the CONTRIBUTING.md version placeholder to x.y.z-nightly-<sha>
• Added a comment clarifying that canary-temp.md is consumed and deleted by changeset version --snapshot — no explicit cleanup needed

On path filter for release.yml: Left it triggering on every push intentionally. changesets/action needs to run on every merge to keep the "Version Packages" PR up to date — if a push doesn't add a new changeset but the PR already exists, the action updates it with the latest commit. Filtering to .changeset/ would break that.

On squash-merge PR lookup: The commits/{sha}/pulls API maps merge commits to their associated PRs and does work for squash merges in practice. It's a known edge case if a commit lands via a method GitHub doesn't index (e.g. direct push with no PR), but in that case the comment step just silently no-ops — publish still succeeds.

[ashish921998]

Here are the consolidated comments, ready to paste on PR #1525. Grouped by severity, each with file:line.

---

🔴 P1 — Blocking

1. @aoagents/ao-cli depends on private @aoagents/ao-web — published installs will break

File: packages/cli/package.json:57 + packages/web/package.json:5

packages/web was just marked "private": true, but packages/cli/package.json:57 still declares "@aoagents/ao-web": "workspace:*" as a runtime dependency. Both workflows also skip building web (pnpm -r --filter '!@aoagents/ao-web' build). When @aoagents/ao-cli publishes to npm, the ao-web dependency can't be resolved — it's private and never published. End users running npm install -g @aoagents/ao will hit a missing dependency or get a stale/broken dashboard.

Fix options:

• Move @aoagents/ao-web to devDependencies and load it dynamically at runtime, or
• Bundle the built dashboard (.next/, dist-server/) into the @aoagents/ao-cli package's files and drop the workspace dep, or
• Don't make ao-web private — publish it alongside.

---

2. @aoagents/ao-web removed from linked but not added to ignore

File: .changeset/config.json:24-37

The PR description says: "add it to ignore alongside ao-integration-tests" — but the actual ignore array only contains @aoagents/ao-integration-tests. ao-web is now neither linked nor ignored. changeset version may still try to bump it when a referenced changeset exists. Either add it to ignore, or confirm private: true alone is sufficient and update the PR description.

---

🟠 P2 — Should fix before merging

3. Temp changeset only bumps @aoagents/ao — misses 6 publishable packages

File: .github/workflows/canary.yml:69-70

printf -- '---\n"@aoagents/ao": patch\n---\n\nchore: canary build\n' > .changeset/canary-temp.md

@aoagents/ao is in the linked group, so all linked packages bump together. But these publishable packages are outside the linked group and won't get a nightly version on the no-changesets path:

• composio-ao-plugin
• @aoagents/ao-plugin-agent-cursor
• @aoagents/ao-plugin-notifier-discord
• @aoagents/ao-plugin-notifier-openclaw
• @aoagents/ao-plugin-scm-gitlab
• @aoagents/ao-plugin-tracker-gitlab

Users running npm install <pkg>@nightly for any of these will get a stale version. Fix: either add them to the linked group, or write a wildcard changeset that bumps all publishable workspaces.

---

4. release.yml runs on every push to main with no dependency on CI passing

File: .github/workflows/release.yml:3-5

canary.yml correctly gates on workflow_run: [CI] + conclusion == 'success'. release.yml triggers directly on push: branches: [main] with no such gate. In practice changesets/action@v1 only publishes when a Version Packages PR is merged, but a broken build can still slip through. Mirror the canary pattern: workflow_run on CI success, or add a gating step that runs pnpm test / pnpm typecheck before publish.

---

5. Canary only waits for CI workflow — ignores Integration Tests and Security

File: .github/workflows/canary.yml:4-7

workflow_run: workflows: [CI] only listens to one workflow. If you have separate Integration Tests or Security workflows running on main, canaries can publish while those are failing. Either consolidate into the CI workflow, or add the others to the workflow_run.workflows list.

---

6. TOCTOU race in SHA-tip-of-main check

File: .github/workflows/canary.yml:30-38

Between the SHA check and the publish step, a new commit can land on main. The publish would then go out for an older (no-longer-tip) SHA. Combined with concurrency: cancel-in-progress: false, two canaries can serialize and publish in either order. Acceptable risk for nightly cadence, but worth either re-checking inside the publish step or relying on the nightly npm tag as the authoritative pointer.

---

🟡 P3 — Polish

7. Hardcoded ./packages/ao/package.json path

File: .github/workflows/canary.yml:82

CANARY_VERSION=$(node -p "require('./packages/ao/package.json').version")

If the package directory is ever renamed, this silently breaks (or picks the wrong version). Use module resolution instead: node -p "require('@aoagents/ao/package.json').version" — works through workspace symlinks regardless of directory name.

---

8. Inconsistent commit message styling between PR description and config

File: .github/workflows/release.yml:36-37

PR description says "Version Packages", the config uses "chore: version packages". Pick one — both work, but the inconsistency is a small confusion source for reviewers.

---

That's the full set. #1 is the actual blocker — the published-cli → private-web dependency will surface as broken npm install for end users the moment release.yml ships.

[i-trytoohard]

Review: feat(ci): add nightly canary and stable release workflows

Verdict: Solid — approved with one concern to address or document.

✅ What landed well

• Canary workflow is well-structured: stale-SHA guard prevents queued runs from publishing old builds, temp changeset creation when none exist is the right workaround for the "Version Packages merge leaves no changesets" gap, and continue-on-error on the PR comment is sensible.
• bundledDependencies for ao-web — the right fix for the private-publish drift problem. ao-web gets inlined into the CLI tarball, so end users always get the matching dashboard. Nice catch addressing illegalcall's concern.
• Changeset frontmatter fix — adding the opening --- to the two existing files prevents the first canary run from exploding. Critical fix.
• useCalculatedVersionForSnapshots: true — correct. Without it, snapshots would be 0.0.0-nightly-<sha> instead of the documented x.y.z-nightly-<sha>.
• New plugins added to linked group — ao-plugin-agent-cursor, notifier-discord, notifier-openclaw, scm-gitlab, tracker-gitlab. Good catch that the temp changeset would have missed these.
• Release workflow gated on CI — not just bare push, mirrors the canary pattern. Smart.
• Integration Tests in canary trigger — prevents canary from publishing while integration tests are failing.

⚠️ One concern: race between release.yml and canary.yml

Both workflows trigger on workflow_run from CI completing on main. If a Version Packages merge triggers both, canary and release could race on pnpm changeset version. The concurrency groups are separate (canary vs release), so they won't cancel each other.

Recommendation: Either document the expected ordering (e.g. "release always runs first because changesets/action consumes pending changesets, canary's temp-changeset path handles the no-changeset case"), or add a job-level dependency / status check so canary waits for release to complete. This isn't a merge blocker since the temp-changeset path handles the overlap gracefully — but worth a comment in the workflow files.

⚠️ Minor (non-blocking)

• bundledDependencies is deprecated in npm docs in favor of bundleDependencies (no 'd'). npm still supports both, but worth noting for future-proofing.
• Tarball size — Next.js builds can be chunky. Worth checking npm pack --dry-run to see the CLI tarball size post-bundle.
• GITHUB_TOKEN permissions — contents: write is set in release.yml, but the default GITHUB_TOKEN in public repos is often read-only. Make sure the repo settings allow the token write access for creating the Version Packages PR.

Overall: illegalcall's concerns are all addressed. The core approach is sound. Approved pending the race condition documentation.

[ashish921998]

npm rolled out OIDC trusted publishing in 2025, basically no token stored anywhere, github vouches for the workflow in real time. solves your secrets concern better than the VPS cron would and no infra to babysit.
as discussed with @AgentWrapper

[ashish921998]

Step 1: New npm token (npmjs.com)
Go to npmjs.com → click your avatar → Access Tokens
Click Generate New Token → choose Granular Access Token
Fill in:
Token name: agent-orchestrator-publish
Expiration: 90 days
Packages and scopes: Select packages → search @aoagents → select all of them → Permission: Read and write
Everything else: leave default
Click Generate Token — copy it immediately, you only see it once
Go back to the token list, find the old Automation or Publish token and delete it
Step 2: GitHub environment (github.com)
Go to your repo → Settings → Environments (left sidebar) → New environment
Name it exactly release (must match what's in the workflow files)
On the environment page:
Under Deployment branches and tags → click Add deployment branch or tag rule → choose Branch → type main
Under Environment secrets → click Add secret → Name: NPM_TOKEN, Value: paste the token from Step 1
Save
What happens after you merge
Every push to main that passes CI triggers canary.yml. When it reaches the publish step, GitHub checks: "is this job running from the release environment? Is the branch main?" — yes and yes, so it injects the NPM_TOKEN secret and the job runs.

If someone pushed from a branch that isn't main, or from a fork, GitHub refuses to inject the secret and the job fails before it can publish anything.

The NPM_TOKEN itself is now scoped to @aoagents/* only — even if it somehow leaked, it can't touch any other packages on npm.

Then go merge the PR.