feat: token-to-spec gap analysis, registry expansion, scaleIndex field, and spec proposals#814
Open
GarthDB wants to merge 17 commits into
Open
feat: token-to-spec gap analysis, registry expansion, scaleIndex field, and spec proposals#814GarthDB wants to merge 17 commits into
GarthDB wants to merge 17 commits into
Conversation
Contributor
🎨 Token Changes ReportTokens Changed (457)Original Branch: Newly Deprecated (194)
Added (190)
Updated (73)Added Properties (4)
Updated Properties (69)
This comment was automatically generated by the token diff tool. 🤖 |
🦋 Changeset detectedLatest commit: 2b22aa6 The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Member
Author
Run report for 2b22aa64Total time: 2m 5s | Comparison time: 5m 22s | Estimated savings: 3m 17s (61.1% faster)
Expanded report
Touched files |
…lysis Analyzes all 2,455 tokens against the new 13-field name object spec. Decomposes each token into semantic fields, scores confidence, identifies structural gaps, and generates reports. Includes: - Registry index loader with greedy multi-word term matching - Token decomposer with compound property and known gap classification - Gap analysis report generator - Token rename generator for ordering mismatches - Exception generator for spec-decision tokens - AVA test suite Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 registry additions based on token-mapping-analyzer results: sizes.json: - Add tokenName field for legacy long-form serialization - Add xxs (2x-small) and xxxs (3x-small) sizes - Add aliases for xxl (2x-large) and xxxl (3x-large) variants.json: - Add color variants: brown, cinnamon, silver, turquoise - Add emphasis variants: subtle, subdued - Add context variants: static, inverse - Add color modifiers: black, white anatomy-terms.json: - Add: title, description, row, step, counter, preview, tip, selection, media, ring states.json: - Add key-focus alias for keyboard-focus Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 2: deprecate tokens with non-standard field ordering and add spec-compliant names. Each renamed token gets deprecated=true, deprecated_comment, and renamed pointer to the new name. Categories: - key-focus → keyboard-focus vocabulary normalization (17 tokens) - Size abbreviation → long form (body-size-l → body-size-large, etc.) - State-first → state-last (disabled-background-color → background-color-disabled) - Icon reordering (icon-color-blue-background → blue-icon-background-color) - Component layout ordering (card-default-width-* → card-width-*-default) - Structure/object ordering (background-base-color → base-background-color) 83 tokens skipped as needing spec decisions (dual-variant, static-color compounds, component-typography patterns). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 3: document remaining 506 MEDIUM-confidence tokens with categorized exceptions and 6 spec amendment proposals. Proposals: - 001: Typography taxonomy (script, family, emphasis fields) - 002: Component property axes (style, staticColor, emphasis) - 003: Numeric scale index field - 004: Drop-shadow as structure - 005: Compound states - 006: Vocabulary gaps (registry additions) naming-exceptions.json updated with 408 new entries (602 total), each categorized and linked to its relevant proposal. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Renamed tokens were copying UUIDs from their deprecated originals, causing 176 SPEC-004 duplicate UUID violations. Regenerate unique UUIDs for all renamed-to tokens (outer and inner set UUIDs) and update the validation snapshot. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GarthDB
force-pushed
the
feat/token-spec-gap-analysis
branch
from
52b2954 to
02c932b
Compare
…yzer Replace hardcoded REGISTRY_FILES and SERIALIZATION_ORDER with dynamic field catalog from packages/design-data-spec/fields, aligning with #816. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Spec-order token names may have a variant prefix before the component (e.g., blue-icon-background-color), so check that the component appears in the name rather than requiring it at position 0. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add scaleIndex as a formal field in the configurable field catalog with a new "numeric" kind. The decomposer no longer flags numeric scale indices as taxonomy gaps when the field is declared. Re-categorize 12 naming exceptions from numeric-scale-index to ordering-mismatch, and regenerate the SDK codegen and validation snapshot. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GarthDB
changed the title
Distinguish small positional integers (1, 2, 3) from design system scale values (25–1600). Tokens like background-layer-1-color and gradient-stop-2-genai use ordinals for rank, not scale indices. Re-categorize 8 exceptions from ordering-mismatch to ordinal-rank. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collaborator
|
Thanks for writing this up so clearly. The inventory + gap surfacing work is genuinely useful. I’m aligned with the direction (we do need standardization). Where I’m not aligned is merging this PR as a combined “solution.” This PR bundles three different things that shouldn’t ship togetherRight now this PR includes:
Those don’t belong in a single pull request. It makes the “analysis” feel like the “answer,” and we’re not at a point where we should be codifying taxonomy decisions this way. Also: this PR explicitly calls out 176 renames as a breaking change. That’s a big impact on migration, and it shouldn’t be bundled into “analysis tooling.” This duplicates (and cuts across) work already in flightWe already have a Foundations-led, phased initiative for semantic token refinement + taxonomy formalization with DACI ownership and a plan for how we make decisions with partner teams. Within that initiative, we explicitly list these as out of scope:
That’s intentional: we’re still doing the work of defining the taxonomy (terms, categories, boundaries, and where “tokens vs guidance vs platform behavior” lines live). So even if parts of this PR are good ideas, merging them here creates a parallel decision path and bypasses the rigor we’re applying to make those decisions. “Roundtrip rate” is mechanical correctness, not semantic correctnessImproving roundtrip/confidence tells us how well a name can be decomposed/serialized against a proposed field catalog. That’s useful as diagnostics, but it’s not proof the taxonomy model is right — especially when the input set (existing token names) is known to contain inconsistency and ambiguity. Our taxonomy is meant to be a hierarchical classification system with mutually exclusive categories of terms, designed to be enforceable and stable over time. In the layout tokens phase, we also proved out that names alone don’t solve comprehension/usage and that semantics aren’t self-evident from strings. That’s why the work includes patterns/structures, guidance, validation, and education — not just ordering fields. So “we can roundtrip more names” can’t be treated as “we solved taxonomy.” Taxonomy categories/terms need to go through the full design + cross-team alignment process before being formalized as schema. Concrete modeling example:
|
Add a formal mechanism for composite tokens: a $valueType field that
declares the value-type schema a token conforms to. Composite value types
(typography, drop-shadow, typography-scale) are defined as JSON Schemas
under schemas/value-types/. Inline alias references ({token-name}) within
composite values follow the same resolution rules as top-level aliases.
Adds SPEC-014 (inline alias exists), SPEC-015 (inline alias type-
compatible), and SPEC-016 (value-type match) to rules.yaml. Includes
conformance fixtures for all three composite types plus a SPEC-016
violation case. No breaking changes — $valueType is optional.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…design-data-spec Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GarthDB
force-pushed
the
feat/token-spec-gap-analysis
branch
from
cf1d250 to
0fdee54
Compare
feat(monorepo): introduce diff architecture, tooling, and spec evolution - Add high-performance diff system: - Introduce @adobe/optimized-diff (up to ~2.8x faster) - Integrate into diff-generator with full API compatibility - Add @adobe/spectrum-diff-core shared library - Implement semantic diff + query engines (spec + SDK + CLI) - Expand tooling ecosystem: - Add token-changeset-generator, release-analyzer, changeset-linter - Add MCP servers (design data, S2 docs) and markdown generator - Add component-diff and spectrum-diff CLI tools - Add Figma export, registry, and design-data SDK/spec packages - Establish design data specification: - Add cascade format, lifecycle model, taxonomy, and query/diff specs - Implement validation rules (SPEC-001–016) and conformance suites - Add migration, roundtrip verification, and UUID tooling - Improve CI/CD and publishing: - Migrate to npm OIDC trusted publishing - Stabilize release workflows, diff actions, and large-output handling - Fix CI issues (EOF parsing, rate limits, empty diffs, caching) - Enhance docs and site: - Build 11ty docs site with markdown generation and AI indexing - Publish full design data spec and S2 documentation - Improve navigation, styling, and deployment workflows - Expand component schemas and tokens: - Add multiple component batches with breaking changes - Introduce semantic layout tokens and lifecycle metadata - Normalize naming to spec-order with taxonomy support - Refactor monorepo and tooling: - Standardize Moon workspace, tasks, and package structure - Replace legacy tasks and remove obsolete code paths - Update dependencies, configs, and schema infrastructure - Fixes and maintenance: - Resolve diff edge cases, YAML/frontmatter issues, and test flakiness - Clean temp files, improve performance thresholds, and stabilize tooling
Introduce a typography-scale composite value type that bundles font-size and line-height for each typographic tier. Deprecates the 18 separate line-height-font-size-* tokens (adding deprecated, deprecated_comment, and renamed lifecycle fields pointing to the new composite names). Adds the legacy token-types schema for typography-scale (desktop/mobile scale-set shape) so the existing @adobe/spectrum-tokens build pipeline continues to emit the correct legacy format. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 8 anatomy terms (field, fill, value, container, chevron, loupe, dot, and remaining terms) and 5 semantic dialog variants (confirmation, destructive, warning, error, information) to close vocabulary gaps surfaced by the token decomposer. Mirrors updates into the embedded registry in the Rust SDK. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Teach the migrator to treat typography-scale.json tokens as scale-sets (desktop/mobile split) rather than flat tokens. Carry set_schema through the cascade representation so the legacy emitter can reconstruct the correct $schema URL (instead of defaulting to scale-set for all scale dimensions). Update the token-mapping-analyzer decomposer to recognise typography-scale as a known compound property and reduce unmatched segment noise. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Teach the migrator to treat typography-scale.json tokens as scale-sets (desktop/mobile split) rather than flat tokens. Carry set_schema through the cascade representation so the legacy emitter can reconstruct the correct $schema URL (instead of defaulting to scale-set for all scale dimensions). Update the token-mapping-analyzer decomposer to recognise typography-scale as a known compound property and reduce unmatched segment noise. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GarthDB
force-pushed
the
feat/token-spec-gap-analysis
branch
from
cc2f2de to
57a63e5
Compare
Resolves add/add conflicts in registry, tokens, and SDK files where beta had the pre-PR-2 state. All resolutions keep the PR-2 changes (ours). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Comprehensive analysis of all 2,455 tokens against the 13-field taxonomy name object spec, with remediation across multiple phases:
scaleIndexas a formal field in the configurable field catalog with a new"numeric"kind, eliminating taxonomy gaps for ~280 scale tokensResults
scaleIndex Field (Proposal 003)
Added
scaleIndexas the first"numeric"kind field in the field catalog:packages/design-data-spec/fields/scaleIndex.jsonvalueType: "integer",validation: "none",serialization.position: 16numeric-scale-indexnaming exceptions re-categorized toordering-mismatch(these tokens have mid-name numerics that still don't roundtrip)Spec Proposals (in
docs/proposals/)script,family,emphasisfields (221 tokens)style,staticColor,emphasisaligned with component schema properties (234 tokens)Related Issue
Related to taxonomy/naming framework work on
feat/taxonomy-naming-frameworkandfeat/taxonomy-registry-splitbranches.Motivation and Context
We needed to understand where the new 13-field token naming taxonomy fits existing tokens and where it doesn't, before building migration tooling. This PR provides that complete picture: what works, what needs renaming, and what needs spec changes — with concrete proposals for each gap.
Key finding: component schemas already define
style,staticColor, andisEmphasizedas independent variant axes. The token taxonomy should mirror this rather than using array variants.How Has This Been Tested?
deprecated/renamedpointers, new tokens preserve values/setsTypes of changes
Note: Token renames are breaking — 176 tokens deprecated with
renamedpointers to new spec-order names.Checklist: