Skip to content

test: schema-validate object literals in doc snippets #202

@aspiers

Description

@aspiers

Problem

The doc snippet test suite (tests/validate-doc-snippets.test.ts) currently guards against several classes of documentation drift:

  • Type-string ($type) literals that don't exist in any generated lexicon
  • validate() calls with arguments in the wrong order
  • Imports of symbols that don't exist in the package exports
  • Namespace/mapping accesses against symbols that no longer exist

However, it does not schema-validate the actual record literals embedded in the documentation snippets. This means a snippet like:

const receipt = {
  $type: FUNDING_RECEIPT_NSID,
  subject: { uri: "...", cid: "..." },   // ← field doesn't exist on the lexicon
  paidAt: new Date().toISOString(),      // ← field doesn't exist on the lexicon
  // ... other fields
};

…passes all current tests even though the record is invalid against org.hypercerts.funding.receipt. This exact failure mode was caught by a human reviewer on #196 (the subject/paidAt funding receipt bug fixed in 44c0343), demonstrating the gap concretely.

Proposal

Extend tests/validate-doc-snippets.test.ts with a new check that:

  1. Extracts object literals from TypeScript code blocks in README.md and .agents/skills/building-with-hypercerts-lexicons/SKILL.md.
  2. Identifies each literal's lexicon via its $type marker (either a string literal or a resolved NSID constant).
  3. Runs each literal through the runtime validate() function from generated/lexicons.
  4. Reports failures with source context (file, line, snippet excerpt) to make fixing easy.

Implementation sketch

A possible helper signature:

interface ExampleLiteral {
  obj: unknown;
  nsid: string;          // resolved from $type
  sourceFile: string;
  sourceContext: string; // ~10 lines of surrounding snippet
}

function collectExampleObjectLiterals(
  source: string,
  sourceFile: string,
  nsidConstants: Record<string, string>, // NSID name → value mapping
): ExampleLiteral[];

The new test suite then iterates these and calls validate(obj, nsid, 'main', false) from generated/lexicons.js, expecting result.success === true.

Extraction challenges

Naively regex-extracting object literals is fragile — nested braces, backtick strings, and comments can all confuse a regex. Two plausible approaches:

(a) Regex + bracket counting: Find const <name> = { or <name>: { openers, then walk forward counting {/} while skipping string/comment contents. Simple but brittle around edge cases.

(b) Lightweight TS parsing: Use a small parser (the typescript package is already a transitive dependency; @babel/parser is another option) to AST-walk each block. More robust but adds parse overhead and a direct dependency on a parser.

Given the docs snippets are mostly straightforward const x = { ... } literals, approach (a) is probably sufficient and avoids a new dependency. Approach (b) is only worth it if (a) turns out to misparse real snippets.

NSID resolution

$type in doc snippets is usually one of:

  • A string literal: $type: "org.hypercerts.funding.receipt"
  • An NSID constant: $type: FUNDING_RECEIPT_NSID
  • A dotted namespace access: $type: HYPERCERTS_NSIDS.FUNDING_RECEIPT

The test already parses the import statements (to verify symbols exist), so extending that parse to build a name→value map of imported NSID constants is straightforward.

Why this matters

  • The funding-receipt snippet was wrong for a long time without being noticed.
  • Every added/fixed docs snippet is essentially trusted code for downstream users who copy-paste it.
  • The rest of the test file already establishes the pattern of "lint the docs against the generated code" — this is the logical next step.

Scope

  • tests/validate-doc-snippets.test.ts only.
  • No changes to the lexicons or generated code.
  • No changes to production code.

Out of scope

  • Validating snippets in other markdown files (ERD.md, SCHEMAS.md, changelogs). Can be added later once the core extractor is in place.
  • Validating non-object-literal record construction (e.g. const r = makeReceipt(...)). Only direct object literals are in scope.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions