Problem
The doc snippet test suite (tests/validate-doc-snippets.test.ts) currently guards against several classes of documentation drift:
- Type-string (
$type) literals that don't exist in any generated lexicon
validate() calls with arguments in the wrong order
- Imports of symbols that don't exist in the package exports
- Namespace/mapping accesses against symbols that no longer exist
However, it does not schema-validate the actual record literals embedded in the documentation snippets. This means a snippet like:
const receipt = {
$type: FUNDING_RECEIPT_NSID,
subject: { uri: "...", cid: "..." }, // ← field doesn't exist on the lexicon
paidAt: new Date().toISOString(), // ← field doesn't exist on the lexicon
// ... other fields
};
…passes all current tests even though the record is invalid against org.hypercerts.funding.receipt. This exact failure mode was caught by a human reviewer on #196 (the subject/paidAt funding receipt bug fixed in 44c0343), demonstrating the gap concretely.
Proposal
Extend tests/validate-doc-snippets.test.ts with a new check that:
- Extracts object literals from TypeScript code blocks in
README.md and .agents/skills/building-with-hypercerts-lexicons/SKILL.md.
- Identifies each literal's lexicon via its
$type marker (either a string literal or a resolved NSID constant).
- Runs each literal through the runtime
validate() function from generated/lexicons.
- Reports failures with source context (file, line, snippet excerpt) to make fixing easy.
Implementation sketch
A possible helper signature:
interface ExampleLiteral {
obj: unknown;
nsid: string; // resolved from $type
sourceFile: string;
sourceContext: string; // ~10 lines of surrounding snippet
}
function collectExampleObjectLiterals(
source: string,
sourceFile: string,
nsidConstants: Record<string, string>, // NSID name → value mapping
): ExampleLiteral[];
The new test suite then iterates these and calls validate(obj, nsid, 'main', false) from generated/lexicons.js, expecting result.success === true.
Extraction challenges
Naively regex-extracting object literals is fragile — nested braces, backtick strings, and comments can all confuse a regex. Two plausible approaches:
(a) Regex + bracket counting: Find const <name> = { or <name>: { openers, then walk forward counting {/} while skipping string/comment contents. Simple but brittle around edge cases.
(b) Lightweight TS parsing: Use a small parser (the typescript package is already a transitive dependency; @babel/parser is another option) to AST-walk each block. More robust but adds parse overhead and a direct dependency on a parser.
Given the docs snippets are mostly straightforward const x = { ... } literals, approach (a) is probably sufficient and avoids a new dependency. Approach (b) is only worth it if (a) turns out to misparse real snippets.
NSID resolution
$type in doc snippets is usually one of:
- A string literal:
$type: "org.hypercerts.funding.receipt"
- An NSID constant:
$type: FUNDING_RECEIPT_NSID
- A dotted namespace access:
$type: HYPERCERTS_NSIDS.FUNDING_RECEIPT
The test already parses the import statements (to verify symbols exist), so extending that parse to build a name→value map of imported NSID constants is straightforward.
Why this matters
- The funding-receipt snippet was wrong for a long time without being noticed.
- Every added/fixed docs snippet is essentially trusted code for downstream users who copy-paste it.
- The rest of the test file already establishes the pattern of "lint the docs against the generated code" — this is the logical next step.
Scope
tests/validate-doc-snippets.test.ts only.
- No changes to the lexicons or generated code.
- No changes to production code.
Out of scope
- Validating snippets in other markdown files (ERD.md, SCHEMAS.md, changelogs). Can be added later once the core extractor is in place.
- Validating non-object-literal record construction (e.g.
const r = makeReceipt(...)). Only direct object literals are in scope.
References
Problem
The doc snippet test suite (
tests/validate-doc-snippets.test.ts) currently guards against several classes of documentation drift:$type) literals that don't exist in any generated lexiconvalidate()calls with arguments in the wrong orderHowever, it does not schema-validate the actual record literals embedded in the documentation snippets. This means a snippet like:
…passes all current tests even though the record is invalid against
org.hypercerts.funding.receipt. This exact failure mode was caught by a human reviewer on #196 (thesubject/paidAtfunding receipt bug fixed in 44c0343), demonstrating the gap concretely.Proposal
Extend
tests/validate-doc-snippets.test.tswith a new check that:README.mdand.agents/skills/building-with-hypercerts-lexicons/SKILL.md.$typemarker (either a string literal or a resolved NSID constant).validate()function fromgenerated/lexicons.Implementation sketch
A possible helper signature:
The new test suite then iterates these and calls
validate(obj, nsid, 'main', false)fromgenerated/lexicons.js, expectingresult.success === true.Extraction challenges
Naively regex-extracting object literals is fragile — nested braces, backtick strings, and comments can all confuse a regex. Two plausible approaches:
(a) Regex + bracket counting: Find
const <name> = {or<name>: {openers, then walk forward counting{/}while skipping string/comment contents. Simple but brittle around edge cases.(b) Lightweight TS parsing: Use a small parser (the
typescriptpackage is already a transitive dependency;@babel/parseris another option) to AST-walk each block. More robust but adds parse overhead and a direct dependency on a parser.Given the docs snippets are mostly straightforward
const x = { ... }literals, approach (a) is probably sufficient and avoids a new dependency. Approach (b) is only worth it if (a) turns out to misparse real snippets.NSID resolution
$typein doc snippets is usually one of:$type: "org.hypercerts.funding.receipt"$type: FUNDING_RECEIPT_NSID$type: HYPERCERTS_NSIDS.FUNDING_RECEIPTThe test already parses the import statements (to verify symbols exist), so extending that parse to build a name→value map of imported NSID constants is straightforward.
Why this matters
Scope
tests/validate-doc-snippets.test.tsonly.Out of scope
const r = makeReceipt(...)). Only direct object literals are in scope.References
NON-BREAKING: fix docs discrepancies between README, AI skill, and lexicon schemas #196 (review)
subject/paidAtbug fixed in 44c0343 on NON-BREAKING: fix docs discrepancies between README, AI skill, and lexicon schemas #196 would have been caught automatically by this check.