Skip to content

Lint and validate generated output files in tests #2449

@kdeldycke

Description

@kdeldycke

Problem

Generated output files (thin-caller workflows, config merges, changelog markdown) are assembled via string building in Python. Structural bugs — like the Python dict literal leak fixed in the _render_trigger_value() regression — are only caught by pattern-matching assertions, not by the same tools that lint these files in CI.

Proposed solution

Add post-generation validation in tests using the project's existing linting tools:

High priority (files written to disk in downstream repos)

Generated output Source Validation tool
Thin-caller workflows workflow_sync.generate_thin_caller() yamllint + actionlint
Workflow headers workflow_sync.generate_workflow_header() yamllint + actionlint
Labeller rule files init_project._init_config_files() yamllint
TOML config merge into pyproject.toml init_project.init_config() ruff format (TOML)
Changelog markdown changelog.py render + write mdformat

Medium priority (ephemeral or stdout-only)

Generated output Source Validation tool
Native TOML configs from [tool.X] bridge tool_runner._dict_to_toml() structural TOML validation
Native JSON configs from [tool.X] bridge tool_runner.resolve_config() jq
PR/issue body markdown pr_body.render_template() mdformat

Context

test_no_python_literals_in_yaml was added as a regression test after thin-caller generation serialized workflow_dispatch input definitions as Python dict literals ({'description': '...'}) instead of block-style YAML. The test catches that specific pattern, but running yamllint and actionlint on the generated output would catch a broader class of structural issues.

Metadata

Metadata

Assignees

No one assigned

    Labels

    🤖 ciContinuous integration and automation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions