Skip to content

feat(openai): Instrument structured outputs (chat.completions.parse)#4416

Open
Nik-Reddy wants to merge 3 commits intoopen-telemetry:mainfrom
Nik-Reddy:feat/openai-structured-outputs-3449
Open

feat(openai): Instrument structured outputs (chat.completions.parse)#4416
Nik-Reddy wants to merge 3 commits intoopen-telemetry:mainfrom
Nik-Reddy:feat/openai-structured-outputs-3449

Conversation

@Nik-Reddy
Copy link
Copy Markdown

@Nik-Reddy Nik-Reddy commented Apr 13, 2026

Description

The OpenAI v2 instrumentation currently wraps chat.completions.create() but not chat.completions.parse(). The parse() method is used for structured outputs. Calls to parse() generate zero telemetry even when instrumentors are configured.

This PR adds instrumentation for both sync and async parse() methods, reusing the existing chat completion wrapper logic.

Fixes #3449

Changes

  • Added _is_parse_supported() version guard and wrap/unwrap calls for parse methods
  • Handle response_format being a Python type by recording json_schema as the output type attribute
  • 8 new test cases (sync + async x content capture x semconv variants)
  • 4 VCR cassettes for structured output calls

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

All 8 new tests pass. All 84 existing tests continue to pass.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added

@Nik-Reddy Nik-Reddy requested a review from a team as a code owner April 13, 2026 00:17
@github-actions github-actions bot requested a review from lmolkova April 13, 2026 00:17
@xrmx xrmx added the gen-ai Related to generative AI label Apr 13, 2026
Copy link
Copy Markdown
Member

@MikeGoldsmith MikeGoldsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, thanks @Nik-Reddy. I've left some suggestions and please can you add a changelog entry.

@github-project-automation github-project-automation bot moved this to Reviewed PRs that need fixes in Python PR digest Apr 13, 2026
@Nik-Reddy Nik-Reddy force-pushed the feat/openai-structured-outputs-3449 branch from 3ec82e7 to 6d00a12 Compare April 13, 2026 20:16
@Nik-Reddy
Copy link
Copy Markdown
Author

Hi @MikeGoldsmith, I've addressed all three of your review comments:

  1. Cached the result of _is_parse_supported()\ as \self._parse_supported\ during _instrument()\
  2. Extracted shared test definitions into \structured_outputs_utils.py\
  3. Added clarifying comment about parse/create relationship

Would appreciate a re-review when you get a chance. Thanks!

@Nik-Reddy Nik-Reddy force-pushed the feat/openai-structured-outputs-3449 branch from 6d00a12 to ce2065f Compare April 15, 2026 01:08
@Nik-Reddy
Copy link
Copy Markdown
Author

Nik-Reddy commented Apr 15, 2026

Rebased on latest main. All review feedback @MikeGoldsmith addressed:

  • Cached _is_parse_supported()\ result on the instrumentor instance
  • Extracted shared test definitions into \structured_outputs_utils.py\
  • Added clarifying comment for parse vs create dispatch

@Nik-Reddy Nik-Reddy requested a review from MikeGoldsmith April 15, 2026 01:16
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some unused imports here; I would suggest running ruff formatting/linting

@MikeGoldsmith
Copy link
Copy Markdown
Member

Thanks for the updates, the changes look good. One thing still to address — there are unused imports in test_structured_outputs.py (import json and server_attributes as ServerAttributes) as flagged by @dehanjl. You can clean these up by running the following locally:

uv run tox -e lint

This needs to be done before we can accept.

Also, I want to raise something directly: the responses to review feedback have been very fast and follows a pattern that suggests an automated agent may be posting comments on your behalf. As per our AGENTS.md at the root of this project, discussions on OpenTelemetry repositories are for humans only — AI-generated comments on issues and PRs are not permitted. Please ensure that all review responses and PR comments are written and posted by you directly. Thanks for understanding.

@Nik-Reddy Nik-Reddy requested a review from dehanjl April 16, 2026 03:45
@Nik-Reddy Nik-Reddy force-pushed the feat/openai-structured-outputs-3449 branch from 3cbb955 to 9a7ae07 Compare April 16, 2026 07:37
@Nik-Reddy
Copy link
Copy Markdown
Author

@dehanjl good catch on the unused imports. Cleaned those up, dropped json, ServerAttributes, EXPECTED_RESPONSE_CONTENT, and pytest (wasn't actually needed in the sync test file). Ran ruff locally and everything's passing now.

Rebased on latest main as well.

Copy link
Copy Markdown
Member

@MikeGoldsmith MikeGoldsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ci is failing because of a new test that was added. precommit check is still failing too.

Please take a look 👍🏻

)


def test_structured_output_no_content(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is failing CI - please fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gen-ai Related to generative AI

Projects

Status: Reviewed PRs that need fixes

Development

Successfully merging this pull request may close these issues.

Instrument OpenAI structured outputs

5 participants