feat(examples): migrate AI examples to OpenTelemetry instrumentation#482
feat(examples): migrate AI examples to OpenTelemetry instrumentation#482richardsolomou merged 10 commits intomasterfrom
Conversation
posthog-python Compliance ReportDate: 2026-04-15 12:04:06 UTC
|
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 517ms |
| Format Validation.Event Has Uuid | ✅ | 1508ms |
| Format Validation.Event Has Lib Properties | ✅ | 1507ms |
| Format Validation.Distinct Id Is String | ✅ | 1508ms |
| Format Validation.Token Is Present | ✅ | 1507ms |
| Format Validation.Custom Properties Preserved | ✅ | 1507ms |
| Format Validation.Event Has Timestamp | ✅ | 1508ms |
| Retry Behavior.Retries On 503 | ✅ | 9520ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 3506ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 3508ms |
| Retry Behavior.Respects Retry After Header | ✅ | 9512ms |
| Retry Behavior.Implements Backoff | ✅ | 23532ms |
| Retry Behavior.Retries On 500 | ✅ | 7507ms |
| Retry Behavior.Retries On 502 | ✅ | 7512ms |
| Retry Behavior.Retries On 504 | ✅ | 7513ms |
| Retry Behavior.Max Retries Respected | ✅ | 23518ms |
| Deduplication.Generates Unique Uuids | ✅ | 1509ms |
| Deduplication.Preserves Uuid On Retry | ✅ | 7514ms |
| Deduplication.Preserves Uuid And Timestamp On Retry | ✅ | 14523ms |
| Deduplication.Preserves Uuid And Timestamp On Batch Retry | ✅ | 7504ms |
| Deduplication.No Duplicate Events In Batch | ✅ | 1508ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 1509ms |
| Compression.Sends Gzip When Enabled | ✅ | 1507ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 1507ms |
| Batch Format.Flush With No Events Sends Nothing | ✅ | 1006ms |
| Batch Format.Multiple Events Batched Together | ✅ | 1505ms |
| Error Handling.Does Not Retry On 403 | ✅ | 3509ms |
| Error Handling.Does Not Retry On 413 | ✅ | 3508ms |
| Error Handling.Retries On 408 | ✅ | 7515ms |
Feature_Flags Tests
View Details
| Test | Status | Duration |
|---|---|---|
| Request Payload.Request With Person Properties Device Id | ❌ | 500ms |
Failures
request_payload.request_with_person_properties_device_id
404, message='NOT FOUND', url='http://sdk-adapter:8080/get_feature_flag'
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
andrewm4894
left a comment
There was a problem hiding this comment.
i think the otel examples should be as full featured as possible and show stuff like setting distinct id and any custom foo-bar properties - am i missing that or something?
Aligns with the endpoint used by the OTel examples in PR #482. Generated-By: PostHog Code Task-Id: 1ba1f07a-1453-4162-90a8-665958c5fe46
ac98a3d to
5c0ac0e
Compare
This stack of pull requests is managed by Graphite. Learn more about stacking. |
Aligns with the endpoint used by the OTel examples in PR #482. Generated-By: PostHog Code Task-Id: 1ba1f07a-1453-4162-90a8-665958c5fe46
ee77800 to
8818397
Compare
5c0ac0e to
6d9b25b
Compare
Aligns with the endpoint used by the OTel examples in PR #482. Generated-By: PostHog Code Task-Id: 1ba1f07a-1453-4162-90a8-665958c5fe46
8818397 to
284aba6
Compare
6d9b25b to
9d18d8f
Compare
Merge activity
|
## Problem Users instrumenting their AI/LLM applications with OpenTelemetry have no way to route AI spans to PostHog without writing custom glue code. This is the Python equivalent of PostHog/posthog-js#3358. ## Changes Adds a `posthog.ai.otel` module with two integration patterns: - **`PostHogSpanProcessor`** (recommended) — self-contained `SpanProcessor` that wraps `BatchSpanProcessor` + `OTLPSpanExporter`, filtering to only forward AI spans to PostHog's OTLP endpoint at `/i/v0/ai/otel` - **`PostHogTraceExporter`** — `SpanExporter` that filters AI spans before forwarding, for setups that only accept an exporter (e.g. passed to `SimpleSpanProcessor`) - **`is_ai_span()`** — shared predicate matching `gen_ai.*`, `llm.*`, `ai.*`, `traceloop.*` prefixes on span names and attribute keys Also adds `posthog[otel]` optional dependency group for `opentelemetry-sdk` and `opentelemetry-exporter-otlp-proto-http`. ### Usage ```python from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.resources import Resource, SERVICE_NAME from posthog.ai.otel import PostHogSpanProcessor resource = Resource( attributes={ SERVICE_NAME: "my-app", "posthog.distinct_id": "user-123", } ) provider = TracerProvider(resource=resource) provider.add_span_processor( PostHogSpanProcessor(api_key="phc_...") ) trace.set_tracer_provider(provider) ``` ### Consistency with #482 Uses the same OTLP endpoint (`/i/v0/ai/otel`) and auth header format (`Authorization: Bearer {api_key}`) as the migrated examples in #482. ## How did you test this code? 31 unit tests covering span filtering, processor forwarding/dropping, exporter filtering, endpoint configuration, and lifecycle delegation.
…tion (#53668) ## Problem The in-app LLM analytics onboarding docs currently show PostHog SDK wrappers (`posthog.ai.openai`, `@posthog/ai`) as the primary integration method. We're moving toward standard OpenTelemetry auto-instrumentation as the recommended approach, keeping the wrappers available as a last resort for users who need them. ## Changes Migrates 28 provider onboarding guides from PostHog SDK wrapper approach to OpenTelemetry auto-instrumentation. Each migrated doc follows a consistent 3-step pattern: 1. **Install dependencies** — OTel SDK + provider-specific instrumentation + provider SDK 2. **Set up OpenTelemetry tracing** — `TracerProvider` with `OTLPSpanExporter` (Python) or `PostHogTraceExporter` (Node) 3. **Call the provider** — Native SDK usage, no `posthog_` parameters User identification uses the `posthog.distinct_id` resource attribute; custom properties like `foo` and `conversation_id` demonstrate the passthrough behavior. **Providers migrated (21 original + 7 more):** - OpenAI-compatible: openai, groq, deepseek, mistral, cohere, fireworks-ai, xai, hugging-face, openrouter, together-ai, ollama, cerebras, perplexity - AI gateways: portkey, helicone, vercel-ai-gateway - Azure OpenAI, Anthropic - Frameworks: langchain, langgraph, llamaindex - Vercel AI (SimpleSpanProcessor update) - **Newly migrated:** autogen, instructor, mirascope, semantic-kernel, smolagents, pydantic-ai (already OTel-based in examples — docs just hadn't caught up) - **Mastra** (migrated to `@mastra/posthog` exporter — Mastra's native integration since there's no mature OTel path yet) **Kept as-is:** google (no Node.js OTel instrumentation for `@google/genai`), aws-bedrock (already OTel), crewai, litellm, dspy, openai-agents, manual **Migration callout:** Every migrated doc has a callout at the top linking to the full Node.js and Python examples on GitHub (current `main`/`master`), plus legacy wrapper examples pinned to the last commit before the migration. This helps both new users (who see the recommended OTel path) and existing users (who can find the old wrapper examples they're already using). **Manual capture doc:** Split into multiple steps so it can have a useful table of contents (Capture LLM events manually → Event properties → Verify traces and generations). **Removed:** "No proxy" callouts from all migrated docs — they made sense when wrapping SDK clients but don't apply to standard OTel tracing. ## How did you test this code? Agent-authored PR. Manually verified: - TypeScript compiles (`pnpm --filter=@posthog/frontend format` runs cleanly) - Each migrated doc follows the same pattern as other OTel docs - Example code in the docs matches what's in the real example repos (PostHog/posthog-js#3349 and PostHog/posthog-python#482) No runtime tests since these are in-app onboarding content (static TSX returning step definitions). ## Publish to changelog? No ## Docs update This IS the docs update. Companion PR: PostHog/posthog.com#16236 (updates TOC frontmatter in posthog.com). ## 🤖 LLM context Co-authored with Claude Code. Related PRs: - PostHog/posthog-js#3349 (Node.js OTel examples) - PostHog/posthog-python#482 (Python OTel examples) - PostHog/posthog.com#16236 (docs.posthog.com TOC updates)
Switch from PostHog direct SDK wrappers to OpenTelemetry auto-instrumentation for all AI provider examples where OTel instrumentations are available. Uses opentelemetry-instrumentation-openai-v2 for OpenAI-compatible providers, opentelemetry-instrumentation-anthropic for Anthropic, opentelemetry-instrumentation-google-generativeai for Gemini, opentelemetry-instrumentation-langchain for LangChain/LangGraph, opentelemetry-instrumentation-llamaindex for LlamaIndex, and opentelemetry-instrumentation-crewai for CrewAI.
Switch from BatchSpanProcessor to SimpleSpanProcessor so spans are exported immediately. This removes the need for provider.shutdown() which could throw export errors on exit.
CrewAI manages its own TracerProvider internally, which conflicts with setting one externally. LiteLLM callbacks remain the correct integration approach for CrewAI.
…el instrumentation
…rter Adopt the new PostHogSpanProcessor API from #494 across all OTEL examples. Drops the manual SimpleSpanProcessor + OTLPSpanExporter wiring in favor of the higher-level processor that handles batching and AI span filtering. Each example now depends on posthog[otel] via a uv path source so it resolves against the workspace checkout while #494 is unreleased.
9d18d8f to
7289c3e
Compare
…tion (#53668) ## Problem The in-app LLM analytics onboarding docs currently show PostHog SDK wrappers (`posthog.ai.openai`, `@posthog/ai`) as the primary integration method. We're moving toward standard OpenTelemetry auto-instrumentation as the recommended approach, keeping the wrappers available as a last resort for users who need them. ## Changes Migrates 28 provider onboarding guides from PostHog SDK wrapper approach to OpenTelemetry auto-instrumentation. Each migrated doc follows a consistent 3-step pattern: 1. **Install dependencies** — OTel SDK + provider-specific instrumentation + provider SDK 2. **Set up OpenTelemetry tracing** — `TracerProvider` with `OTLPSpanExporter` (Python) or `PostHogTraceExporter` (Node) 3. **Call the provider** — Native SDK usage, no `posthog_` parameters User identification uses the `posthog.distinct_id` resource attribute; custom properties like `foo` and `conversation_id` demonstrate the passthrough behavior. **Providers migrated (21 original + 7 more):** - OpenAI-compatible: openai, groq, deepseek, mistral, cohere, fireworks-ai, xai, hugging-face, openrouter, together-ai, ollama, cerebras, perplexity - AI gateways: portkey, helicone, vercel-ai-gateway - Azure OpenAI, Anthropic - Frameworks: langchain, langgraph, llamaindex - Vercel AI (SimpleSpanProcessor update) - **Newly migrated:** autogen, instructor, mirascope, semantic-kernel, smolagents, pydantic-ai (already OTel-based in examples — docs just hadn't caught up) - **Mastra** (migrated to `@mastra/posthog` exporter — Mastra's native integration since there's no mature OTel path yet) **Kept as-is:** google (no Node.js OTel instrumentation for `@google/genai`), aws-bedrock (already OTel), crewai, litellm, dspy, openai-agents, manual **Migration callout:** Every migrated doc has a callout at the top linking to the full Node.js and Python examples on GitHub (current `main`/`master`), plus legacy wrapper examples pinned to the last commit before the migration. This helps both new users (who see the recommended OTel path) and existing users (who can find the old wrapper examples they're already using). **Manual capture doc:** Split into multiple steps so it can have a useful table of contents (Capture LLM events manually → Event properties → Verify traces and generations). **Removed:** "No proxy" callouts from all migrated docs — they made sense when wrapping SDK clients but don't apply to standard OTel tracing. ## How did you test this code? Agent-authored PR. Manually verified: - TypeScript compiles (`pnpm --filter=@posthog/frontend format` runs cleanly) - Each migrated doc follows the same pattern as other OTel docs - Example code in the docs matches what's in the real example repos (PostHog/posthog-js#3349 and PostHog/posthog-python#482) No runtime tests since these are in-app onboarding content (static TSX returning step definitions). ## Publish to changelog? No ## Docs update This IS the docs update. Companion PR: PostHog/posthog.com#16236 (updates TOC frontmatter in posthog.com). ## 🤖 LLM context Co-authored with Claude Code. Related PRs: - PostHog/posthog-js#3349 (Node.js OTel examples) - PostHog/posthog-python#482 (Python OTel examples) - PostHog/posthog.com#16236 (docs.posthog.com TOC updates)

Problem
Our AI examples use PostHog's direct SDK wrappers (
posthog.ai.openai,posthog.ai.anthropic, etc.) for tracking LLM calls. We want to silently deprecate these in favor of standard OpenTelemetry auto-instrumentation, which is more portable and follows industry conventions.Changes
Migrates Python AI examples from PostHog wrappers to OpenTelemetry auto-instrumentation:
opentelemetry-instrumentation-openai-v2opentelemetry-instrumentation-openai-v2opentelemetry-instrumentation-anthropicopentelemetry-instrumentation-langchainopentelemetry-instrumentation-llamaindexopentelemetry-instrumentation-google-generativeaiAll OTel-based examples set resource attributes to demonstrate the full feature set:
These map to
distinct_idand custom event properties via PostHog's OTLP ingestion endpoint.Kept as-is: CrewAI (uses LiteLLM callbacks, internally manages its own TracerProvider), LiteLLM/DSPy (use LiteLLM's built-in PostHog callback), OpenAI Agents (uses dedicated
posthog.ai.openai_agents.instrument()), Pydantic AI (already OTel viaAgent.instrument_all()), AWS Bedrock (already OTel viaopentelemetry-instrumentation-botocore).Key implementation details:
SimpleSpanProcessorinstead ofBatchSpanProcessorso spans export immediately without needingprovider.shutdown()# noqa: E402on intentional late imports afterInstrumentor().instrument()callsgpt-4omodel name instead of a deployment-specific oneHow did you test this code?
Manually ran each example against real provider API keys via
llm-analytics-apps/run-examples.shto verify:$ai_generationeventsposthog.distinct_id,foo,conversation_id) flow through as event propertiesdistinct_idis correctly set on each eventAll examples passing
ruff formatandruff check.This is an agent-authored PR — I haven't manually tested each provider end-to-end beyond spot checks, though all examples follow the same pattern and the migration was verified on several providers.
Publish to changelog?
No
Docs update
The onboarding docs will be updated separately in PostHog/posthog#53668 and PostHog/posthog.com#16236.
🤖 LLM context
Co-authored with Claude Code. Related PRs: