chore(ci_visibility): fix test isolation for xdist and Docker worktree#17458
Conversation
…e xdist env detection Four sources of module-level bleed-through prevented running the ddtrace plugin in the outer pytest process when inline_run() starts inner sessions: 1. PYTEST_XDIST_WORKER_VALUE frozen at import time — add is_xdist_worker_env() which reads os.environ live so inner inline_run() sessions (which clear PYTEST_XDIST_WORKER from the env) correctly detect they are not workers. 2. skipped_suites and skip_pytest_runtest_protocol never reset between inline_run() calls — reset both in pytest_configure() for every new main-process session (guarded by not hasattr(config, "workerinput")). 3. CIVisibility singleton has no isolation support — add _suspend()/_resume() class methods that remove/restore _instance, enabled, and the service registry pointer without going through stop()/start() on the outer session's instance. enable() and disable() are unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…orktree
ci_visibility suite:
- Add tests/ci_visibility/conftest.py with an autouse _ci_visibility_isolation
fixture using the new _suspend()/_resume() API so the outer CIVisibility
singleton is never stopped by tests that call enable()/disable() directly.
- Replace the _disable_ci_visibility fixture in api_client/_util.py with the
same _suspend()/_resume() pattern so TestTestVisibilityAPIClientBase tests
are properly isolated when running with --ddtrace in the outer session.
- Fix TestUploadGitMetadata: mock _get_git_dir() to avoid silent FAILED status
from git worktrees in Docker (the real call returns '' in that environment).
- Fix test_ci_visibility_service_enable_without_service: mock ci.tags() and
DEFAULT_SPAN_SERVICE_NAME so the expected service name is deterministic
regardless of the outer git/env context.
- Fix test_encoder.py: replace index-based span access with type-based lookup
(xdist does not guarantee ordering); remove PYTEST_XDIST_WORKER leakage.
pytest/testing suites:
- Wrap inline_run() in test_pytest.py with _suspend()/_resume() so the outer
CIVisibility instance survives inner sessions' disable()/enable() cycles.
- Remove PYTEST_XDIST_WORKER/PYTEST_XDIST_TESTRUNUID leakage from inner runs
so inline_run() tests are not misidentified as xdist workers.
- Add monkeypatch.delenv("PYTEST_XDIST_WORKER") in test_pytest_benchmark.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
✅ Tests 🎉 All green!❄️ No new flaky tests detected 🔗 Commit SHA: 7222b1d | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback! |
Performance SLOsComparing candidate gnufede/xdist-pr2-test-fixes (2c410e7) with baseline gnufede/xdist-pr1-plugin-fixes (3ea8971) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 102.818µs (SLO: <130.000µs 📉 -20.9%) vs baseline: +0.4% Memory: ✅ 43.841MB (SLO: <46.000MB -4.7%) vs baseline: +5.3% ✅ add_inplace_aspectTime: ✅ 101.045µs (SLO: <130.000µs 📉 -22.3%) vs baseline: -1.9% Memory: ✅ 43.995MB (SLO: <46.000MB -4.4%) vs baseline: +5.6% ✅ add_inplace_noaspectTime: ✅ 28.423µs (SLO: <40.000µs 📉 -28.9%) vs baseline: +1.1% Memory: ✅ 43.898MB (SLO: <46.000MB -4.6%) vs baseline: +5.6% ✅ add_noaspectTime: ✅ 48.538µs (SLO: <70.000µs 📉 -30.7%) vs baseline: -1.7% Memory: ✅ 43.902MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ bytearray_aspectTime: ✅ 259.879µs (SLO: <400.000µs 📉 -35.0%) vs baseline: -0.7% Memory: ✅ 43.862MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ bytearray_extend_aspectTime: ✅ 653.418µs (SLO: <800.000µs 📉 -18.3%) vs baseline: +1.5% Memory: ✅ 43.821MB (SLO: <46.000MB -4.7%) vs baseline: +5.2% ✅ bytearray_extend_noaspectTime: ✅ 277.900µs (SLO: <400.000µs 📉 -30.5%) vs baseline: +6.3% Memory: ✅ 43.884MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ bytearray_noaspectTime: ✅ 143.701µs (SLO: <300.000µs 📉 -52.1%) vs baseline: +0.8% Memory: ✅ 43.861MB (SLO: <46.000MB -4.7%) vs baseline: +5.2% ✅ bytes_aspectTime: ✅ 220.634µs (SLO: <300.000µs 📉 -26.5%) vs baseline: -1.8% Memory: ✅ 43.839MB (SLO: <46.000MB -4.7%) vs baseline: +4.9% ✅ bytes_noaspectTime: ✅ 135.815µs (SLO: <200.000µs 📉 -32.1%) vs baseline: +0.9% Memory: ✅ 43.884MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ bytesio_aspectTime: ✅ 3.889ms (SLO: <5.000ms 📉 -22.2%) vs baseline: +1.0% Memory: ✅ 43.903MB (SLO: <46.000MB -4.6%) vs baseline: +5.1% ✅ bytesio_noaspectTime: ✅ 321.436µs (SLO: <420.000µs 📉 -23.5%) vs baseline: +0.3% Memory: ✅ 43.885MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ capitalize_aspectTime: ✅ 88.572µs (SLO: <300.000µs 📉 -70.5%) vs baseline: ~same Memory: ✅ 43.948MB (SLO: <46.000MB -4.5%) vs baseline: +5.7% ✅ capitalize_noaspectTime: ✅ 264.454µs (SLO: <300.000µs 📉 -11.8%) vs baseline: +4.4% Memory: ✅ 43.960MB (SLO: <46.000MB -4.4%) vs baseline: +5.6% ✅ casefold_aspectTime: ✅ 91.445µs (SLO: <500.000µs 📉 -81.7%) vs baseline: +3.0% Memory: ✅ 43.922MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ casefold_noaspectTime: ✅ 305.597µs (SLO: <500.000µs 📉 -38.9%) vs baseline: -2.1% Memory: ✅ 43.942MB (SLO: <46.000MB -4.5%) vs baseline: +5.5% ✅ decode_aspectTime: ✅ 87.226µs (SLO: <100.000µs 📉 -12.8%) vs baseline: +0.5% Memory: ✅ 43.888MB (SLO: <46.000MB -4.6%) vs baseline: +5.5% ✅ decode_noaspectTime: ✅ 159.807µs (SLO: <210.000µs 📉 -23.9%) vs baseline: +2.9% Memory: ✅ 43.924MB (SLO: <46.000MB -4.5%) vs baseline: +5.5% ✅ encode_aspectTime: ✅ 84.590µs (SLO: <200.000µs 📉 -57.7%) vs baseline: +0.6% Memory: ✅ 43.864MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ encode_noaspectTime: ✅ 147.545µs (SLO: <200.000µs 📉 -26.2%) vs baseline: +3.3% Memory: ✅ 43.866MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ format_aspectTime: ✅ 14.570ms (SLO: <19.200ms 📉 -24.1%) vs baseline: -0.3% Memory: ✅ 43.977MB (SLO: <46.000MB -4.4%) vs baseline: +5.4% ✅ format_map_aspectTime: ✅ 16.355ms (SLO: <21.500ms 📉 -23.9%) vs baseline: -0.5% Memory: ✅ 43.872MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ format_map_noaspectTime: ✅ 368.406µs (SLO: <500.000µs 📉 -26.3%) vs baseline: -2.7% Memory: ✅ 43.885MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ format_noaspectTime: ✅ 314.590µs (SLO: <500.000µs 📉 -37.1%) vs baseline: -2.2% Memory: ✅ 43.905MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ index_aspectTime: ✅ 122.385µs (SLO: <300.000µs 📉 -59.2%) vs baseline: -3.5% Memory: ✅ 43.937MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ index_noaspectTime: ✅ 40.670µs (SLO: <300.000µs 📉 -86.4%) vs baseline: -1.1% Memory: ✅ 43.796MB (SLO: <46.000MB -4.8%) vs baseline: +5.1% ✅ join_aspectTime: ✅ 207.320µs (SLO: <300.000µs 📉 -30.9%) vs baseline: -3.4% Memory: ✅ 43.899MB (SLO: <46.000MB -4.6%) vs baseline: +5.5% ✅ join_noaspectTime: ✅ 140.756µs (SLO: <300.000µs 📉 -53.1%) vs baseline: -4.6% Memory: ✅ 43.863MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ ljust_aspectTime: ✅ 502.494µs (SLO: <700.000µs 📉 -28.2%) vs baseline: -2.4% Memory: ✅ 43.941MB (SLO: <46.000MB -4.5%) vs baseline: +5.5% ✅ ljust_noaspectTime: ✅ 264.629µs (SLO: <300.000µs 📉 -11.8%) vs baseline: -1.5% Memory: ✅ 43.927MB (SLO: <46.000MB -4.5%) vs baseline: +5.5% ✅ lower_aspectTime: ✅ 305.762µs (SLO: <500.000µs 📉 -38.8%) vs baseline: -1.9% Memory: ✅ 43.952MB (SLO: <46.000MB -4.5%) vs baseline: +5.6% ✅ lower_noaspectTime: ✅ 246.797µs (SLO: <300.000µs 📉 -17.7%) vs baseline: +3.6% Memory: ✅ 43.828MB (SLO: <46.000MB -4.7%) vs baseline: +5.4% ✅ lstrip_aspectTime: ✅ 0.347ms (SLO: <3.000ms 📉 -88.4%) vs baseline: 📈 +25.2% Memory: ✅ 43.957MB (SLO: <46.000MB -4.4%) vs baseline: +5.5% ✅ lstrip_noaspectTime: ✅ 0.179ms (SLO: <3.000ms 📉 -94.0%) vs baseline: +0.8% Memory: ✅ 43.922MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ modulo_aspectTime: ✅ 14.329ms (SLO: <18.750ms 📉 -23.6%) vs baseline: +0.6% Memory: ✅ 43.918MB (SLO: <46.000MB -4.5%) vs baseline: +5.2% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.794ms (SLO: <19.350ms 📉 -23.5%) vs baseline: -0.3% Memory: ✅ 43.964MB (SLO: <46.000MB -4.4%) vs baseline: +5.1% ✅ modulo_aspect_for_bytesTime: ✅ 14.462ms (SLO: <18.900ms 📉 -23.5%) vs baseline: +0.6% Memory: ✅ 43.918MB (SLO: <46.000MB -4.5%) vs baseline: +5.2% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.542ms (SLO: <19.150ms 📉 -24.1%) vs baseline: -0.7% Memory: ✅ 43.884MB (SLO: <46.000MB -4.6%) vs baseline: +5.1% ✅ modulo_noaspectTime: ✅ 0.366ms (SLO: <3.000ms 📉 -87.8%) vs baseline: -0.8% Memory: ✅ 43.910MB (SLO: <46.000MB -4.5%) vs baseline: +5.3% ✅ replace_aspectTime: ✅ 18.948ms (SLO: <24.000ms 📉 -21.1%) vs baseline: +3.0% Memory: ✅ 43.985MB (SLO: <46.000MB -4.4%) vs baseline: +5.7% ✅ replace_noaspectTime: ✅ 287.372µs (SLO: <400.000µs 📉 -28.2%) vs baseline: +1.5% Memory: ✅ 43.945MB (SLO: <46.000MB -4.5%) vs baseline: +5.5% ✅ repr_aspectTime: ✅ 318.117µs (SLO: <420.000µs 📉 -24.3%) vs baseline: -2.5% Memory: ✅ 43.877MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ repr_noaspectTime: ✅ 46.693µs (SLO: <90.000µs 📉 -48.1%) vs baseline: -0.5% Memory: ✅ 43.822MB (SLO: <46.000MB -4.7%) vs baseline: +5.3% ✅ rstrip_aspectTime: ✅ 383.187µs (SLO: <500.000µs 📉 -23.4%) vs baseline: -0.9% Memory: ✅ 43.917MB (SLO: <46.000MB -4.5%) vs baseline: +5.3% ✅ rstrip_noaspectTime: ✅ 187.311µs (SLO: <300.000µs 📉 -37.6%) vs baseline: +3.6% Memory: ✅ 43.886MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ slice_aspectTime: ✅ 186.605µs (SLO: <300.000µs 📉 -37.8%) vs baseline: +1.3% Memory: ✅ 43.910MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ slice_noaspectTime: ✅ 54.229µs (SLO: <90.000µs 📉 -39.7%) vs baseline: +0.5% Memory: ✅ 43.852MB (SLO: <46.000MB -4.7%) vs baseline: +5.3% ✅ stringio_aspectTime: ✅ 3.913ms (SLO: <5.000ms 📉 -21.7%) vs baseline: +0.7% Memory: ✅ 43.942MB (SLO: <46.000MB -4.5%) vs baseline: +5.6% ✅ stringio_noaspectTime: ✅ 356.886µs (SLO: <500.000µs 📉 -28.6%) vs baseline: +1.0% Memory: ✅ 43.890MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ strip_aspectTime: ✅ 280.371µs (SLO: <350.000µs 📉 -19.9%) vs baseline: +0.5% Memory: ✅ 43.936MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ strip_noaspectTime: ✅ 182.302µs (SLO: <240.000µs 📉 -24.0%) vs baseline: +3.6% Memory: ✅ 43.813MB (SLO: <46.000MB -4.8%) vs baseline: +5.2% ✅ swapcase_aspectTime: ✅ 345.243µs (SLO: <500.000µs 📉 -31.0%) vs baseline: -2.2% Memory: ✅ 43.907MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ swapcase_noaspectTime: ✅ 282.906µs (SLO: <400.000µs 📉 -29.3%) vs baseline: +3.1% Memory: ✅ 43.867MB (SLO: <46.000MB -4.6%) vs baseline: +5.1% ✅ title_aspectTime: ✅ 332.067µs (SLO: <500.000µs 📉 -33.6%) vs baseline: -1.8% Memory: ✅ 43.890MB (SLO: <46.000MB -4.6%) vs baseline: +5.5% ✅ title_noaspectTime: ✅ 266.974µs (SLO: <400.000µs 📉 -33.3%) vs baseline: +1.6% Memory: ✅ 43.925MB (SLO: <46.000MB -4.5%) vs baseline: +5.6% ✅ translate_aspectTime: ✅ 565.142µs (SLO: <700.000µs 📉 -19.3%) vs baseline: 📈 +12.7% Memory: ✅ 43.825MB (SLO: <46.000MB -4.7%) vs baseline: +4.9% ✅ translate_noaspectTime: ✅ 431.725µs (SLO: <500.000µs 📉 -13.7%) vs baseline: ~same Memory: ✅ 43.851MB (SLO: <46.000MB -4.7%) vs baseline: +5.0% ✅ upper_aspectTime: ✅ 305.002µs (SLO: <500.000µs 📉 -39.0%) vs baseline: -2.7% Memory: ✅ 43.979MB (SLO: <46.000MB -4.4%) vs baseline: +5.7% ✅ upper_noaspectTime: ✅ 248.004µs (SLO: <400.000µs 📉 -38.0%) vs baseline: +4.0% Memory: ✅ 43.892MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 525.870µs (SLO: <700.000µs 📉 -24.9%) vs baseline: 📈 +21.8% Memory: ✅ 43.813MB (SLO: <46.000MB -4.8%) vs baseline: +5.3% ✅ ospathbasename_noaspectTime: ✅ 446.925µs (SLO: <700.000µs 📉 -36.2%) vs baseline: +1.6% Memory: ✅ 43.847MB (SLO: <46.000MB -4.7%) vs baseline: +5.3% ✅ ospathjoin_aspectTime: ✅ 637.704µs (SLO: <700.000µs -8.9%) vs baseline: +1.0% Memory: ✅ 43.785MB (SLO: <46.000MB -4.8%) vs baseline: +5.2% ✅ ospathjoin_noaspectTime: ✅ 644.430µs (SLO: <700.000µs -7.9%) vs baseline: -0.4% Memory: ✅ 43.884MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ ospathnormcase_aspectTime: ✅ 363.521µs (SLO: <700.000µs 📉 -48.1%) vs baseline: +3.5% Memory: ✅ 43.862MB (SLO: <46.000MB -4.6%) vs baseline: +5.1% ✅ ospathnormcase_noaspectTime: ✅ 363.703µs (SLO: <700.000µs 📉 -48.0%) vs baseline: ~same Memory: ✅ 43.885MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ ospathsplit_aspectTime: ✅ 500.259µs (SLO: <700.000µs 📉 -28.5%) vs baseline: +1.9% Memory: ✅ 43.911MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ ospathsplit_noaspectTime: ✅ 504.813µs (SLO: <700.000µs 📉 -27.9%) vs baseline: -0.1% Memory: ✅ 43.886MB (SLO: <46.000MB -4.6%) vs baseline: +5.6% ✅ ospathsplitdrive_aspectTime: ✅ 383.926µs (SLO: <700.000µs 📉 -45.2%) vs baseline: +1.7% Memory: ✅ 43.896MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ ospathsplitdrive_noaspectTime: ✅ 72.761µs (SLO: <700.000µs 📉 -89.6%) vs baseline: +0.4% Memory: ✅ 43.802MB (SLO: <46.000MB -4.8%) vs baseline: +5.2% ✅ ospathsplitext_aspectTime: ✅ 476.819µs (SLO: <700.000µs 📉 -31.9%) vs baseline: +2.7% Memory: ✅ 43.891MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ ospathsplitext_noaspectTime: ✅ 478.789µs (SLO: <700.000µs 📉 -31.6%) vs baseline: +1.4% Memory: ✅ 43.944MB (SLO: <46.000MB -4.5%) vs baseline: +5.7% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 2.345µs (SLO: <20.000µs 📉 -88.3%) vs baseline: 📈 +10.6% Memory: ✅ 36.274MB (SLO: <38.000MB -4.5%) vs baseline: +5.4% ✅ 1-count-metrics-100-timesTime: ✅ 154.569µs (SLO: <220.000µs 📉 -29.7%) vs baseline: +1.5% Memory: ✅ 36.235MB (SLO: <38.000MB -4.6%) vs baseline: +5.3% ✅ 1-distribution-metric-1-timesTime: ✅ 2.415µs (SLO: <20.000µs 📉 -87.9%) vs baseline: -4.6% Memory: ✅ 36.235MB (SLO: <38.000MB -4.6%) vs baseline: +5.4% ✅ 1-distribution-metrics-100-timesTime: ✅ 164.077µs (SLO: <230.000µs 📉 -28.7%) vs baseline: -1.3% Memory: ✅ 36.255MB (SLO: <38.000MB -4.6%) vs baseline: +5.4% ✅ 1-gauge-metric-1-timesTime: ✅ 1.999µs (SLO: <20.000µs 📉 -90.0%) vs baseline: -0.5% Memory: ✅ 36.196MB (SLO: <38.000MB -4.7%) vs baseline: +5.5% ✅ 1-gauge-metrics-100-timesTime: ✅ 138.455µs (SLO: <150.000µs -7.7%) vs baseline: +0.3% Memory: ✅ 36.274MB (SLO: <38.000MB -4.5%) vs baseline: +5.2% ✅ 1-rate-metric-1-timesTime: ✅ 2.273µs (SLO: <20.000µs 📉 -88.6%) vs baseline: +1.3% Memory: ✅ 36.215MB (SLO: <38.000MB -4.7%) vs baseline: +5.0% ✅ 1-rate-metrics-100-timesTime: ✅ 168.595µs (SLO: <250.000µs 📉 -32.6%) vs baseline: +0.8% Memory: ✅ 36.196MB (SLO: <38.000MB -4.7%) vs baseline: +5.1% ✅ 100-count-metrics-100-timesTime: ✅ 15.548ms (SLO: <22.000ms 📉 -29.3%) vs baseline: +0.9% Memory: ✅ 36.215MB (SLO: <38.000MB -4.7%) vs baseline: +5.0% ✅ 100-distribution-metrics-100-timesTime: ✅ 1.726ms (SLO: <2.550ms 📉 -32.3%) vs baseline: -1.7% Memory: ✅ 36.196MB (SLO: <38.000MB -4.7%) vs baseline: +5.1% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.404ms (SLO: <1.550ms -9.4%) vs baseline: -0.5% Memory: ✅ 36.274MB (SLO: <38.000MB -4.5%) vs baseline: +5.5% ✅ 100-rate-metrics-100-timesTime: ✅ 1.741ms (SLO: <2.550ms 📉 -31.7%) vs baseline: +1.6% Memory: ✅ 36.215MB (SLO: <38.000MB -4.7%) vs baseline: +5.3% ✅ flush-1-metricTime: ✅ 3.629µs (SLO: <20.000µs 📉 -81.9%) vs baseline: +1.5% Memory: ✅ 36.667MB (SLO: <38.000MB -3.5%) vs baseline: +6.5% ✅ flush-100-metricsTime: ✅ 174.022µs (SLO: <250.000µs 📉 -30.4%) vs baseline: -1.7% Memory: ✅ 36.608MB (SLO: <38.000MB -3.7%) vs baseline: +5.2% ✅ flush-1000-metricsTime: ✅ 2.197ms (SLO: <2.500ms 📉 -12.1%) vs baseline: -0.2% Memory: ✅ 37.041MB (SLO: <38.750MB -4.4%) vs baseline: +5.3% 🟡 Near SLO Breach (5 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 19.735ms (SLO: <22.300ms 📉 -11.5%) vs baseline: +0.4% Memory: ✅ 69.325MB (SLO: <73.500MB -5.7%) vs baseline: +5.1% ✅ exception-replay-enabledTime: ✅ 1.370ms (SLO: <1.450ms -5.5%) vs baseline: ~same Memory: ✅ 67.672MB (SLO: <71.500MB -5.4%) vs baseline: +5.2% ✅ iastTime: ✅ 19.664ms (SLO: <22.250ms 📉 -11.6%) vs baseline: -0.2% Memory: ✅ 69.374MB (SLO: <75.000MB -7.5%) vs baseline: +5.0% ✅ profilerTime: ✅ 15.149ms (SLO: <16.550ms -8.5%) vs baseline: -0.2% Memory: ✅ 59.971MB (SLO: <61.000MB 🟡 -1.7%) vs baseline: +5.4% ✅ resource-renamingTime: ✅ 19.611ms (SLO: <21.750ms -9.8%) vs baseline: +0.4% Memory: ✅ 69.499MB (SLO: <73.500MB -5.4%) vs baseline: +5.3% ✅ span-code-originTime: ✅ 20.155ms (SLO: <28.200ms 📉 -28.5%) vs baseline: +1.2% Memory: ✅ 69.512MB (SLO: <75.000MB -7.3%) vs baseline: +5.4% ✅ tracerTime: ✅ 19.804ms (SLO: <21.750ms -8.9%) vs baseline: +0.9% Memory: ✅ 69.393MB (SLO: <75.000MB -7.5%) vs baseline: +5.0% ✅ tracer-and-profilerTime: ✅ 20.973ms (SLO: <23.500ms 📉 -10.8%) vs baseline: -0.2% Memory: ✅ 71.349MB (SLO: <75.000MB -4.9%) vs baseline: +5.1% ✅ tracer-dont-create-db-spansTime: ✅ 19.718ms (SLO: <21.500ms -8.3%) vs baseline: +0.4% Memory: ✅ 69.471MB (SLO: <75.000MB -7.4%) vs baseline: +5.3% ✅ tracer-minimalTime: ✅ 16.647ms (SLO: <17.500ms -4.9%) vs baseline: -0.1% Memory: ✅ 69.282MB (SLO: <75.000MB -7.6%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 19.593ms (SLO: <21.750ms -9.9%) vs baseline: +0.6% Memory: ✅ 69.351MB (SLO: <72.500MB -4.3%) vs baseline: +5.1% ✅ tracer-no-cachesTime: ✅ 17.471ms (SLO: <19.650ms 📉 -11.1%) vs baseline: ~same Memory: ✅ 69.294MB (SLO: <75.000MB -7.6%) vs baseline: +5.1% ✅ tracer-no-databasesTime: ✅ 19.260ms (SLO: <20.100ms -4.2%) vs baseline: +0.2% Memory: ✅ 69.264MB (SLO: <75.000MB -7.6%) vs baseline: +4.9% ✅ tracer-no-middlewareTime: ✅ 19.274ms (SLO: <21.500ms 📉 -10.4%) vs baseline: -0.6% Memory: ✅ 69.353MB (SLO: <75.000MB -7.5%) vs baseline: +5.1% ✅ tracer-no-templatesTime: ✅ 19.624ms (SLO: <22.000ms 📉 -10.8%) vs baseline: +1.2% Memory: ✅ 69.424MB (SLO: <73.500MB -5.5%) vs baseline: +5.2% 🟡 otelspan - 22/22✅ add-eventTime: ✅ 40.821ms (SLO: <47.150ms 📉 -13.4%) vs baseline: -0.5% Memory: ✅ 41.374MB (SLO: <47.000MB 📉 -12.0%) vs baseline: +5.8% ✅ add-metricsTime: ✅ 235.650ms (SLO: <344.800ms 📉 -31.7%) vs baseline: -0.5% Memory: ✅ 45.623MB (SLO: <47.500MB -4.0%) vs baseline: +5.4% ✅ add-tagsTime: ✅ 278.594ms (SLO: <330.000ms 📉 -15.6%) vs baseline: +2.7% Memory: ✅ 45.704MB (SLO: <47.500MB -3.8%) vs baseline: +5.2% ✅ get-contextTime: ✅ 83.711ms (SLO: <92.350ms -9.4%) vs baseline: ~same Memory: ✅ 41.602MB (SLO: <46.500MB 📉 -10.5%) vs baseline: +5.3% ✅ is-recordingTime: ✅ 39.030ms (SLO: <44.500ms 📉 -12.3%) vs baseline: -0.5% Memory: ✅ 41.221MB (SLO: <47.500MB 📉 -13.2%) vs baseline: +5.6% ✅ record-exceptionTime: ✅ 61.272ms (SLO: <67.650ms -9.4%) vs baseline: ~same Memory: ✅ 41.751MB (SLO: <47.000MB 📉 -11.2%) vs baseline: +5.3% ✅ set-statusTime: ✅ 45.130ms (SLO: <50.400ms 📉 -10.5%) vs baseline: ~same Memory: ✅ 41.187MB (SLO: <47.000MB 📉 -12.4%) vs baseline: +5.7% ✅ startTime: ✅ 39.931ms (SLO: <44.500ms 📉 -10.3%) vs baseline: +3.5% Memory: ✅ 41.040MB (SLO: <47.000MB 📉 -12.7%) vs baseline: +5.1% ✅ start-finishTime: ✅ 90.423ms (SLO: <91.000ms 🟡 -0.6%) vs baseline: +0.4% Memory: ✅ 38.673MB (SLO: <46.500MB 📉 -16.8%) vs baseline: +4.9% ✅ start-finish-telemetryTime: ✅ 91.905ms (SLO: <92.000ms 🟡 -0.1%) vs baseline: ~same Memory: ✅ 38.732MB (SLO: <46.500MB 📉 -16.7%) vs baseline: +5.2% ✅ update-nameTime: ✅ 40.206ms (SLO: <45.150ms 📉 -10.9%) vs baseline: -0.4% Memory: ✅ 41.247MB (SLO: <47.000MB 📉 -12.2%) vs baseline: +5.2% 🟡 recursivecomputation - 8/8✅ deepTime: ✅ 312.624ms (SLO: <320.950ms -2.6%) vs baseline: +0.3% Memory: ✅ 37.434MB (SLO: <38.750MB -3.4%) vs baseline: +5.4% ✅ deep-profiledTime: ✅ 328.253ms (SLO: <359.150ms -8.6%) vs baseline: -0.2% Memory: ✅ 43.765MB (SLO: <46.000MB -4.9%) vs baseline: +5.5% ✅ mediumTime: ✅ 7.405ms (SLO: <7.450ms 🟡 -0.6%) vs baseline: -0.5% Memory: ✅ 36.176MB (SLO: <38.000MB -4.8%) vs baseline: +5.4% ✅ shallowTime: ✅ 1.048ms (SLO: <1.050ms 🟡 -0.2%) vs baseline: +1.5% Memory: ✅ 36.215MB (SLO: <38.000MB -4.7%) vs baseline: +5.1% 🟡 span - 26/26✅ add-eventTime: ✅ 19.593ms (SLO: <22.500ms 📉 -12.9%) vs baseline: -1.4% Memory: ✅ 38.426MB (SLO: <53.000MB 📉 -27.5%) vs baseline: +5.5% ✅ add-metricsTime: ✅ 89.918ms (SLO: <93.500ms -3.8%) vs baseline: +0.8% Memory: ✅ 42.861MB (SLO: <53.000MB 📉 -19.1%) vs baseline: +5.2% ✅ add-tagsTime: ✅ 149.107ms (SLO: <155.000ms -3.8%) vs baseline: +0.6% Memory: ✅ 42.953MB (SLO: <53.000MB 📉 -19.0%) vs baseline: +5.5% ✅ get-contextTime: ✅ 18.667ms (SLO: <20.500ms -8.9%) vs baseline: -2.2% Memory: ✅ 38.342MB (SLO: <53.000MB 📉 -27.7%) vs baseline: +5.6% ✅ is-recordingTime: ✅ 18.762ms (SLO: <20.500ms -8.5%) vs baseline: -1.8% Memory: ✅ 38.424MB (SLO: <53.000MB 📉 -27.5%) vs baseline: +5.7% ✅ record-exceptionTime: ✅ 38.512ms (SLO: <41.000ms -6.1%) vs baseline: -0.2% Memory: ✅ 38.869MB (SLO: <53.000MB 📉 -26.7%) vs baseline: +5.6% ✅ set-statusTime: ✅ 20.663ms (SLO: <22.000ms -6.1%) vs baseline: -1.4% Memory: ✅ 38.255MB (SLO: <53.000MB 📉 -27.8%) vs baseline: +5.4% ✅ startTime: ✅ 19.715ms (SLO: <20.500ms -3.8%) vs baseline: +4.7% Memory: ✅ 38.336MB (SLO: <53.000MB 📉 -27.7%) vs baseline: +5.6% ✅ start-finishTime: ✅ 58.077ms (SLO: <58.500ms 🟡 -0.7%) vs baseline: -0.4% Memory: ✅ 36.235MB (SLO: <38.000MB -4.6%) vs baseline: +5.3% ✅ start-finish-telemetryTime: ✅ 59.200ms (SLO: <60.000ms 🟡 -1.3%) vs baseline: -0.7% Memory: ✅ 36.255MB (SLO: <38.000MB -4.6%) vs baseline: +5.9% ✅ start-finish-traceid128Time: ✅ 60.551ms (SLO: <62.000ms -2.3%) vs baseline: -0.6% Memory: ✅ 36.294MB (SLO: <38.000MB -4.5%) vs baseline: +5.4% ✅ start-traceid128Time: ✅ 18.776ms (SLO: <22.500ms 📉 -16.5%) vs baseline: -0.8% Memory: ✅ 38.320MB (SLO: <53.000MB 📉 -27.7%) vs baseline: +5.3% ✅ update-nameTime: ✅ 19.233ms (SLO: <22.000ms 📉 -12.6%) vs baseline: -1.5% Memory: ✅ 38.408MB (SLO: <53.000MB 📉 -27.5%) vs baseline: +5.6% 🟡 tracer - 6/6✅ largeTime: ✅ 33.067ms (SLO: <32.950ms +0.4%) vs baseline: -0.3% Memory: ✅ 37.827MB (SLO: <39.250MB -3.6%) vs baseline: +6.0% ✅ mediumTime: ✅ 3.343ms (SLO: <3.500ms -4.5%) vs baseline: ~same Memory: ✅ 36.215MB (SLO: <38.750MB -6.5%) vs baseline: +5.4% ✅ smallTime: ✅ 385.459µs (SLO: <390.000µs 🟡 -1.2%) vs baseline: +3.4% Memory: ✅ 36.176MB (SLO: <38.750MB -6.6%) vs baseline: +5.3%
|
Codeowners resolved as |
1fd4792 to
2c410e7
Compare
…on (#17457) ## Summary Foundational production-code changes that allow the outer pytest session to run with `--ddtrace` and xdist without corrupting inner `inline_run()` sessions. Three sources of module-level bleed-through are fixed: - **`CIVisibility._suspend()`/`_resume()`** — removes/restores the active singleton without calling `stop()`, so the outer session's tracer keeps running while inner sessions get a clean slate. `enable()` and `disable()` are completely unchanged. - **`is_xdist_worker_env()`** — replaces the frozen-at-import-time `PYTEST_XDIST_WORKER_VALUE` constant with a live `os.environ` read, so inner `inline_run()` sessions (which clear `PYTEST_XDIST_WORKER` from the env) correctly detect they are not workers. - **Module-level state reset in `pytest_configure`** — resets `skipped_suites` and `skip_pytest_runtest_protocol` for each new main-process session (guarded by `not hasattr(config, "workerinput")`), preventing stale state from accumulating across `inline_run()` calls. ## Test plan - [x] `ci_visibility::pytest` suite passes with `--ddtrace -n auto` (validated in follow-up PR) - [x] `ci_visibility::ci_visibility` suite passes with `--ddtrace -n auto` (validated in follow-up PR) - [x] CI green ## Stacked PRs 1. **This PR** — plugin changes 2. #17458 test isolation fixes (depends on this PR) 3. #17459 enable `--ddtrace -n auto` in riotfile (depends on PR 1 + 2) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: federico.mon <federico.mon@datadoghq.com>
… suites (#17459) ## Summary Switch `pytest`, `ci_visibility`, and `testing` suite commands from `--no-ddtrace` to `--ddtrace -n auto`. **Depends on:** - #17457 — plugin changes - #17458 — test isolation fixes Snapshot sub-venvs (`pytest-snapshot`, `ci_visibility-snapshot`) are kept without pytest-xdist as the testagent would receive spans in parallel and that breaks the tests. ## Test plan - [x] All three suites pass in CI with `--ddtrace -n auto` - [x] Snapshot sub-venvs also pass with `--ddtrace` (without `-n auto`) - [x] CI green ## Stacked PRs 1. #17457 — production plugin fixes 2. #17458 — test isolation fixes *(base of this PR)* 3. **This PR** — enable `--ddtrace -n auto` in riotfile 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: datadog-prod-us1-4[bot] <266080418+datadog-prod-us1-4[bot]@users.noreply.github.com> Co-authored-by: federico.mon <federico.mon@datadoghq.com>
Summary
Test isolation fixes that make the
ci_visibility,pytest, andtestingtest suites robust when running with--ddtrace -n autoin the outer session.Depends on: #17457 (
_suspend()/_resume()API)ci_visibility suite
tests/ci_visibility/conftest.pywith an autouse_ci_visibility_isolationfixture using_suspend()/_resume()so the outer CIVisibility singleton is never stopped by tests that callenable()/disable()directly._disable_ci_visibilityfixture inapi_client/_util.pywith the same pattern.TestUploadGitMetadata: mock_get_git_dir()to avoid silentFAILEDstatus from git worktrees in Docker.test_ci_visibility_service_enable_without_service: mockci.tags()andDEFAULT_SPAN_SERVICE_NAMEso the expected service name is deterministic regardless of outer git/env context.test_encoder.py: replace index-based span access with type-based lookup (xdist does not guarantee ordering); removePYTEST_XDIST_WORKERenv leakage.pytest/testing suites
inline_run()intest_pytest.pywith_suspend()/_resume()so the outer CIVisibility instance survives inner sessions'disable()/enable()cycles.PYTEST_XDIST_WORKER/PYTEST_XDIST_TESTRUNUIDleakage from inner runs soinline_run()tests are not misidentified as xdist workers.monkeypatch.delenv("PYTEST_XDIST_WORKER")intest_pytest_benchmark.py.Test plan
ci_visibility::ci_visibilityvenvseef30c1,1f30a84passci_visibility::pytestvenvs16b0319,407c34dpassStacked PRs
--ddtrace -n autoin riotfile (depends on this PR)🤖 Generated with Claude Code