feat: support custom dataset display names in reports#1860
Open
Nik-Reddy wants to merge 2 commits intoevidentlyai:mainfrom
Open
feat: support custom dataset display names in reports#1860Nik-Reddy wants to merge 2 commits intoevidentlyai:mainfrom
Nik-Reddy wants to merge 2 commits intoevidentlyai:mainfrom
Conversation
Addresses issue evidentlyai#820: users can now override the default 'current' and 'reference' dataset labels shown across all report and test suite visualizations. Added current_name and reference_name fields to RenderOptions with backward-compatible defaults. Updated the rendering pipeline so that Report and TestSuite propagate these options to all renderers. Changed shared rendering utilities (render_utils, html_widgets, tests/utils, visualization) and individual metric renderers to use the configurable names in plot traces, subplot titles, table headers, and counter labels. Usage: report = Report( metrics=[...], options={'render': {'current_name': 'ModelA', 'reference_name': 'ModelB'}} )
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #820
I kept running into this when generating reports for A/B tests. The labels always say "Current" and "Reference" which doesn't really tell stakeholders anything useful. You end up having to explain what's what every time you share a report.
This adds
current_nameandreference_nameoptions toRenderOptionsso you can set them to something meaningful like "Model v2" / "Model v1" or "Treatment" / "Control".Usage
What changed
The new fields live in
RenderOptionswith defaults of "Current" and "Reference" to match existing behavior. The configurable names flow throughbase_renderer.pyintohtml_widgets.py,render_utils.py, and 14 metric renderer files that previously had the strings hardcoded.report.pyandtest_suite.pynow pass render_options through to renderers.Tests in
tests/options/test_custom_dataset_names.pycover defaults, custom names, backward compatibility, and edge cases (10 tests total).Backward compatibility
Defaults are "Current" and "Reference" (capitalized), matching what users see today. Nothing changes unless you explicitly set these options.
23 files changed, 203 additions, 53 deletions.