Skip to content

BOT: Fix #942: Add plot_discrimination() for binary forecasts#1079

Open
nikosbosse wants to merge 7 commits intomainfrom
fix/942-discrimination-plot-binary
Open

BOT: Fix #942: Add plot_discrimination() for binary forecasts#1079
nikosbosse wants to merge 7 commits intomainfrom
fix/942-discrimination-plot-binary

Conversation

@nikosbosse
Copy link
Copy Markdown
Collaborator

Summary

  • Adds a new plot_discrimination() function that visualises the discrimination ability of binary forecasts by plotting overlapping density curves of predicted probabilities, stratified by observed outcome level
  • Includes 7 tests covering basic usage, faceting, plain data.frame input, missing columns, single-model data, separation verification, and edge cases
  • Follows existing plot function conventions (ensure_data.table(), check_columns_present(), theme_scoringutils())

Closes #942

Root cause

No binary-specific plot functions existed in the codebase. The forecast_binary class and example_binary dataset provided the data structures but there was no way to visualise discrimination ability.

What changed

  • New file R/plot-discrimination.R with plot_discrimination() — accepts a data.frame/data.table with observed (factor) and predicted (numeric) columns, renders overlapping density plots via ggplot2
  • New test file tests/testthat/test-plot_discrimination.R with 7 test cases and 2 visual regression snapshots
  • NAMESPACE updated to export the new function

Test plan

  • All 7 new tests pass
  • Full test suite passes (689 tests, 0 failures)
  • R CMD check: 0 errors, 0 warnings, 2 notes (pre-existing)

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.00%. Comparing base (d4bbd6e) to head (8dfd005).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1079      +/-   ##
==========================================
+ Coverage   97.98%   98.00%   +0.02%     
==========================================
  Files          37       38       +1     
  Lines        1984     2006      +22     
==========================================
+ Hits         1944     1966      +22     
  Misses         40       40              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Collaborator Author

@nikosbosse nikosbosse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLAUDE: Approved. Clean implementation of plot_discrimination() that follows all existing package conventions (ensure_data.table, check_columns_present, theme_scoringutils). All 7 tests match specifications and pass. Only minor nitpick: unused @importFrom checkmate assert_data_frame import — harmless but unnecessary.

@nikosbosse nikosbosse marked this pull request as draft February 13, 2026 08:27
@nikosbosse nikosbosse changed the title Fix #942: Add plot_discrimination() for binary forecasts BOT: Fix #942: Add plot_discrimination() for binary forecasts Feb 13, 2026
- Use expect_gte() instead of expect_true(x >= y) to satisfy
  expect_comparison_linter
- Skip vdiffr snapshot tests on R < 4.2 where density estimation
  produces different SVG output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@nikosbosse
Copy link
Copy Markdown
Collaborator Author

Here is an example that it produced:

devtools::load_all()
  library(ggplot2)
  p <- plot_discrimination(na.omit(example_binary))
  ggsave("/tmp/plot_discrimination.png", p, width = 8, height = 5, dpi = 150)
image

I like it, though I think we maybe would want a histogram version as well.

Default to histogram with density on y-axis; density curves available
via type = "density". Removes old vdiffr snapshots that only worked
with one plot type and updates tests accordingly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@nikosbosse
Copy link
Copy Markdown
Collaborator Author

nikosbosse commented Apr 5, 2026

image

nikosbosse and others added 3 commits April 5, 2026 16:48
Allows users to pass arguments like bins, binwidth, bw, or adjust
directly through to the underlying geom.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Proportions are computed within each observed level so both groups
sum to 1, making the distribution shapes directly comparable
regardless of sample size differences.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@nikosbosse nikosbosse requested a review from seabbs April 5, 2026 14:56
@nikosbosse nikosbosse marked this pull request as ready for review April 5, 2026 14:56
Add nolint annotations for false positive object_usage_linter
warnings caused by ... forwarding, and add group to globalVariables.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread R/plot-discrimination.R
#'
#' @param forecast A data.table (or data.frame) containing at least columns
#' `observed` (factor with two levels) and `predicted` (numeric probabilities
#' between 0 and 1). Typically a `forecast_binary` object or the output of
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or? it is the output? Do we want to support input that isn't a forecast binary object? Based on package flow I would assume we are mving towards not doing so.

Comment thread R/plot-discrimination.R
labs(y = "Density")
} else {
plot <- plot +
geom_histogram( # nolint object_usage_linter
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lintr really works this badly with standard dply?

Copy link
Copy Markdown
Contributor

@seabbs seabbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks pretty good. My main concern is do we want to support non forecast objects here or insist that they are first as_ before use in these functions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Discrimination plots for binary forecasts

2 participants