[new release] dune (18 packages) (3.23.0~alpha2)

[shonfeder]

Fast, portable, and opinionated build system

• Project page: https://github.com/ocaml/dune
• Documentation: https://dune.readthedocs.io/

CHANGES:

Fixed

• Auto-inject "menhir" {>= "20180523"} into generated opam files when the
  menhir extension is used. Dune's menhir rules rely on --infer-write-query
  and --infer-read-reply, which require at least this version; without the
  lower bound, builds fail with older menhir installations.
  (Add correct lower bound to menhir dune#10707, @robinbb)

• Fix --display=quiet not suppressing "Entering directory" and "Leaving
  directory" messages when using --root. These messages are now deferred
  until there is actual output, so silent builds produce no noise.
  (feat: don't print entering/leaving directory message when no ouput dune#12974, fixes --display=quiet has no effect on --root-changing messages dune#12854, @Alizter)

• Fix an internal error when a module is shared between a rocq.theory and
  rocq.extraction stanza. The error now includes a hint pointing to the
  conflicting stanza. (fix(rocq): add error handling and hints for Rocq modules duplication dune#13733, @Durbatuluk1701)

• Fix cookies defined on ppx_rewriter being lost when that rewriter was used
  as a dependency of another ppx_rewriter. (fix(ppx): collect cookies from transitive ppx dependencies dune#13737, fixes Interaction between nested ppx_rewriters and cookies dune#3426, @Alizter)

• Improve opam file generation for packages that set (dir ..). Such packages
  will now have a test target that is limited to this directory. (fix: generate a better test target for packages with (dir ..) dune#13778, @rgrinberg)

• Stop duplicating dune diagnostics to subscribers over RPC (fix(rpc): stop duplicating diagnostics dune#13816,
  @rgrinberg)

• Fix dependency cycle when using the package with a library
  conditionally enabled via enabled_if. (fix(scope): fix enabled_if evaluation in lib_entries_of_package dune#13833, @toots)

• Fix underspecification of dependencies in the sandbox for modules with
  interfaces. ( fix(root_module): missing dependencies in the sandbox dune#13842, @anmonteiro)

• Honor sandbox settings specified inside (:include ..) expressions in the deps field
  (sandbox: honor sandbox config from included deps dune#13898, @rgrinberg)

• Fix dune pkg lock failing when a pin stanza contains a file:// URL with
  a relative path outside the workspace (fix(pkg): allow locking relative path outside the workspace dune#13915, fixes pkg: error when locking when a pin stanza contains a relative path outside the workspace dune#10254, @shunueda)

• Use all stat attributes to detect patch back source tree changes (fix: use regular stat for patch back dune#13986, @rgrinberg)

• Use device and inode number for invalidating cached digests (fix: use device/inode for digest caching dune#13991, @rgrinberg)

• Stop trying to set SO_REUSEADDR on unix sockets when setting up dune rpc
  (fix(rpc): do not try to set SO_REUSEADDR on unix servers dune#14056, @rgrinberg)

• Fix autolocking to correctly detect changes to (depends) in watch mode
  (fix(pkg): autolocking detects (depends) changes in watch mode dune#14066, fixes Auto rebuild when new dependencies added to dune-project and running in watch mode dune#13234, @Alizter)

• Fix incremental builds for libraries using (wrapped (transition ...)).
  The compat shim modules were never recompiled when the inner module's
  interface changed, causing "inconsistent assumptions" errors.
  (fix: depend on inner module of wrapped compat module  dune#14090, fixes Incremental builds with (wrapped (transition ...)) are broken dune#14089, @Alizter)

• Fix Rocq configuration detection to use rocq c --config subcommand
  instead of rocq --config (Rocq --config Fix dune#14093, fixes Rocq rules do not support Rocq 9.0 dune#13774, @Durbatuluk1701)

• Bump dune rpc's request pending request limit 10 to 100 (fix(rpc): bump backlog to 100 dune#14094, @rginberg)

• Invalidate stale cached digests in all build commands (fix: move stale timestamp invalidation to the top dune#14126, @rgrinberg)

• Fix root_module generating duplicate module definitions when a
  findlib sub-package shares a directory with its parent (e.g.,
  logs.lwt alongside logs). (fix: deduplicate entry modules in root_module stanza dune#14135, fixes root_module generates duplicate module definition when logs.lwt is a dependency. dune#6148, @robinbb)

• Fix dune subst prepending a duplicate version: field to opam
  files that already contain one, instead of replacing it in place.
  (fix: replace existing version field in opam files during dune subst dune#14136, fixes dune subst can add a duplicate version field in opam dune#878, @robinbb)

• Fix @ocaml-index alias being generated for all contexts, causing build
  errors in multi-context workspaces where some libraries are disabled in
  non-default contexts. The alias is now only generated for the merlin context.
  (fix: only generate @ocaml-index alias for merlin-enabled contexts dune#14137, fixes dune build @ocaml-index builds too many files dune#12007, @robinbb)

• Remove the warning about the deprecation of the coq stanza for
  dune lang before 3.21 since it has been introduced in this
  version ([Coq] disable deprecated_coq_lang before lang 3.21 dune#14187, @bobot)

• $ dune init now generates projects with correct .opam files (fix: generate correct .opam file in dune init dune#14192, @rgrinberg)

• Send SIGTERM before SIGKILL when cancelling child processes in watch
  mode, giving cleanup handlers a chance to run before escalating.
  (Send SIGTERM before SIGKILL on watch-mode cancellation dune#14224, Send SIGTERM before SIGKILL when shutting down child processes dune#14170, fixes dune forwards signal to process but sends SIGKILL right away dune#2445, @robinbb)

• Correctly specify dependencies for generate_runner in inline tests (fix: track inline test runner input deps dune#14276, @rgrinberg)

• Fix duplicate dune version bounds in generated opam files. When users
  declare (dune (>= X.Y)) matching or exceeding the (lang dune X.Y)
  version, the generated opam depends no longer contains redundant
  constraints like {>= "2.7" & >= "2.7"}. (Duplicate bounds on autogenerated opam files dune#3916, Redundant dune version constraint dune#11106, @robinbb)

• Fix inline tests not being executed for byte-only libraries. When a library
  has (modes byte), the inline test runner is now linked in byte mode so the
  library's test code is included. (dune runtest compiles but does not execute (inline_tests) (modes byte) dune#9757, @robinbb)

Added

• Add support for c_library_flags in foreign_stubs (Add c_library_flags to foreign_library dune#13484, @madroach)

• Move the management of Jsoo config details out of dune ( Move the management of Jsoo config details out of dune dune#13613, @vouillon)

• Js_of_ocaml: share standalone runtimes (Js_of_ocaml: share standalone runtimes dune#13621, @vouillon)

• Allow multiple (dirs ..) stanzas in the same Dune file. Starting with
  (lang dune 3.23), Dune takes the union of the specified directories
  (feat: allow multiple (dirs ..) stanzas dune#13734, fixes Feature request: allow multiple (dirs ... ) stanzas dune#6249, @anmonteiro)

• Sandboxed rules are now allowed to produce diff promotions when (corrections produce) is set on the rules. Whenever such an action produces foo.corrected, it will
  be automatically registered as a promotion for foo (feature: allow all sandboxed actions to produce corrected files dune#13813, @rgrinberg)

• $ dune clean now accepts arguments to only remove certain targets from the
  _build directory (feature/fix/refactor: move storing per target digests into rule cache dune#13875, @rgrinberg)

• $ dune promote now promotes all paths that are a prefix of the path provided.
  For example, $ dune promote foo will promote foo, foo/bar, foo/bar/baz
  (but not foobar). (feature: allow promoting prefixes with [$ dune promote] dune#13876, @rgrinberg)

• Allow for the diff action to diff entire directories (feature: allow diffing directories dune#13880, fixes diffing directories dune#3567, @rgrinberg)

• A hint is now emitted when trying to build a target inside a directory that
  was excluded by a (dirs ...) stanza (Show a hint when trying to build something excluded by a dirs stanza. dune#13919, @mefyl).

• Add --debug-backtraces to $ dune {promote,trace,cache} (feature: add --debug-backtraces to dune {promote,cache,trace} dune#13933, @rgrinberg)

• Actions are now able to observe their project directory via
  DUNE_PROJECT_ROOT (feature: introduce DUNE_PROJECT_ROOT dune#13934, @rgrinberg)

• Support hardlink sandboxing on Windows. (feat(sandbox): hardlinking on Windows dune#13987, addresses part of Use hard links when sandboxing on Windows when possible dune#4362, @Alizter)

• rocq.extraction: Add extracted_files field in (rocq 0.13), replacing
  extracted_modules, supporting extraction to languages other than OCaml
  (Adding extracted_files field to rocq.extraction stanza dune#13997, @Durbatuluk1701).

• Infer source directories as dependencies when they're as reference
  directories in diff actions.

• Add trace events for build start/stop/restarts (feature: watch mode trace events dune#14163, feature: add watch mode iterations to build events dune#14166, @rgrinberg)

• Generate compile_commands.json for C/C++ foreign stubs when building
  @check with (lang dune 3.23). This enables clangd, ccls, and other
  tools that consume compilation databases. (feat: compile_commands.json generation dune#14185, fixes Generate JSON Compilation Database dune#3531, @Alizter)

• Add trace events for accepting clients and shutting down RPC (feature: RPC lifecycle trace events dune#14232, @rgrinberg)

• Back up the default trace file as trace.csexp.old before each build,
  preserving the previous trace for comparison. This does not apply when
  --trace-file is used. (feat(trace): create trace backups dune#14269, @Alizter)

Changed

• Dune digests target files and directories in background threads (Asynchronous digests part #2 dune#13341,
  @rgrinbreg)

• Dyn.pp now prints valid OCaml literals for more constructors:

  • Char is quoted (e.g., 'a')
  • Int32/Int64/Nativeint include literal suffixes (l/L/n)
  • Float handles infinity, neg_infinity, and nan
    (dyn: format more values as valid literals dune#13394, grants Request to make Dyn.char printed as valid OCaml literals dune#13378, @Alizter)

• Bump the minimal OCaml version required to build Dune from 4.08 to 4.14.
  (chore: bump minimal OCaml version to 4.14 dune#13533, @Alizter)

• Starting from 3.23, user rules are sandboxed by default (feature: enable sandboxing for user rules starting from 3.23 dune#13805, @rgrinberg)

• Dune will no longer stat paths in the _build directory. This was done as protection
  from user rules accidentally touching things in the _build directory. This is forbidden,
  and dune offers rule sandboxing to prevent this (feature/fix/refactor: move storing per target digests into rule cache dune#13875, @rgrinberg)

• Improve mtime precision by no longer approximating them with floats (fix: improved mtime precision dune#13962,
  @rgrinberg).

• Remove --makefile (or -m) in $ dune rules. The sexp output is the only
  supported way of reflecting dune rules (chore: get rid of makefile mode for printing rules dune#14069, @rgrinberg)

• Do not automatically promote files. Now, opam files must be manually
  promoted with $ dune promote. Their generation is triggered by @install,
  @runtest, and @opam. In release mode, .opam files aren't generated at all
  and whatever is in the source is used (feature: use diff rather (mode promote) for generating opam files dune#14108, @rgrinberg)

• Inline test runner generation is now sandboxed (fix: sandbox generation of the inline test runner dune#14257, @rgrinberg)

[shonfeder]

Failures not present in the last release (3.22.2)

Lint error

• (lint) (failed):
  • (): (lint) (failed)

See ocaml/dune#14395, with the first part of a required fix into this repo at #29838

Internal CI error

• fs-io:
  • (revdeps/compilers/4.14): volgo.0.0.22 (failed: Failed: Build failed)

Unable to find image 'ocaml/opam:debian-13-ocaml-4.14@sha256:9fe84ffeeaa92507d198c7af8adb382726e06ae36c031f5ef645a3241cac0353' locally
Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:49593->127.0.0.53:53: i/o timeout
Failure("\"docker\" \"create\" \"--\" \"ocaml/opam:debian-13-ocaml-4.14@sha256:9fe84ffeeaa92507d198c7af8adb382726e06ae36c031f5ef645a3241cac0353\" failed with exit status 1")

Conclusion

All other errors were either present also in the previous release (and just part of the broken package debt we're carrying in the repo) or are cases of know irrelevant errors (opam 2 solve errors, sources getting 404 etc.).

I propose that we cut the actual release, which will just be the exact same package, with the depext linting errors fixed in the opam file, if other maintainers feel that the results here are also satisfactory, we could merge the full release once the main CI checks are going green, then cancel the revdeps, to avoid eating up all the compute.

[avsm]

closed in favour of #29844. Please do try to not have concurrent massive PRs open if possible, as it causes a lot of recalculation in the CI in the solver every time there's a merge on head.

[shonfeder]

Thanks! I'll be more vigilant about closing other open draft PRs in the future.