Skip to content

Compile module init files in background to increase startup speed#2144

Open
TrevorBurnham wants to merge 1 commit into
sorin-ionescu:masterfrom
TrevorBurnham:feat/zcompile-modules
Open

Compile module init files in background to increase startup speed#2144
TrevorBurnham wants to merge 1 commit into
sorin-ionescu:masterfrom
TrevorBurnham:feat/zcompile-modules

Conversation

@TrevorBurnham
Copy link
Copy Markdown

Fixes #1418

This PR uses zompile to precompile the top-level init.zsh and every loaded module's init.zsh (or <module>.plugin.zsh).

Context

#1418 asked for zcompile support in Prezto. PR #2028 partially addressed it by compiling the compinit dump. The remaining parse cost sits in pmodload sourcing init.zsh files. Zsh auto-prefers a .zwc when newer than the source, so source calls transparently upgrade to wordcode loads once the files exist.

Benchmarks

zprof (averaged across multiple runs) with the modules environment, syntax-highlighting, history-substring-search, osx, autosuggestions, completion, directory, history, prompt (pure theme), spectrum, terminal:

Before After Δ
pmodload total 81.5ms 55.1ms −26.4ms (−32%)
pmodload self 34.0ms 22.3ms −11.7ms (−34%)
compinit 7.2ms 3.7ms −3.5ms (−49%)

Changes

  • Extend runcoms/zlogin to precompile init modules in the background, alongside the existing zcompdump compile.
  • Sweep stale .zwc.lock directories (older than 5 minutes) before mkdir, in both the existing zcompdump block and the new module-compile block, so a killed shell can no longer permanently disable recompilation.
  • Document the new zstyle in runcoms/zpreztorc: zstyle ':prezto:load' zcompile 'no' to opt out. Default is on (zstyle -T).

Design notes

Where: a second { ... } &! block appended after the existing zcompdump compile in runcoms/zlogin. Login-shell cadence is right — these files only change on zprezto-update / git pull, not between shells. Integrating into pmodload would pay stat + lock cost on every interactive shell including exec zsh; the zlogin placement doesn't.

What gets compiled:

  1. $ZPREZTODIR/init.zsh
  2. For each subdirectory of modules/, contrib/, and any user pmodule-dirs where zstyle -t ":prezto:module:$name" loaded 'yes' succeeds (set by pmodload on successful source, init.zsh:136): the module's init.zsh, falling back to <module>.plugin.zsh — same resolution order as pmodload.

external/ directories are skipped; third-party submodules are not Prezto's to compile.

Concurrency: per-target command mkdir "${target}.zwc.lock" / rmdir, matching the existing zcompdump pattern (commit e50b93c, PR #2028). Verified with 8 parallel login shells: all .zwc files produced, zero leftover locks.

Stale-lock recovery: both blocks now sweep .zwc.lock(N/mm+5) — any lock directory older than 5 minutes is rmdir'd before the mkdir attempt. Without this, a SIGKILL / OOM / power loss between mkdir and rmdir would leave a lock dir that permanently blocks future compilation (silently — no error, no retry). The sweep is applied to the existing zcompdump block too since the hazard exists there as well.

Correctness details:

  • [[ ! -s "${t}.zwc" || "$t" -nt "${t}.zwc" ]] triggers regeneration whenever the source is newer. git pull updates mtimes, so upgrades invalidate correctly.
  • zstyle -T treats unset as true, giving default-on behavior without forcing users to touch .zpreztorc.
  • $0 / ${0:h} still resolve to the original source path when source reads a sibling .zwcZPREZTODIR=${0:h} at init.zsh:168 continues to work. Verified.
  • Modules with early returns (e.g. completion/init.zsh exiting on TERM=dumb) compile correctly — the early return is expressed in the bytecode.
  • zcompile stderr is not suppressed; parse errors surface on the first post-update login so users can diagnose them, rather than silently never-producing a .zwc.

Verification

Tested on macOS 15.4 / zsh 5.9. All scenarios pass:

  1. Clean creationrm all .zwc, log in, new shell produces 14 .zwc files (1 top-level + 13 module wrappers including transitive deps). No leftover locks.
  2. Invalidation on edittouch modules/history/init.zsh, log in twice, .zwc mtime refreshed to match source.
  3. Opt-outzstyle ':prezto:load' zcompile 'no', log in, no .zwc created.
  4. Concurrency — 8 parallel login shells, all targets compile, zero leftover locks.
  5. Stale-lock recovery — pre-create a .zwc.lock dated 10 minutes ago, log in, lock is swept and .zwc produced. (A fresh lock is correctly respected.)
  6. Read-only filesystemchmod -w a module dir, log in, shell init completes cleanly; zcompile failure is visible in the terminal (user can diagnose) but does not block startup.
  7. .plugin.zsh fallback — synthetic contrib module with only a .plugin.zsh is correctly compiled.
  8. Broken module init — synthetic module with a parse error surfaces the zcompile error on stderr (no silent failure loop).

Extends the existing zlogin zcompdump pattern to also precompile the
top-level init.zsh and every loaded module's init.zsh (or .plugin.zsh,
matching pmodload's resolution order). Subsequent shells source
wordcode instead of reparsing. Current shell pays no cost
(backgrounded via &!) and does not benefit from its own compile.

Reuses the mkdir-based lock from the zcompdump block (commit e50b93c)
to serialize concurrent compiles across simultaneous login shells.
Both lock sites now sweep lock dirs older than 5 minutes before
attempting mkdir, so a killed shell (SIGKILL, OOM, power loss) can
no longer permanently disable recompilation.

Gated by zstyle ':prezto:load' zcompile, default on via zstyle -T.
Modules are enumerated under modules/, contrib/, and any pmodule-dirs;
compilation is restricted to entries with loaded=yes (set by pmodload
on successful source).

Measured on a typical config: pmodload total 81.5ms -> 55.1ms (-32%),
compinit 7.2ms -> 3.7ms (-49%).

Partially addresses sorin-ionescu#1418.
@TrevorBurnham TrevorBurnham changed the title Compile module init files in background to increase startup speed. Compile module init files in background to increase startup speed May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add zcompile to completion init

1 participant