Apply rename_map to fresh-built processor pipelines for all policies#3428
Open
ihlej wants to merge 2 commits intohuggingface:mainfrom
Open
Apply rename_map to fresh-built processor pipelines for all policies#3428ihlej wants to merge 2 commits intohuggingface:mainfrom
ihlej wants to merge 2 commits intohuggingface:mainfrom
Conversation
Previously --rename_map only took effect on the pretrained-load path via
preprocessor_overrides. On the fresh-build path (pretrained_path is None),
per-policy make_*_pre_post_processors hardcode
RenameObservationsProcessorStep(rename_map={}), so --rename_map was silently
dropped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix: apply --rename_map on fresh-built processor pipelines
Summary / Motivation
Any PI05/PI0/PI0Fast training run that combines use_relative_actions=true with --rename_map crashes at step 0. The CLI rename map is silently dropped on the fresh-build path.
--rename_mapis only wired into the processor pipeline on the pretrained-load path, viapreprocessor_overrides["rename_observations_processor"]inside theif processor_pretrained_path is not None:branch atscripts/lerobot_train.py:288. On the fresh-build path (whenprocessor_pretrained_path is None), the per-policymake_*_pre_post_processorsfunctions (for examplemake_pi05_pre_post_processors()) all construct the rename step from a hardcodedRenameObservationsProcessorStep(rename_map={}), so the CLI rename_map is never used.Two ways to hit the fresh-build path:
--policy.pathnot set), which affects any policy.use_relative_actions=trueon a non-resume run, which forcesprocessor_pretrained_path = Noneatscripts/lerobot_train.py:275so processors are rebuilt from the current config (otherwise old checkpoints can silently drop the relative-action transform steps).In both cases,
--rename_mapis silently ignored. For PI05 thesymptom is
ValueError: All image features are missing from the batchat step 0; for other policies this typically surfaces as a
KeyErroron the renamed key when the policy tries to read its image features.
This PR applies
rename_mapon the fresh-build path; thepretrained-load path is unchanged.
Related issues
Fixes: #3425
What changed
scripts/lerobot_train.py: forwardcfg.rename_mapintoprocessor_kwargsunconditionally, not only whenprocessor_pretrained_path is not None.policies/factory.py: after the fresh-build branch constructs the pipeline, walkpreprocessor.stepsand setrename_mapon theRenameObservationsProcessorStep. This is the fresh-build counterpart to theoverridesargument thatPolicyProcessorPipeline.from_pretrainedalready applies on the pretrained-load path.rename_mapviapreprocessor_overrides["rename_observations_processor"].How was this tested (or how to run locally)
lerobot/pusht(~115 kB of meta + video), buildspolicy.input_featureswith a renamed image key so the dataset and policy keys intentionally disagree, and passes--rename_mapto bridge that mismatch. It then runs 5 training steps. Without the fix,--rename_mapis silently ignored and the run crashes at step 0 with aKeyErroron the renamed key; with the fix, the rename is applied and the 5 steps complete.Checklist (required before merge)
pre-commit run -a)pytest)Reviewer notes