Flow Graph Editor: Phase 2 glTF round-trip + review fixes#18299
Flow Graph Editor: Phase 2 glTF round-trip + review fixes#18299sebavan merged 17 commits intoBabylonJS:masterfrom
Conversation
- Import glTF with KHR_interactivity: dropping a .glb/.gltf file on the preview pane now detects FlowGraphCoordinator flow graphs loaded by the KHR_interactivity extension and imports them into the editor. - Import/export BABYLON_flow_graph custom extension: a round-trip format that embeds the FlowGraph JSON inside a .glb file. 'Export glTF (.glb)' button in the FILE section exports the graph (and optionally the scene via GLTF2Export if available). 'Load glTF' button imports from the custom extension. - GLB construction: SerializationTools gains _BuildMinimalGlb and _InjectExtensionIntoGlb helpers for creating and augmenting GLB files at the byte level. - Composite template system: 11 curated multi-block templates organized into 4 categories (Common Patterns, Animation Patterns, Communication, glTF Interactivity). Templates appear in the palette under 'Templates:' sections and create multiple blocks with pre-wired connections when dropped on the canvas. - Updated MANUAL.md with glTF Import/Export and Composite Templates docs. - Updated helpContent.ts with new help topic IDs and content. - Marked C3 and C7 as done in ISSUES-Phase-2.md.
…lay fixes Core fixes: - Preserve pathConverter through serialize/deserialize round-trip (GLTFPathToObjectConverter contains closures, not JSON-serializable) - Add TransformNode to IsMeshClassName for multi-primitive glTF nodes - Retry loop for KHR_interactivity async race condition - Block factory fallback lookup by short class name - Defensive constructor in FlowGraphGLTFDataProvider - Fix dagre layout edge ID mismatch Editor fixes: - Snapshot and restore context _connectionValues across setScene/stop (preserves unconnected input defaults like divide-by-2) - Sync context connection values into _defaultValue for editor display (SyncConnectionValuesToDefaults) - Fix BlockNodeData.name for binary operation blocks showing 'undefined' (base constructor calls getClassName before parameter property init) - Wire preview scene via setScene before start for correct event routing - Snapshot user variables before stop to survive restart cycles - Rebind context user variables when scene changes - Deferred auto-layout after graph load
- Binary operation blocks return correct getClassName after construction - Unconnected input values (context _connectionValues) survive round-trip - Math graph with unconnected inputs produces correct results after serialize/parse - TransformNode references serialize and resolve correctly
…tivity round-trip Performance — large graph support (100+ blocks): - Parser: O(1) Map-based connection lookup instead of O(B×P) linear scans - Serialization: skip non-serializable objects (pathConverter, closures) immediately to avoid expensive JSON.stringify attempts per block - Layout: grid fallback for 100+ block graphs (avoids O(V×E) dagre freeze) - Grid layout: fix _isLoading not cleared + _refreshLinks not called, which prevented visual connections from rendering - Debug mode: batch highlights via rAF instead of per-execution-callback DOM work; per-frame cap of 30 nodes; viewport culling for offscreen nodes; generation-toggle CSS trick eliminates forced reflows in port pulse animation; replace full graphNode.refresh() with lightweight execution-time label update; O(1) block→GraphNode Map cache - GraphNode: expose executionTimeElement getter for targeted updates Mesh identity — multiple meshes with same id/name: - Serialization: preserve uniqueId in mesh descriptors so same-id meshes are disambiguated during deserialization (fixes SnailRace, Flocking) - Add InstancedMesh to IsMeshClassName check - _rebindContextUserVariables: use uniqueId when multiple candidates share the same id/name after scene reload - _rebindMeshReference: same uniqueId-based disambiguation glTF export: - Access GLTF2Export via globalThis.BABYLON global (webpack only externalizes core/, not serializers/) - Add serializers CDN bundle to script loading Misc: - SyncConnectionValuesToDefaults: copy context _connectionValues into _defaultValue for editor display of unconnected inputs
- Multiple meshes with same id distinguished by uniqueId through serialize → parse round-trip (covers SnailRace scenario with 3 MeshPickEvent blocks on different same-named meshes)
…emplates lookup instead
…at start FlowGraphEventBlock._execute() only fired 'done', but connections made in the editor or via _startPendingTasks used 'out'. KHR_interactivity maps glTF flows to 'done', so downstream blocks never executed after round-trip. - _execute now activates both done and out signals - Override _startPendingTasks to skip the premature out fire at graph start (event blocks should only fire signals when the actual event triggers) - Add regression test verifying both out and done connections work after serialize → parse round-trip
GLTFDataProvider blocks store live Babylon object arrays (nodes, animationGroups) computed from the IGLTF parse tree in their constructor. This data is not serializable and was lost during the editor's serialize→parse round-trip, leaving the blocks with empty arrays. Downstream blocks (IndexOf for selectedNodeIndex, ArrayIndex for animation lookup) received empty arrays and silently produced wrong results, causing no visible action when clicking meshes in Sundial etc. Fix: - Extract liveGLTF from original GLTFDataProvider blocks before serialize - After deserialization, re-inject it: set config.glTF and re-compute _defaultValue for nodes and animationGroups outputs - Also call _rebindContextUserVariables after restoring saved user vars in patched createContext so mesh descriptors resolve to real objects
FlowGraphGLTFDataProvider called super() without passing config, so this.config was undefined. The editor's glTF extraction code checked block.config?.glTF to grab the live glTF object before serialization — but it was always undefined, so re-injection of nodes/animationGroups never happened after round-trip. Fix: super() → super(config) so the glTF reference is accessible on the block instance for the editor to extract and re-inject.
Use top-level type imports instead of inline import('...').Type
syntax in serializationTools.ts and scenePreviewComponent.tsx.
- flowGraphSerialization.test.ts: 7 new tests covering uniqueId in mesh refs, TransformNode/InstancedMesh class recognition, pathConverter skip, function-property skip, plain JSON objects, and plain array round-trip - flowGraphEventNodes.test.ts: 3 new tests covering event blocks firing both done and out signals, no premature out at graph start, and round-trip preservation of dual-signal behavior - flowGraphGLTFDataProvider.test.ts: 5 new tests covering config storage via super(config), nodes/animationGroups output population, graceful handling of undefined/empty glTF, and className correctness
- Remove fragile duck-typing check in serialization.ts (Comment 1) - Replace O(N) connection lookup fallbacks with explicit throws (Comments 2-3) - Restore fail-fast for missing pathConverter (Comment 4) - Revert getClassName to 'FlowGraphGLTFDataProvider' for backcompat (Comment 5) - Add O(1) ShortNameToFullKey reverse lookup in block factory (Comment 6) - Add Logger warnings on silent GLB export fallback, fix lint (Comment 7) - Extract GLB magic numbers into named constants (Comment 8) - Fix await-in-loop with Promise.all pre-resolution (Comment 9) - Cache textContent comparison to skip unnecessary DOM writes (Comment 10) - Extract repeated object-ref type check in _rebindMeshReference (Comment 11) - Add comment explaining dual done/out signal firing (Comment 12) - Document setScene() context-clearing contract (Comment 13)
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
There was a problem hiding this comment.
Pull request overview
Implements Phase 2 Flow Graph Editor features by enabling glTF/GLB round-trip of flow graphs (via a custom BABYLON_flow_graph extension), improving editor runtime behavior (debug highlighting, layout, composite templates), and updating core FlowGraph serialization/parsing behavior to support interactivity scenarios and fix prior review items.
Changes:
- Add glTF import/export for flow graphs (custom extension embed + extraction) and integrate serializers loading for full-scene GLB export.
- Improve editor UX/perf: composite templates, large-graph grid layout fallback, O(1) block→node lookup caching, and batched debug highlighting.
- Update FlowGraph core serialization/parsing: uniqueId-preserving mesh refs, pathConverter handling, faster connection resolution, and event-block signal behavior; add/extend unit tests.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/tools/flowGraphEditor/tsconfig.build.json | Fix TS path mappings to use correct relative paths for build. |
| packages/tools/flowGraphEditor/src/serializationTools.ts | Add GLB/GLTF import/export utilities and post-parse default syncing for editor display. |
| packages/tools/flowGraphEditor/src/graphSystem/blockNodeData.ts | Make node naming resilient when data.name is falsy during construction. |
| packages/tools/flowGraphEditor/src/graphEditor.tsx | Add composite template creation, block→node cache, visibility checks, and large-graph layout fallback. |
| packages/tools/flowGraphEditor/src/globalState.ts | Batch debug highlighting with rAF, avoid forced reflow, and preserve/restores context variables across scene changes. |
| packages/tools/flowGraphEditor/src/compositeTemplates.ts | Introduce curated multi-block “template” definitions for palette drop. |
| packages/tools/flowGraphEditor/src/components/propertyTab/propertyTabComponent.tsx | Add “Load glTF” and “Export glTF (.glb)” actions wired to serialization tools. |
| packages/tools/flowGraphEditor/src/components/preview/scenePreviewComponent.tsx | Auto-import flow graphs from loaded scenes/files; switch to createDefaultCamera/Light. |
| packages/tools/flowGraphEditor/src/components/nodeList/nodeListComponent.tsx | Add Templates sections to the palette, filterable by search text. |
| packages/tools/flowGraphEditor/src/components/help/helpContent.ts | Add help topics for glTF import/export and composite templates. |
| packages/tools/flowGraphEditor/src/components/graphControls/graphControlsComponent.tsx | Call setScene() before start and snapshot variables before stop. |
| packages/tools/flowGraphEditor/public/index.js | Load serializers bundle so GLTF2Export is available for full-scene GLB export. |
| packages/tools/flowGraphEditor/MANUAL.md | Document new glTF import/export and composite templates features. |
| packages/tools/flowGraphEditor/ISSUES-Phase-2.md | Mark Phase 2 issues complete (glTF round-trip + templates). |
| packages/dev/sharedUiComponents/src/nodeGraphSystem/graphNode.ts | Expose execution-time element for lightweight label updates. |
| packages/dev/sharedUiComponents/src/nodeGraphSystem/graphCanvas.tsx | Improve dagre edge targeting to prefer actual node IDs when available. |
| packages/dev/loaders/test/unit/Interactivity/flowGraphGLTFDataProvider.test.ts | Add unit tests for GLTF data provider output defaults and config retention. |
| packages/dev/loaders/src/glTF/2.0/Extensions/KHR_interactivity/flowGraphGLTFDataProvider.ts | Fix config propagation (super(config)) and handle undefined glTF gracefully. |
| packages/dev/core/test/unit/FlowGraph/flowGraphSerialization.test.ts | Add tests for event signals, binary op class names, defaults round-trip, mesh/node refs, and serialization edge cases. |
| packages/dev/core/test/unit/FlowGraph/flowGraphEventNodes.test.ts | Add tests for event block done/out behavior and round-trip behavior. |
| packages/dev/core/src/FlowGraph/serialization.ts | Add uniqueId to serialized mesh refs; skip non-serializable objects; refine array parsing behavior. |
| packages/dev/core/src/FlowGraph/flowGraphParser.ts | Switch to O(1) connection lookup maps and improve error messaging for missing pathConverter. |
| packages/dev/core/src/FlowGraph/flowGraphEventBlock.ts | Ensure event blocks fire both done and out, and suppress premature out on graph start. |
| packages/dev/core/src/FlowGraph/flowGraph.ts | Clear execution contexts on setScene() with documented contract. |
| packages/dev/core/src/FlowGraph/Blocks/flowGraphBlockFactory.ts | Add reverse lookup map for O(1) custom block factory fallback by short name. |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18299/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18299/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18299/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
You have changed file(s) that made possible changes to the sandbox. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18299/merge/ |
|
You have made possible changes to the playground. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18299/merge/ The snapshot playground with the CDN snapshot (only when available): Note that neither Babylon scenes nor textures are uploaded to the snapshot directory, so some playgrounds won't work correctly. |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
- Add side-effect import for scene.createDefaultCamera/Light (scenePreviewComponent) - Use instanceof FlowGraphEventBlock instead of duck-typing _executeEvent (graphEditor) - Remove dead isSignal ternary branches in template wiring (graphEditor) - Tighten event config array predicate to check for 'eventData' key (serialization) - Add GLB bounds checks: validate chunk header size, chunk type, and data length (serializationTools)
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18299/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18299/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18299/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
You have changed file(s) that made possible changes to the sandbox. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18299/merge/ |
|
You have made possible changes to the playground. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18299/merge/ The snapshot playground with the CDN snapshot (only when available): Note that neither Babylon scenes nor textures are uploaded to the snapshot directory, so some playgrounds won't work correctly. |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
You have changed file(s) that made possible changes to the sandbox. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18299/merge/ |
|
You have made possible changes to the playground. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18299/merge/ The snapshot playground with the CDN snapshot (only when available): Note that neither Babylon scenes nor textures are uploaded to the snapshot directory, so some playgrounds won't work correctly. |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
Summary
Implements Phase 2 of the Flow Graph Editor with glTF round-trip support and addresses all 13 Copilot review comments.
Key changes
doneandoutsignals, skip prematureoutat startCopilot review feedback addressed
2-3. Replaced O(N) connection fallbacks with explicit throws