fix: multi-sink routing, cloud hardware-source relay, and per-node recording#854
Merged
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
leszko
changed the title
…loud When the local instance relays hardware-source frames (Syphon/NDI/Spout) to the cloud via WebRTC, the cloud receives them as plain video tracks. Previously the cloud's handle_offer only routed tracks matching webrtc_source_node_ids (which excludes hardware sources), so Syphon tracks were silently dropped and their pipeline source queues starved — producing no output. Changes: - _parse_graph_node_ids now returns all_source_node_ids alongside the filtered webrtc_source_node_ids. The cloud's handle_offer uses the full list so every incoming track is routed to the correct SourceInputHandler. - handle_offer_with_relay routes browser tracks through CloudSourceInputHandler (per-track cloud input) instead of _input_loop → FrameProcessor.put() which would always send to generic track 0, colliding with hardware-source frames. - When all sources are hardware, the browser video track is ignored entirely. - Added diagnostic logging to CloudRelay and FrameProcessor for first-frame send/receive and video_mode detection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Rafał Leszko <rafal@livepeer.org>
Add compute_relay_video_mode() in cloud_relay so CloudRelay forwards frames when WebRTC params lack input_mode: video but the graph has source nodes or server-side capture (Syphon, NDI, Spout, video file). FrameProcessor uses it when building the relay. Signed-off-by: Rafał Leszko <rafal@livepeer.org> Made-with: Cursor
Re-skip aliasing when the node id is already registered so WebRTC does not overwrite per-node pipeline instances with a stale registry-key singleton. Fixes wrong pipeline output on multi-node graphs (e.g. passthrough showing split-screen). Signed-off-by: Rafal Leszko <rafal@livepeer.org> Made-with: Cursor
…cture Add per-sink frame capture in HeadlessSession, fix attribute paths in cloud output wiring, fix alias_pipeline collision, add recording file fallback after stop, and restructure CLAUDE.md to separate single-instance and local cloud testing docs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Rafal Leszko <rafal@livepeer.org>
- Treat record nodes as graph execution nodes (not ui-only); load them from graph.nodes in graphConfigToFlow and skip duplicate record entries from ui_state. - Pass sink + record node IDs to WebRTC so recvonly transceivers match backend extra outputs (sinks then record tracks). - Add ?node_id= to recording download/start/stop; route graph sessions to RecordingCoordinator; skip session-wide auto-recording when record nodes exist. - Clarify stripUIFields: topology including record stays in nodes/edges. Signed-off-by: Rafal Leszko <rafal@livepeer.org> Made-with: Cursor
leszko
force-pushed
the
rafal/9-multi-bug-fixes
branch
from
0738358 to
01cdf79
Compare
Contributor
🚀 fal.ai Preview Deployment
Livepeer Runner
Testing Livepeer Mode |
…ources Graph Source nodes call onSourceModeChange with a node id, so switchMode never ran for Syphon/NDI/Spout and the default file stream was still sent via updateVideoTrack. Sync global useVideoSource on those modes and skip the mid-stream track replace when the graph is server-side only. Signed-off-by: Rafał Leszko <rafal@livepeer.org> Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes multiple bugs in multi-sink graph execution and cloud relay that caused broken output, dropped frames, and recording failures when using graphs with multiple sinks, record nodes, or hardware sources (Syphon/NDI/Spout).
Also adds MCP server support for multi-source/multi-sink graph workflows (headless cloud mode).
Bugs Fixed
Per-sink frame routing returned identical frames for all sinks
HeadlessSessiondid not maintain per-sink frame buffers — every sink returned the same (primary) frame. Added_last_frames_by_sinkso each sink captures its own output independently.Pipeline aliasing collision caused wrong output on multi-node graphs
alias_pipelinewas overwriting per-node pipeline instances with a stale registry-key singleton. For example, a graph with bothpassthroughandsplit-screennodes would showsplit-screenoutput on both. Restored the guard that skips aliasing when the node ID is already registered.Record nodes were silently dropped from graph execution
Record nodes were treated as UI-only (
ui_state) and never loaded into the execution graph. FixedgraphConfigToFlowto include record nodes as execution nodes, and wired WebRTC transceivers so extra sinks and record tracks receive output.Recording download failed after stopping
The recording file lookup had no fallback after stop, causing download requests to 404. Added fallback path resolution in the recording coordinator.
Cloud relay dropped Syphon/NDI/Spout source frames when
input_modewas omittedCloudRelay.video_modestayedFalsewheninput_mode: "video"was not explicitly set, even when the graph contained hardware source nodes (Syphon, NDI, Spout, video file). Addedcompute_relay_video_mode()that infers video mode from graph source nodes.Cloud ignored hardware-source tracks (Syphon/NDI/Spout)
The cloud's
handle_offeronly routed incoming WebRTC tracks matchingwebrtc_source_node_ids, which excludes hardware sources relayed from the local instance. Syphon/NDI/Spout tracks were silently dropped and their pipeline source queues starved. Fixed_parse_graph_node_idsto return all source node IDs so every incoming track is routed correctly.VP8 keyframe request only sent for primary video track
The PLI (Picture Loss Indication) keyframe request was only sent for video track index 0. Extra sink and record tracks (index 1+) never received a keyframe, so their VP8 decoders started on P-frames — producing blocky/glitchy artifacts. Now sends PLI for every video receiver after connection.
Wrong attribute name in MCP cloud output wiring
_wire_cloud_outputsreferencedextra_output_handlerswhich does not exist onCloudWebRTCClient. Fixed to use the correctoutput_handlersattribute.Files Changed
cloud_relay.py,cloud_webrtc_client.py,frame_processor.pywebrtc.py,pipeline_manager.pyheadless.py,sink_manager.py,recording_coordinator.pymcp_router.py,mcp_server.py,app.pygraphUtils.ts,StreamPage.tsxCLAUDE.mdTest plan
🤖 Generated with Claude Code