Skip to content

Commit ec10f5d

Browse files
leszkoclaude
andauthored
feat(plugins): plugin-registered input sources + source-kind cloud routing (#987)
Lets plugins register their own video input sources (e.g. YouTube, RTSP, screen capture) by subclassing `InputSource` and exposing them via the existing `register_nodes` hook. Replaces the hardcoded `spout/ndi/syphon[/video_file]` tuples that gated input-source behavior with a registry-driven path, so plugin sources flow through the same code as built-ins. Source-kind plugins (declared via `__scope_kind__ = "source"` on the plugin's top-level package) are pinned to the local instance even in cloud mode, since their frames originate locally. ## Backend — registry & dispatch - `PluginManager.register_plugin_nodes` dispatches by class: `InputSource` subclasses go into a new `_plugin_input_sources` dict keyed by `source_id`; everything else still lands in `NodeRegistry`. Exposed via `get_plugin_input_sources()` on both `PluginManager` and the module. - `get_input_source_classes()` merges the plugin dict into the built-in map. Built-ins win on `source_id` collision, so a rogue plugin can't hijack `spout`/`ndi`/`syphon`/`video_file`. - `SourceManager.setup_multi_sources` drops the source-mode whitelist — any node whose `source_mode` resolves in the registry spawns a receive thread. - `webrtc._parse_graph_node_ids` uses the registry instead of a hardcoded tuple when splitting WebRTC-driven sources from server-side ones. **Behavior change:** `video_file` is now classified as a server-side source here (it already was in `SourceManager`), so graphs with a `video_file` node no longer allocate a WebRTC track for it. ## Backend — `__scope_kind__` and cloud routing - `probe_plugin_kind(package_spec)` detects source-kind plugins at install time by AST-parsing the package's `__init__.py` for a top-level `__scope_kind__ = "<literal>"`. Supports local paths (walk + AST-parse) and `git+` URLs (shallow-clone to tempdir, then walk). PyPI specifiers return `None` (no probe) — install via git URL or local path is the workaround for source-kind plugins published to PyPI. - `_read_scope_kind` reads the same attribute on already-installed plugins via `importlib.import_module(top_level)`. Result is cached per top-level module name to keep plugin-list refreshes cheap; cache is cleared on install/uninstall/reload. - `app.list_plugins`/`install_plugin`/`uninstall_plugin`/`reload_plugin` route by kind: - **list**: in cloud mode, merges local source-kind plugins (tagged `origin=local`) with cloud plugins (tagged `origin=cloud`). Local-listing failures degrade gracefully to cloud-only. - **install**: probes `__scope_kind__`; source-kind plugins install locally despite cloud mode. - **uninstall/reload**: targets local if the plugin name is locally installed; otherwise proxies to cloud. ## Backend — install/uninstall lifecycle - `install_plugin_async` re-fires `register_plugin_nodes` after a successful install so the new plugin's nodes/input sources show up immediately — no server restart needed. - `_uninstall_plugin_sync` walks the distribution's scope entry points and unregisters them from pluggy by entry-point name (the plugin object is often a class instance with no useful `__name__`). Then rebuilds plugin-derived caches by re-firing the hooks. Without this, a subsequent reload of any other plugin would resurrect the uninstalled plugin's registrations. - `_split_git_spec` handles `git+ssh://user@host/repo.git@branch`, strips pip-style `#fragment`/`?query` (e.g. `#egg=…`) before splitting on `@`. ## Frontend - The Source node dropdown is built dynamically from `/api/v1/input-sources`: browser-driven File/Camera are prepended, every available backend source (plugin-registered included) is appended, and `video_file` is hidden as a headless-only alias for File. The currently selected mode stays visible as `"<name> (unavailable)"` if the host doesn't support it, so cross-machine graphs don't silently lose UI. - Unknown modes render a generic text input with the backend's `source_description` as help text, so plugin sources get a reasonable UI with no frontend changes. - `sourceMode` is widened from a narrow union (`video | camera | spout | ndi | syphon`) to `string`. The 8 ad-hoc `=== "video" || === "camera"` checks across `StreamPage`, `useVideoSource`, and `SourceNode` are consolidated behind `isBrowserSourceMode` in `graphUtils.ts`. - `availableInputSources` is threaded through the graph-editor prop chain (`StreamPage` → `GraphEditor` → `useGraphState` → `enrichNodes` → `SourceNode`) and refetched whenever pipelines change, so newly-installed plugin sources surface without a page reload. - `PluginsTab` shows a `Local`/`Cloud` origin badge next to each plugin, and `restartAndRefresh` makes the plugin/pipeline list refresh even if the restart-wait times out (the install/uninstall/reload itself already succeeded server-side). ## Companion plugin Paired with an out-of-tree plugin at `~/scope-youtube` (`scope-youtube`) that registers a `YouTubeInputSource` via `@hookimpl register_nodes` and declares `__scope_kind__ = "source"` so it stays local in cloud mode. --------- Signed-off-by: Rafal Leszko <rafal@livepeer.org> Signed-off-by: Rafał Leszko <rafal@livepeer.org> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 002b11c commit ec10f5d

19 files changed

Lines changed: 741 additions & 153 deletions

File tree

frontend/src/components/PluginsDialog.tsx

Lines changed: 44 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,8 @@ export function PluginsDialog({
7474
update_available: p.update_available,
7575
package_spec: p.package_spec,
7676
bundled: p.bundled,
77+
kind: p.kind,
78+
origin: p.origin,
7779
})),
7880
[pluginInfos]
7981
);
@@ -105,6 +107,28 @@ export function PluginsDialog({
105107
}
106108
};
107109

110+
// Restart the server and refresh plugins+pipelines. Refresh runs even
111+
// if the restart wait times out (e.g. the cloud's restart took longer
112+
// than the local poll budget) — the install/uninstall/reload itself
113+
// already succeeded server-side, so the UI should pick up the new
114+
// state regardless. Returns a label for the toast.
115+
const restartAndRefresh = async (
116+
successLabel: string,
117+
fallbackLabel: string
118+
): Promise<string> => {
119+
let label = successLabel;
120+
try {
121+
const oldStartTime = await restartServer();
122+
await waitForServer(oldStartTime);
123+
} catch (e) {
124+
console.warn("Server restart wait did not complete:", e);
125+
label = fallbackLabel;
126+
}
127+
await refreshPlugins();
128+
await refetchPipelines();
129+
return label;
130+
};
131+
108132
const handleInstallPlugin = async (packageSpec: string) => {
109133
setIsInstalling(true);
110134
isModifyingPluginsRef.current = true;
@@ -121,12 +145,11 @@ export function PluginsDialog({
121145
});
122146
setPluginInstallPath("");
123147

124-
const oldStartTime = await restartServer();
125-
await waitForServer(oldStartTime);
126-
toast.success("Server restarted", { id: toastId });
127-
128-
await refreshPlugins();
129-
await refetchPipelines();
148+
const label = await restartAndRefresh(
149+
"Server restarted",
150+
`Installed ${pluginName}`
151+
);
152+
toast.success(label, { id: toastId });
130153
} else {
131154
toast.error(response.message, { id: toastId });
132155
}
@@ -158,13 +181,11 @@ export function PluginsDialog({
158181
toast.loading(`Updated ${pluginName}. Restarting server...`, {
159182
id: toastId,
160183
});
161-
162-
const oldStartTime = await restartServer();
163-
await waitForServer(oldStartTime);
164-
toast.success("Server restarted", { id: toastId });
165-
166-
await refreshPlugins();
167-
await refetchPipelines();
184+
const label = await restartAndRefresh(
185+
"Server restarted",
186+
`Updated ${pluginName}`
187+
);
188+
toast.success(label, { id: toastId });
168189
} else {
169190
toast.error(response.message, { id: toastId });
170191
}
@@ -189,13 +210,11 @@ export function PluginsDialog({
189210
toast.loading(`Uninstalled ${pluginName}. Restarting server...`, {
190211
id: toastId,
191212
});
192-
193-
const oldStartTime = await restartServer();
194-
await waitForServer(oldStartTime);
195-
toast.success("Server restarted", { id: toastId });
196-
197-
await refreshPlugins();
198-
await refetchPipelines();
213+
const label = await restartAndRefresh(
214+
"Server restarted",
215+
`Uninstalled ${pluginName}`
216+
);
217+
toast.success(label, { id: toastId });
199218
} else {
200219
toast.error(response.message, { id: toastId });
201220
}
@@ -214,11 +233,11 @@ export function PluginsDialog({
214233
isModifyingPluginsRef.current = true;
215234
try {
216235
toast.info(`Reloading ${pluginName}. Restarting server...`);
217-
const oldStartTime = await restartServer();
218-
await waitForServer(oldStartTime);
219-
toast.success("Server restarted");
220-
await refreshPlugins();
221-
await refetchPipelines();
236+
const label = await restartAndRefresh(
237+
"Server restarted",
238+
`Reloaded ${pluginName}`
239+
);
240+
toast.success(label);
222241
} catch (error) {
223242
toast.error(
224243
error instanceof Error ? error.message : "Failed to reload node"

frontend/src/components/graph/GraphEditor.tsx

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,7 @@ interface GraphEditorProps {
191191
spoutAvailable?: boolean;
192192
ndiAvailable?: boolean;
193193
syphonAvailable?: boolean;
194+
availableInputSources?: import("../../lib/api").InputSourceType[];
194195
onSpoutSourceChange?: (name: string) => void;
195196
onNdiSourceChange?: (identifier: string) => void;
196197
onSyphonSourceChange?: (identifier: string) => void;
@@ -241,6 +242,7 @@ export const GraphEditor = forwardRef<GraphEditorHandle, GraphEditorProps>(
241242
spoutAvailable = false,
242243
ndiAvailable = false,
243244
syphonAvailable = false,
245+
availableInputSources = [],
244246
onSpoutSourceChange,
245247
onNdiSourceChange,
246248
onSyphonSourceChange,
@@ -343,6 +345,7 @@ export const GraphEditor = forwardRef<GraphEditorHandle, GraphEditorProps>(
343345
spoutOutputAvailable,
344346
ndiOutputAvailable,
345347
syphonOutputAvailable,
348+
availableInputSources,
346349
},
347350
{
348351
tempoState,

frontend/src/components/graph/hooks/graph/useGraphState.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,7 @@ export interface GraphEditorAvailability {
122122
spoutOutputAvailable: boolean;
123123
ndiOutputAvailable: boolean;
124124
syphonOutputAvailable: boolean;
125+
availableInputSources: import("../../../../lib/api").InputSourceType[];
125126
}
126127

127128
export function useGraphState(
@@ -290,6 +291,7 @@ export function useGraphState(
290291
spoutOutputAvailable: availability.spoutOutputAvailable,
291292
ndiOutputAvailable: availability.ndiOutputAvailable,
292293
syphonOutputAvailable: availability.syphonOutputAvailable,
294+
availableInputSources: availability.availableInputSources,
293295
handleEdgeDelete,
294296
isStreaming: streams.isStreaming,
295297
isLoading: streams.isLoading ?? false,
@@ -339,6 +341,7 @@ export function useGraphState(
339341
availability.spoutOutputAvailable,
340342
availability.ndiOutputAvailable,
341343
availability.syphonOutputAvailable,
344+
availability.availableInputSources,
342345
tempo.tempoState,
343346
tempo.tempoSources,
344347
]);

frontend/src/components/graph/nodes/SourceNode.tsx

Lines changed: 71 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { useEffect, useRef, useCallback, useState } from "react";
22
import { Handle, Position } from "@xyflow/react";
33
import type { NodeProps, Node } from "@xyflow/react";
4+
import { isBrowserSourceMode } from "../../../lib/graphUtils";
45
import type { FlowNodeData } from "../../../lib/graphUtils";
56
import { getInputSourceSources, type DiscoveredSource } from "../../../lib/api";
67
import { useNodeData } from "../hooks/node/useNodeData";
@@ -21,14 +22,20 @@ const HEADER_H = 28;
2122
const BODY_PAD = 6;
2223
const SELECT_ROW_H = 20;
2324

24-
const SOURCE_MODE_OPTIONS = [
25+
// Browser-driven source modes (WebRTC) prepended to the backend's list.
26+
// Every other entry in the dropdown comes from /api/v1/input-sources.
27+
const BROWSER_SOURCE_OPTIONS = [
2528
{ value: "video", label: "File" },
2629
{ value: "camera", label: "Camera" },
27-
{ value: "spout", label: "Spout" },
28-
{ value: "ndi", label: "NDI" },
29-
{ value: "syphon", label: "Syphon" },
3030
];
3131

32+
// Built-ins that have bespoke UI branches below; everything else falls
33+
// through to the generic text-input branch.
34+
const SOURCES_WITH_CUSTOM_UI = new Set(["spout", "ndi", "syphon"]);
35+
36+
// "video_file" is a headless-only alias for browser "video"; hide it.
37+
const HIDDEN_BACKEND_SOURCES = new Set(["video_file"]);
38+
3239
export function SourceNode({ id, data, selected }: NodeProps<SourceNodeType>) {
3340
const { updateData } = useNodeData(id);
3441
const { collapsed, toggleCollapse } = useNodeCollapse();
@@ -45,6 +52,10 @@ export function SourceNode({ id, data, selected }: NodeProps<SourceNodeType>) {
4552
const spoutAvailable = data.spoutAvailable ?? false;
4653
const ndiAvailable = data.ndiAvailable ?? false;
4754
const syphonAvailable = data.syphonAvailable ?? false;
55+
const availableInputSources = data.availableInputSources ?? [];
56+
const currentSourceInfo = availableInputSources.find(
57+
s => s.source_id === sourceMode
58+
);
4859
const onSpoutSourceChange = data.onSpoutSourceChange as
4960
| ((name: string) => void)
5061
| undefined;
@@ -129,11 +140,12 @@ export function SourceNode({ id, data, selected }: NodeProps<SourceNodeType>) {
129140
}, [syphonAvailable]);
130141

131142
const handleSourceModeChange = (newMode: string) => {
143+
// Preserve sourceName for every server-side mode (anything that
144+
// isn't the browser-driven "video"/"camera"). Clearing it on every
145+
// switch would lose the URL/path/sender the user just typed.
132146
updateData({
133-
sourceMode: newMode as "video" | "camera" | "spout" | "ndi" | "syphon",
134-
...(newMode !== "spout" && newMode !== "ndi" && newMode !== "syphon"
135-
? { sourceName: undefined }
136-
: {}),
147+
sourceMode: newMode,
148+
...(isBrowserSourceMode(newMode) ? { sourceName: undefined } : {}),
137149
});
138150
onSourceModeChange?.(newMode);
139151
};
@@ -174,17 +186,36 @@ export function SourceNode({ id, data, selected }: NodeProps<SourceNodeType>) {
174186
[onVideoFileUpload]
175187
);
176188

177-
const showPreview = sourceMode === "video" || sourceMode === "camera";
189+
const showPreview = isBrowserSourceMode(sourceMode);
178190
const showFilePicker = sourceMode === "video";
179191
const handleY = HEADER_H + BODY_PAD + SELECT_ROW_H / 2;
180192

181-
// Filter source mode options based on availability
182-
const filteredSourceModeOptions = SOURCE_MODE_OPTIONS.filter(opt => {
183-
if (opt.value === "spout") return spoutAvailable;
184-
if (opt.value === "ndi") return ndiAvailable;
185-
if (opt.value === "syphon") return syphonAvailable;
186-
return true;
187-
});
193+
// Browser-driven options + every available backend source. Plugin-
194+
// registered sources appear automatically. If the graph was saved with
195+
// a mode that isn't currently available on this machine (e.g. NDI graph
196+
// opened on a host without NDI), keep it in the list as "(unavailable)"
197+
// so the user can see what's set instead of the dropdown silently
198+
// collapsing to a different selection.
199+
const filteredSourceModeOptions = [
200+
...BROWSER_SOURCE_OPTIONS,
201+
...availableInputSources
202+
.filter(s => s.available && !HIDDEN_BACKEND_SOURCES.has(s.source_id))
203+
.map(s => ({ value: s.source_id, label: s.source_name })),
204+
];
205+
if (
206+
sourceMode &&
207+
!filteredSourceModeOptions.some(o => o.value === sourceMode)
208+
) {
209+
const label = currentSourceInfo?.source_name ?? sourceMode;
210+
filteredSourceModeOptions.push({
211+
value: sourceMode,
212+
label: `${label} (unavailable)`,
213+
});
214+
}
215+
216+
const handleGenericSourceNameChange = (value: string | number) => {
217+
updateData({ sourceName: String(value) });
218+
};
188219

189220
const ndiOptions = ndiSources.map(s => ({
190221
value: s.identifier,
@@ -399,13 +430,30 @@ export function SourceNode({ id, data, selected }: NodeProps<SourceNodeType>) {
399430
</div>
400431
)}
401432

402-
{!showPreview &&
403-
sourceMode !== "spout" &&
404-
sourceMode !== "ndi" &&
405-
sourceMode !== "syphon" && (
406-
<div className="flex items-center justify-center rounded-md bg-black/30 text-[10px] text-[#8c8c8d] flex-1 min-h-[40px]">
407-
Waiting for input...
408-
</div>
433+
{/* Generic text-input for plugin-registered sources (or any
434+
backend source we don't have bespoke UI for). source_name
435+
is a free-form identifier (URL, path, ...); source_description
436+
from the backend is rendered below as help text. */}
437+
{!SOURCES_WITH_CUSTOM_UI.has(sourceMode) &&
438+
!isBrowserSourceMode(sourceMode) && (
439+
<>
440+
<div className="px-2">
441+
<NodeParamRow
442+
label={currentSourceInfo?.source_name ?? "Source"}
443+
>
444+
<NodePillInput
445+
type="text"
446+
value={sourceName}
447+
onChange={handleGenericSourceNameChange}
448+
/>
449+
</NodeParamRow>
450+
</div>
451+
{currentSourceInfo?.source_description && (
452+
<div className="px-2 text-[10px] text-[#8c8c8d]">
453+
{currentSourceInfo.source_description}
454+
</div>
455+
)}
456+
</>
409457
)}
410458
</div>
411459
)}

frontend/src/components/graph/utils/nodeEnrichment.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ export interface EnrichNodesDeps {
5050
spoutOutputAvailable: boolean;
5151
ndiOutputAvailable: boolean;
5252
syphonOutputAvailable: boolean;
53+
availableInputSources: import("../../../lib/api").InputSourceType[];
5354
handleEdgeDelete: (edgeId: string) => void;
5455
isStreaming: boolean;
5556
isLoading: boolean;
@@ -171,6 +172,7 @@ export function enrichNodes(
171172
spoutAvailable: deps.spoutAvailable,
172173
ndiAvailable: deps.ndiAvailable,
173174
syphonAvailable: deps.syphonAvailable,
175+
availableInputSources: deps.availableInputSources,
174176
onSpoutSourceChange: (name: string) =>
175177
deps.onSpoutSourceChangeRef.current?.(name),
176178
onNdiSourceChange: (identifier: string) =>

frontend/src/components/settings/PluginsTab.tsx

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,37 @@ const isElectron =
3232
typeof window !== "undefined" &&
3333
navigator.userAgent.toLowerCase().includes("electron");
3434

35+
const ORIGIN_BADGE_CLASSES =
36+
"text-[10px] uppercase tracking-wide px-1.5 py-0.5 rounded";
37+
38+
function OriginBadge({ plugin }: { plugin: InstalledPlugin }) {
39+
if (plugin.origin === "local") {
40+
return (
41+
<span
42+
className={`${ORIGIN_BADGE_CLASSES} bg-primary/15 text-primary`}
43+
title={
44+
plugin.kind === "source"
45+
? "Source-kind plugin: runs on this machine even in cloud mode"
46+
: "Installed on this machine"
47+
}
48+
>
49+
Local
50+
</span>
51+
);
52+
}
53+
if (plugin.origin === "cloud") {
54+
return (
55+
<span
56+
className={`${ORIGIN_BADGE_CLASSES} bg-muted text-muted-foreground`}
57+
title="Installed in the cloud-hosted backend"
58+
>
59+
Cloud
60+
</span>
61+
);
62+
}
63+
return null;
64+
}
65+
3566
// Transform plain Git host URLs to git+ format on paste
3667
const transformGitUrl = (value: string): string => {
3768
const trimmed = value.trim();
@@ -158,6 +189,7 @@ export function PluginsTab({
158189
v{plugin.version}
159190
</span>
160191
)}
192+
<OriginBadge plugin={plugin} />
161193
</div>
162194
{plugin.author && (
163195
<p className="text-xs text-muted-foreground">

0 commit comments

Comments
 (0)