Skip to content

Commit ffe7049

Browse files
hthillmanclaude
hthillman
and
claude
committed
docs: add OSC integration guide
A user-facing reference for the OSC integration: what's reachable, how to opt nodes in via Configure OSC, address format conventions, discovery surfaces (in-app panel + HTML reference + JSON endpoint), defaults handling, validation, routing internals, TouchDesigner setup, Python snippets, REST API, and current limitations. Companion to the feature work in #996. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> Signed-off-by: Hunter Hillman <hthillman@gmail.com>
1 parent caa747a commit ffe7049

1 file changed

Lines changed: 279 additions & 0 deletions

File tree

‎docs/osc.md‎

Lines changed: 279 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,279 @@
1+
# Using OSC
2+
3+
Scope exposes its parameters over [OSC (Open Sound Control)](https://opensoundcontrol.stanford.edu/) so external tools — TouchDesigner, Resolume, MaxMSP, hardware controllers, custom Python scripts — can drive the running graph in real time. The OSC server is always on; it shares a UDP socket with the HTTP API on the same port (default `8000`).
4+
5+
## Table of Contents
6+
7+
- [Quick Start](#quick-start)
8+
- [What's Reachable via OSC](#whats-reachable-via-osc)
9+
- [Configure OSC per Node](#configure-osc-per-node)
10+
- [Address Format](#address-format)
11+
- [Discovering Paths](#discovering-paths)
12+
- [Defaults in the Description](#defaults-in-the-description)
13+
- [Validation](#validation)
14+
- [Routing Internals](#routing-internals)
15+
- [TouchDesigner Setup](#touchdesigner-setup)
16+
- [Python Examples](#python-examples)
17+
- [REST API](#rest-api)
18+
- [Limitations](#limitations)
19+
20+
---
21+
22+
## Quick Start
23+
24+
1. Start Scope (`uv run daydream-scope`). The OSC server starts automatically and listens on UDP `8000`.
25+
2. Send a message:
26+
27+
```python
28+
from pythonosc.udp_client import SimpleUDPClient
29+
client = SimpleUDPClient("127.0.0.1", 8000)
30+
client.send_message("/scope/prompt", "a beautiful sunset over the ocean")
31+
client.send_message("/scope/noise_scale", 0.5)
32+
```
33+
34+
3. Open the live OSC reference at `http://localhost:8000/api/v1/osc/docs` to see every address, type, range, default, and a copy-paste example.
35+
36+
> [!NOTE]
37+
> Set the OSC port via the `SCOPE_PORT` environment variable. UDP and HTTP coexist on the same port, so changing one changes both.
38+
39+
---
40+
41+
## What's Reachable via OSC
42+
43+
Scope exposes three layers of OSC paths:
44+
45+
| Layer | Source | Default address | Notes |
46+
|---|---|---|---|
47+
| **Runtime globals** | Built-in (`prompt`, `noise_scale`, `paused`, `manage_cache`, `reset_cache`, `transition_steps`, `interpolation_method`, …) | `/scope/<param>` | Always on. Backwards-compatible with pre-existing TouchDesigner setups. |
48+
| **Pipeline runtime params** | Each loaded pipeline's `is_load_param=False` fields | `/scope/<param>` (default) or `/scope/<node>/<param>` (per-instance, opt-in) | One bare-address entry per param while no per-node override is set; switches to namespaced when you customize the node. |
49+
| **Graph nodes** (Source / Sink / Slider / XYPad / Bool / Trigger / Tempo / Output / Note / Primitive) | The graph editor's per-node `oscConfig` | `/scope/<node>/<param>` | Off by default; opted in via right-click → **Configure OSC…** |
50+
51+
The Source / Sink / UI-node layer is the new piece — previously only pipeline params could be reached.
52+
53+
---
54+
55+
## Configure OSC per Node
56+
57+
The graph editor's right-click menu has a **"Configure OSC…"** item:
58+
59+
1. Right-click any node → **Configure OSC…**
60+
2. The modal lists every OSC-eligible param for that node type. Each row has:
61+
62+
| Column | Behavior |
63+
|---|---|
64+
| **Expose** | Tick to publish this param's address. Off by default (except for pipeline runtime params, which are auto-exposed at the legacy flat address until you customize anything). |
65+
| **Address** | Auto-fills as `/scope/<node-slug>/<param>`. Editable — paste any address you want (e.g. `/scope/tempo`, `/scope/main/prompt`). |
66+
| **Default** | Advisory metadata published in the OSC docs so external clients can mirror Scope's starting state. Defaults to the node's current value. |
67+
68+
3. **Save**. The graph re-publishes its OSC inventory to the backend within ~300ms; the new address becomes reachable immediately.
69+
70+
`oscConfig` is stored on the node and round-trips with the rest of the graph (saving, exporting, re-importing all preserve it).
71+
72+
> [!NOTE]
73+
> A param set's default is **advisory** — it appears in the description so OSC clients can initialize their UI to match Scope, but Scope does not auto-emit it on session start. Auto-apply is a deliberate follow-up.
74+
75+
### What to expose, by node type
76+
77+
| Node | Exposable params |
78+
|---|---|
79+
| Source | `sourceMode` (enum), `sourceFlipVertical` (bool) |
80+
| Output | `outputSinkEnabled` (bool), `outputSinkType` (enum) |
81+
| Slider | `value` (float) |
82+
| XY Pad | `padX`, `padY` (float) |
83+
| Bool | `value` (bool) |
84+
| Trigger | `value` (bool — send `true` to fire) |
85+
| Tempo | `tempoBpm` (float, 20–999), `tempoEnabled` (bool) |
86+
| Primitive | `value` (string) |
87+
| Note | `noteText` (string) |
88+
| Pipeline | every `is_load_param=False` field from the pipeline's schema |
89+
90+
Composite-shape params (knobs[], MIDI channels[], tuple values[]) are intentionally not exposed in the MVP — they need a richer addressing scheme.
91+
92+
---
93+
94+
## Address Format
95+
96+
```
97+
/scope/<node-slug>/<param>
98+
```
99+
100+
`<node-slug>` is derived from the node's display title (the user-editable header), slugified to kebab-case. Falls back to the React Flow node id when the title has no slug-able characters.
101+
102+
Examples:
103+
104+
| Node title | Field | Default address |
105+
|---|---|---|
106+
| `Tempo` (Slider) | `value` | `/scope/tempo/value` |
107+
| `Source` | `sourceMode` | `/scope/source/sourceMode` |
108+
| `Main` (Pipeline) | `prompt` | `/scope/main/prompt` |
109+
| `Secondary` (Pipeline) | `prompt` | `/scope/secondary/prompt` |
110+
111+
> [!IMPORTANT]
112+
> If two nodes resolve to the same slug (two sliders both titled "Tempo"), the most recently re-saved one wins. Rename one or override the address explicitly to avoid collisions. A UI warning is planned.
113+
114+
### Backwards compatibility
115+
116+
Pipeline runtime params keep their **flat** address `/scope/<param>` until you open Configure OSC on the pipeline node and save any change. Existing rigs sending `/scope/prompt`, `/scope/noise_scale`, `/scope/paused`, etc. continue to work without graph-side configuration.
117+
118+
The moment a user opts a pipeline param in (or out) explicitly, the legacy flat alias is replaced by the user's namespaced address for that node. This is what enables driving two pipeline instances independently when they share param names.
119+
120+
---
121+
122+
## Discovering Paths
123+
124+
### In-app
125+
126+
**Settings → OSC** has a "Currently exposed paths" panel. Each row is click-to-copy.
127+
128+
### HTML reference
129+
130+
Open [`http://localhost:8000/api/v1/osc/docs`](http://localhost:8000/api/v1/osc/docs) for an auto-generated reference page. Every path includes its address, type, constraints (min / max / enum), default, and a one-click Python snippet you can paste into TouchDesigner's text DAT.
131+
132+
### Programmatic
133+
134+
```bash
135+
curl -s http://localhost:8000/api/v1/osc/paths | jq
136+
```
137+
138+
Response shape:
139+
140+
```json
141+
{
142+
"active": {
143+
"Runtime": [
144+
{ "key": "prompt", "type": "string", "osc_address": "/scope/prompt", … }
145+
],
146+
"streamdiffusionv2": [
147+
{ "key": "noise_scale", "type": "float", "min": 0.0, "max": 1.0, "default": 0.7, "osc_address": "/scope/noise_scale", … }
148+
],
149+
"Tempo": [
150+
{ "osc_address": "/scope/tempo/value", "type": "float", "default": 1.0, "node_id": "slider-1", "param": "value", … }
151+
]
152+
},
153+
"available": { … },
154+
"active_pipeline_ids": ["streamdiffusionv2"]
155+
}
156+
```
157+
158+
`active` groups everything currently reachable. `available` lists pipelines that exist in the registry but aren't loaded yet — their addresses become reachable as soon as the pipeline is loaded.
159+
160+
---
161+
162+
## Defaults in the Description
163+
164+
Every path entry that has a default value carries it in two places:
165+
166+
- **`/api/v1/osc/paths`** — `default` field on the entry.
167+
- **`/api/v1/osc/docs`** — Default column in the rendered table.
168+
169+
Defaults come from (in order):
170+
171+
1. The user-set value in **Configure OSC…** (per node).
172+
2. The node's current `data.<param>` value (e.g. the slider's current position).
173+
3. The pipeline schema's `default` (for pipeline runtime params).
174+
175+
This lets external clients initialize their UI to match Scope's starting state without an extra round-trip.
176+
177+
---
178+
179+
## Validation
180+
181+
The OSC server validates every incoming message against the path's type / min / max / enum constraints before broadcasting. Invalid messages are logged but never reach the pipeline. Toggle **Settings → OSC → Log Messages** to see all messages (valid + invalid) in the Scope logs.
182+
183+
| Type | Accepts | Rejection example |
184+
|---|---|---|
185+
| `float` / `number` | `int` or `float` | string → "type mismatch" |
186+
| `integer` | `int` | float → "type mismatch" |
187+
| `bool` / `boolean` | `bool` / `int` / `float` (truthy = on) | string → "type mismatch" |
188+
| `string` | `str` | int → "type mismatch" |
189+
| `integer_list` | non-empty list of ints | list with non-ints → "type mismatch: item N of type …" |
190+
191+
Out-of-range numeric values are rejected with `"value X below minimum Y"` / `"above maximum Y"`. Enum-violating strings get `"value 'foo' not in allowed values […]"`.
192+
193+
---
194+
195+
## Routing Internals
196+
197+
The OSC server splits incoming messages into three routing buckets based on the matched path entry:
198+
199+
1. **Pipeline node** (entry has both `node_id` and `pipeline_id`) — broadcasts `{node_id, <param>: value}` to all WebRTC sessions; `frame_processor.update_parameters` routes it to the matching pipeline processor.
200+
2. **UI-only node** (entry has `node_id` but no `pipeline_id`) — emits an SSE `osc_command` event the frontend listens to; the React Flow state for the matching node updates locally. No backend processor is involved (Slider / Bool / etc. are frontend-only state).
201+
3. **Registry-derived flat path** (entry has no `node_id`) — broadcasts `{<key>: value}` to all WebRTC sessions, matching legacy behavior.
202+
203+
In all three cases the SSE stream is also fanned out so the frontend can mirror the param change in the UI.
204+
205+
---
206+
207+
## TouchDesigner Setup
208+
209+
1. Add an **OSC Out CHOP** (or **OSC Out DAT** for strings).
210+
2. Set **Network Address** to `127.0.0.1` (or the IP of the machine running Scope).
211+
3. Set **Network Port** to `8000`.
212+
4. Add a channel with the address you want to drive — `/scope/prompt`, `/scope/noise_scale`, `/scope/tempo/value`, etc.
213+
5. Animate or bind the channel value to your TD parameters.
214+
215+
> [!TIP]
216+
> Click the address row in `http://localhost:8000/api/v1/osc/docs` to copy a working Python snippet. Paste it into a TD **Text DAT** for offline testing before wiring up CHOPs.
217+
218+
---
219+
220+
## Python Examples
221+
222+
```python
223+
from pythonosc.udp_client import SimpleUDPClient
224+
225+
client = SimpleUDPClient("127.0.0.1", 8000)
226+
227+
# Drive the active pipeline's prompt
228+
client.send_message("/scope/prompt", "a glowing reef at night")
229+
230+
# Animate noise scale
231+
import time
232+
for v in (0.0, 0.25, 0.5, 0.75, 1.0):
233+
client.send_message("/scope/noise_scale", v)
234+
time.sleep(0.5)
235+
236+
# Toggle a per-node Bool you've exposed at /scope/strobe/value
237+
client.send_message("/scope/strobe/value", True)
238+
239+
# Trigger a node — the same as clicking it once in the UI
240+
client.send_message("/scope/cue/value", True)
241+
```
242+
243+
To listen for parameter changes that originate elsewhere (UI clicks, MIDI, other OSC senders), connect to the SSE stream:
244+
245+
```python
246+
import json, requests
247+
248+
with requests.get("http://localhost:8000/api/v1/osc/stream", stream=True) as r:
249+
for line in r.iter_lines():
250+
if not line or not line.startswith(b"data:"):
251+
continue
252+
event = json.loads(line[len(b"data: "):])
253+
# event = {"type": "osc_command", "key": "tempo/value", "value": 0.85,
254+
# "node_id": "slider-1", "param": "value"}
255+
print(event)
256+
```
257+
258+
---
259+
260+
## REST API
261+
262+
| Method | Endpoint | Purpose |
263+
|---|---|---|
264+
| `GET` | `/api/v1/osc/status` | Listening state, port, host, log-verbosity flag |
265+
| `PUT` | `/api/v1/osc/settings` | Toggle `log_all_messages` |
266+
| `GET` | `/api/v1/osc/paths` | Active + available paths, JSON |
267+
| `GET` | `/api/v1/osc/docs` | Self-contained HTML reference page |
268+
| `GET` | `/api/v1/osc/stream` | Server-Sent Events stream of every received OSC command |
269+
| `POST` | `/api/v1/osc/inventory` | (Internal) Replace the graph-supplied path inventory; called by the frontend whenever `oscConfig` changes |
270+
271+
---
272+
273+
## Limitations
274+
275+
- **Slug collisions** are not yet warned in the UI. Two nodes that share a derived slug share the address; the most recently registered wins.
276+
- **Composite params** (knobs[], midiChannels[], tupleValues[]) are skipped for now.
277+
- **Auto-apply of defaults at session start** is intentionally not implemented; the per-param default is description-only metadata.
278+
- **HDR pipeline params** that don't fit the float/int/bool/string/integer-list type system aren't reachable via OSC.
279+
- **OSC port** is shared with the HTTP server. To run multiple Scope instances on one machine, give each a different `SCOPE_PORT`.

0 commit comments

Comments
 (0)