Muse Studio — AI Story & Video Workspace

Muse Studio is a local-first workspace for planning, visualizing, and iterating on stories and video concepts.

About Basic Usage Of Muse Studio : https://youtu.be/N9FTtNYHzIU

This is create as my hobby, if company want to customize for your workflow, feel free to reach out.

Any developers want to modify this open source project, feel free to reach out, so I can add you in the Contributor list.

It combines:

A frontend (muse-studio) with kanban-style scenes, characters, and “Muse” suggestions
A Python FastAPI backend (muse_backend) that orchestrates LLMs and video/image providers
Tight integration with ComfyUI for image and video generation via your own workflows

This README covers installation, configuration, and day‑to‑day usage of the frontend and backend on your machine.

What's New
Prerequisites
Clone / Open the Project
Frontend Setup — muse-studio (Next.js)
Backend Setup — muse_backend (Python / FastAPI)
ComfyUI — Image & Video Generation
Running the Stack
Using Muse Agent (UI Overview)
Verify Everything Works
Quick Command Reference
Upgrading PyTorch
Troubleshooting
Polished export (Remotion)
Publishing on GitHub
Credits & acknowledgments
Security checks
Changelog (releases)

1. What's New

Release 1.5.5 (2026-09-04) https://github.com/benjiyaya/Muse-Studio/releases/tag/v1.5.5

Extensions + Ask Muse consolidation - Ask Muse capabilities are unified in the Extensions chat flow, with /ask-muse redirected to /mcp-extensions (query-compatible).
Built-in Muse orchestration tools - Added execution support for muse.story, muse.visual, and muse.motion in the unified executor while keeping the MCP tools panel MCP-only.
Session and context upgrades - Multi-session UX now supports create/switch/rename/delete/pin/search; session context persists project_id and scene_id; chat shows active project context chip and improved empty-session start flow.
Attachment/upload improvements - Removed manual upload target selector and now derive upload target from session context (project-level vs session-level) with more reliable ID fallback behavior.
Stability fixes - Added resilient DB/session schema checks, fixed crypto.randomUUID fallback in client IDs, and prevented strict MCP schema argument mismatches from extra injected fields.

Release 1.5.0 (2026-03-28)

Full release notes: see CHANGELOG.md (copy into the GitHub Releases description for tag v1.5.0).

Highlights — features

Version 1.5.0 everywhere that matters: Settings → About, GET /health (version field), and npm packages (muse-studio, @muse/plugin-host, @muse/plugin-sdk, mcp-muse-studio).
OpenRouter as a first-class LLM provider (env-based keys; Settings for model/base URL).
Plugin extensions — Settings → Plugins; scene/playground generation can use ComfyUI or plugin providers with documented fallback behavior.
MCP — mcp-muse-studio/ MCP adapter for external tooling.
Security — scripts/security_dependency_guard.py + SECURITY.md for dependency/supply-chain checks.

Highlights — codebase

muse-studio/lib/generation/ — story + batch scenes logic extracted from API routes (storyGenerationInternals, scenesBatchSupport, openRouterHeaders, comfyPluginGeneration).
muse_backend/app/providers/llm/shared_prompts.py — shared Story Muse prompts for cloud/local LLM providers.
useJobPoll / lib/jobs/jobPolling.ts — shared polling for Comfy and plugin jobs; lib/server/ helpers for paths/ids.
muse-studio/lib/plugin-extension/ + plugin API routes; packages/plugin-host, packages/plugin-sdk, template under packages/plugin-template-zimage-turbo.
APP_VERSION in muse_backend/app/main.py; HOST_MUSE_VERSION in plugin types for third-party compatibility.

Exporting your film with the Video Editor Agent (2026-03-13)

When all your scene videos are ready, Muse can help you turn them into a full film using the Video Editor Agent. You can let the agent assemble a master cut instead of downloading every clip and editing by hand.

Simple Stitch — Use when you’re happy with each scene as it is. The agent takes all scenes marked as Final, arranges them in story order, and joins them into one continuous video. Fast and predictable.
Smart Edit — The agent looks at each final scene, listens to audio, and samples key frames; it prepares an edited version of each scene (today mostly full content, designed to get smarter over time), then assembles them into the final film. Good foundation for future pacing and trimming.

In the UI, choose Simple Stitch or Smart Edit before clicking Export Full Film By Agent. Muse shows progress and then gives you a link to open or download the final film.

Local LLMs — LM Studio support

Muse Studio supports LM Studio as a local LLM provider alongside OpenAI, Ollama, and Claude.

Run the LM Studio app and enable its OpenAI-compatible local server.
In muse_backend/.env (optional):
- LMSTUDIO_BASE_URL (default http://127.0.0.1:1234)
- LMSTUDIO_MODEL (model id from LM Studio’s /v1/models)
- LMSTUDIO_API_KEY (if you enabled API auth in LM Studio)
In muse-studio/.env.local: NEXT_PUBLIC_LMSTUDIO_BASE_URL=http://127.0.0.1:1234
In Muse’s provider settings, choose LM Studio (Local) and pick a model. Story Muse (Ask Muse, storyline, scene dialogs) will use LM Studio instead of cloud or Ollama.

Disabling “thinking” output for Qwen models in LM Studio
Some models stream internal reasoning (“thinking”) plus the final answer. To hide that in Muse:

In LM Studio → My Models, select your Qwen model.
Open the Inference tab and find Prompt Template (Jinja).
Add at the very beginning of the template: {%- set enable_thinking = false %}
Save and restart inference for that model. Muse will then only show the final answer.

OpenRouter (cloud, OpenAI-compatible)

OpenRouter exposes many models behind one OpenAI-style API. Muse treats it as a first-class provider alongside OpenAI and Claude.

API keys are never stored in the database or Settings UI. Set them only in environment files (same pattern as OpenAI and Anthropic).
muse-studio/.env.local (or your Next.js host env): OPENROUTER_API_KEY — required for storyline generation, Ask Muse, batch scene generation, and other routes that call the LLM from Next.js.
muse_backend/.env: OPENROUTER_API_KEY — required when the active provider is OpenRouter for Python paths: POST /generate/story, long-form scenes (POST /agent/generate-scenes), and LangGraph agents that use get_chat_model.
Optional attribution headers (recommended by OpenRouter): OPENROUTER_HTTP_REFERER, OPENROUTER_APP_TITLE in the same env files you use for that process.
In Settings → Language Model, choose OpenRouter, set the model id (e.g. openai/gpt-4o-mini), and save. Model and base URL sync to muse_config.json via the backend; the key stays in .env only.

2. Prerequisites

Install these tools before starting.

Node.js (frontend)

Platform	Install
Windows	Download from https://nodejs.org (LTS, v18+)
Linux	`sudo apt install nodejs npm` or use nvm

Verify:

node --version
npm --version

Python (backend)

Python 3.11 or 3.12 recommended (3.13 is also supported).

Platform	Install
Windows	Download from https://python.org — check "Add to PATH" during install
Linux	`sudo apt install python3 python3-venv python3-pip`

Verify:

python --version

NVIDIA GPU + CUDA Toolkit (recommended)

Recommended for GPU‑accelerated features. Most flows will still run on CPU but more slowly.

Platform	Install
Windows	Download CUDA Toolkit from https://developer.nvidia.com/cuda-downloads
Linux	`sudo apt install nvidia-cuda-toolkit` or use the NVIDIA runfile installer

Verify:

nvidia-smi

Git

Platform	Install
Windows	https://git-scm.com
Linux	`sudo apt install git`

3. Clone / Open the Project

git clone https://github.com/benjiyaya/Muse-Studio.git
cd Muse-Studio

Project layout:

/                              ← repo / project root
├── muse-studio/               ← Next.js frontend
├── muse_backend/              ← Python FastAPI backend
├── packages/
│   └── remotion-film/         ← Remotion “FilmMaster” composition (polished export)
└── README.md                  ← This file

Image and video generation run through ComfyUI; no local model folder is required in this repo.

3.1 Quick Start (TL;DR)

Clone & install

Backend:

cd muse_backend
python -m venv .venv
# Activate venv (see OS-specific commands below)
pip install -r requirements.txt
cp .env.example .env  # or use Copy-Item on Windows

Frontend:

cd ../muse-studio
npm install
# If .env.local.example exists:
#   cp .env.local.example .env.local
# Else create .env.local and set at minimum:
#   MUSE_BACKEND_URL=http://localhost:8000
#   NEXT_PUBLIC_OLLAMA_BASE_URL=http://127.0.0.1:11434   # if you use Ollama
#   NEXT_PUBLIC_LMSTUDIO_BASE_URL=http://127.0.0.1:1234  # if you use LM Studio

Polished Remotion export (optional — needed for SMART_EDIT_REMOTION / FilmMaster renders):
```
cd ../packages/remotion-film
npm install
```

Configure keys
- Edit muse_backend/.env:
  - OPENAI_API_KEY (Story Muse)
  - HF_TOKEN (for gated HuggingFace models, if you use them)
  - Optional video APIs: KLING_API_KEY, SEEDDANCE_API_KEY, RUNWAY_API_KEY.
Start services
- Backend:
```
cd muse_backend
# activate .venv
python run.py
```
  Backend: http://localhost:8000
- Frontend:
```
cd muse-studio
npm run dev
```
  Frontend: http://localhost:3000
Run ComfyUI
- Start your ComfyUI server (local or remote).
- In the app UI (/settings → ComfyUI), set the ComfyUI base URL and register one or more workflows.
Create a project
- Open http://localhost:3000, create a new project, pick a Muse Control Level (Observer / Assistant / Collaborator), then add scenes and generate images / video via your ComfyUI workflows.

4. Frontend Setup — muse-studio (Next.js)

Windows (CMD or PowerShell)

cd muse-studio
npm install

Create the local environment file:

copy .env.local.example .env.local

If .env.local.example does not exist, create .env.local manually with:

MUSE_BACKEND_URL=http://localhost:8000
# Optional local LLM servers (frontend URLs)
NEXT_PUBLIC_OLLAMA_BASE_URL=http://127.0.0.1:11434
NEXT_PUBLIC_LMSTUDIO_BASE_URL=http://127.0.0.1:1234

Start the dev server:

npm run dev

Frontend runs at http://localhost:3000

Linux / macOS

cd muse-studio
npm install
cp .env.local.example .env.local   # or create manually — see above
npm run dev

5. Backend Setup — muse_backend (Python / FastAPI)

All commands below are run from inside the muse_backend/ directory unless stated otherwise.

5.1 Create the virtual environment

The .venv isolates AI model dependencies (especially CUDA PyTorch) from your system Python. Never skip this step.

Windows (CMD)

cd muse_backend
python -m venv .venv

Windows (PowerShell)

cd muse_backend
python -m venv .venv

Linux / macOS

cd muse_backend
python3 -m venv .venv

5.2 Activate the virtual environment

You must activate the venv every time you open a new terminal. Your prompt will show (.venv) when active.

Windows (CMD)

.venv\Scripts\activate.bat

Windows (PowerShell)

.\.venv\Scripts\Activate.ps1

If you see an execution policy error, run once: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Linux / macOS

source .venv/bin/activate

Deactivate when done:

deactivate

5.3 Install PyTorch with CUDA

Critical: Do NOT install torch from requirements.txt or plain pip install torch. Both give a CPU-only build with no GPU support. Always use the --index-url flag.

First, check your CUDA version:

nvidia-smi

Look for the CUDA Version in the top-right of the output, then pick the matching install command below.

CUDA 12.8 (RTX 50xx Blackwell — e.g. RTX PRO 6000)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

CUDA 12.4

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

CUDA 12.1 (RTX 30xx / 40xx Ada)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

CUDA 11.8 (older GPUs)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Verify GPU is detected after install:

python -c "import torch; print('Torch:', torch.__version__); print('CUDA:', torch.cuda.is_available()); print('GPU:', torch.cuda.get_device_name(0))"

Expected output example:

Torch: 2.10.0+cu128
CUDA: True
GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition

5.4 Install remaining dependencies

pip install -r requirements.txt

This installs FastAPI, Uvicorn, OpenAI SDK, and other non-GPU packages.

5.5 Configure environment variables

Windows (CMD)

copy .env.example .env

Windows (PowerShell)

Copy-Item .env.example .env

Linux / macOS

cp .env.example .env

Then open .env and fill in your API keys:

# OpenAI (Story Muse, cloud)
OPENAI_API_KEY=sk-...

# HuggingFace (required for gated models like FLUX.2-klein safetensors)
HF_TOKEN=hf_...

# Local LLMs (optional)
# Ollama — local LLM server
# OLLAMA_BASE_URL=http://localhost:11434
# OLLAMA_MODEL=llama3.2

# LM Studio — local LLM server (OpenAI-compatible API)
# LMSTUDIO_BASE_URL=http://localhost:1234
# LMSTUDIO_MODEL=gpt-4o-mini
# LMSTUDIO_API_KEY=your_token_if_enabled

# Cloud video APIs (optional — set whichever you use)
KLING_API_KEY=
SEEDDANCE_API_KEY=
RUNWAY_API_KEY=

6. ComfyUI — Image & Video Generation

Image and video generation are handled by ComfyUI. Run ComfyUI separately and integrate its workflows into Muse Studio:

In ComfyUI, rename your input and output nodes so their titles end with (Input) or (Output) — Muse uses these suffixes to detect dynamic inputs and output.
In ComfyUI, open your workflow and export it as API JSON (or copy the JSON from the workflow editor).
In Muse Studio, go to Settings → ComfyUI → Add workflow and paste the ComfyUI JSON.
- Give the workflow a clear name, choose its kind (image or video), and save it.
Your saved workflows now appear in the Muse workflow library and can be:
- Assigned per scene on the kanban board (image / video workflow fields).
- Used from “Generate with ComfyUI” dialogs for scenes and character sheets.

You do not need a models/ folder in this repo — model weights and pipelines live entirely in your ComfyUI installation.

7. Running the Stack

Both services must be running simultaneously. Use two separate terminal windows.

Terminal 1 — Python backend

Windows (CMD)

cd muse_backend
.venv\Scripts\activate.bat
python run.py

Windows (PowerShell) for Agent Debug and In Browser

cd muse_backend
.\.venv\Scripts\Activate.ps1
python run.py

Linux / macOS

cd muse_backend
source .venv/bin/activate
python run.py

Backend starts at:

API: http://localhost:8000
Docs: http://localhost:8000/docs
Health: http://localhost:8000/health

Terminal 2 — Next.js frontend

Windows (CMD or PowerShell) for Agent Debug and In Browser

cd muse-studio
npm run dev

Linux / macOS

cd muse-studio
npm run dev

Frontend starts at http://localhost:3000

8. Using Muse Agent (UI Overview)

Once both backend and frontend are running and ComfyUI is configured:

Create a project
- Go to http://localhost:3000.
- Click New Project and enter:
  - Title and optional Description.
  - Muse Control Level:
    - Observer – light, occasional suggestions.
    - Assistant – balanced help (default).
    - Collaborator – very proactive suggestions.
Storyline and scenes
- Define your storyline for the project.
- Add scenes on the kanban board; Muse can suggest improvements based on your control level.
Characters
- Use the Characters panel to define key characters.
- Optionally generate character sheet images via ComfyUI workflows when available.
ComfyUI workflows (image & video)
- In Settings → ComfyUI, register one or more workflows from your ComfyUI instance.
- On the kanban board, assign:
  - An image workflow per scene for stills.
  - A video workflow per scene for motion.
- For video workflows, ensure the workflow's (Output) node uses the native SaveVideo node from comfy-core, so the backend downloads an actual playable .mp4 (not a PNG thumbnail).
- Use the Generate with ComfyUI dialogs on scenes or character sheets to run those workflows, then review results.
Muse suggestions
- As you confirm storylines, edit scenes, and complete video jobs, Muse surfaces suggestions in the UI.
- You can adjust the project’s Muse Control Level later from the project header if you want more or fewer suggestions.

9. Verify Everything Works

Check backend health

Open in browser or run:

Windows (CMD / PowerShell)

curl http://localhost:8000/health

Linux / macOS

curl http://localhost:8000/health

Expected response (example):

{
  "status": "ok",
  "version": "1.5.5",
  "available_providers": { ... }
}

Verify GPU in Python

python -c "import torch; print(torch.__version__, '| CUDA:', torch.cuda.is_available(), '| GPU:', torch.cuda.get_device_name(0))"

Verify diffusers

python -c "from diffusers import LTXImageToVideoPipeline; print('diffusers OK')"

Verify ltx-core upsampler

python -c "from ltx_core.model.upsampler import upsample_video; print('ltx_core OK')"

10. Quick Command Reference

Task	Windows CMD	Windows PowerShell	Linux / macOS
Activate venv	`.venv\Scripts\activate.bat`	`.\.venv\Scripts\Activate.ps1`	`source .venv/bin/activate`
Deactivate venv	`deactivate`	`deactivate`	`deactivate`
Start backend	`python run.py`	`python run.py`	`python run.py`
Start frontend	`npm run dev`	`npm run dev`	`npm run dev`
Check GPU	`nvidia-smi`	`nvidia-smi`	`nvidia-smi`
Check CUDA in Python	`python -c "import torch; print(torch.cuda.is_available())"`	same	same
Install a new package	`pip install <pkg>`	`pip install <pkg>`	`pip install <pkg>`
Freeze dependencies	`pip freeze > requirements.txt`	same	same

11. Upgrading PyTorch

When a new PyTorch version is released, upgrade with the same CUDA index URL. Do not use plain pip install torch --upgrade — that installs a CPU build.

All platforms (replace cu128 with your CUDA version)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 --upgrade

After upgrading, verify CUDA is still available:

python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

12. Troubleshooting

`CUDA available: False` after install

PyTorch was installed without the --index-url flag. Fix:

pip uninstall torch torchvision torchaudio -y
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

PowerShell execution policy error

.venv\Scripts\Activate.ps1 cannot be loaded because running scripts is disabled

Fix (run once as your user):

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

`ltx-core` installs a CPU torch

After installing ltx-core, always verify and reinstall if needed:

python -c "import torch; print(torch.__version__)"
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

`nmake not found` / `CMAKE_C_COMPILER not set` when installing llama-cpp-python

Visual Studio C++ Build Tools are missing. Install them first (one-time):

winget install Microsoft.VisualStudio.2022.BuildTools --override "--wait --passive --add Microsoft.VisualStudio.Workload.VCTools --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.Windows11SDK.22621"

After it finishes, close and reopen your terminal, then retry the llama-cpp-python install step in Section 5 (Backend Setup).

`llama-cpp-python` has no GPU support (CPU-only build)

It was installed without CMAKE_ARGS. Uninstall and rebuild:

Windows (CMD) — open a fresh terminal after installing Build Tools

pip uninstall llama-cpp-python -y
set CMAKE_ARGS=-DGGML_CUDA=on
set FORCE_CMAKE=1
pip install llama-cpp-python --no-cache-dir

Linux / macOS

pip uninstall llama-cpp-python -y
CMAKE_ARGS="-DGGML_CUDA=on" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir

`ModuleNotFoundError` when starting the backend

You are not inside the .venv. Activate it first, then retry:

# Windows CMD
.venv\Scripts\activate.bat && python run.py

# Linux / macOS
source .venv/bin/activate && python run.py

Frontend cannot reach the backend (`503 Backend unreachable`)

Make sure the Python backend is running in a separate terminal
Check muse-studio/.env.local contains MUSE_BACKEND_URL=http://localhost:8000
Confirm the backend is healthy: curl http://localhost:8000/health

Port already in use

Windows (CMD / PowerShell) — kill process on port 8000

netstat -ano | findstr :8000
taskkill /PID <PID> /F

Linux / macOS

lsof -ti:8000 | xargs kill -9

13. Polished export (Remotion)

The packages/remotion-film package is a Remotion composition used for Muse FilmMaster exports (polished timeline render). The Python backend can shell out to npx remotion render from this directory when you use the Remotion-based export path. For backend environment variables (MUSE_VIDEO_HTTP_BASE, MUSE_REMOTION_PACKAGE_PATH), see muse_backend/README.md → Polished export (Remotion) — optional.

Prerequisites

Node.js 18+ (LTS recommended) and npm on your PATH.
npx (ships with npm) — the backend invokes npx remotion render from this package directory.

Installation

From the repository root:

cd packages/remotion-film
npm install

Run npm install again after dependency changes or a fresh clone.

Paths and HTTP video sources

By default the backend expects this project at <repo>/packages/remotion-film.
If it lives elsewhere, set MUSE_REMOTION_PACKAGE_PATH on the backend to the absolute path of the folder that contains package.json.

Remotion loads scene clips over HTTP(S) during render. The backend rewrites timeline URLs using MUSE_VIDEO_HTTP_BASE (see muse_backend/README.md).

Default: http://127.0.0.1:3000 (typical muse-studio dev server).
Set MUSE_VIDEO_HTTP_BASE on the Python backend to match wherever Muse Studio is reachable from the machine running the render (often the same idea as MUSE_FRONTEND_BASE_URL in muse-studio/.env.local, but Next.js does not load that file into Python — set the backend env explicitly or via muse_backend/.env if your launcher loads it).

Muse Studio must be running and able to serve /api/outputs/... at that origin while an export runs.

Dev — Remotion Studio

Preview the composition locally:

cd packages/remotion-film
npm run studio

Render (CLI)

Requires a props JSON file (same shape as the backend FilmTimeline):

cd packages/remotion-film
npx remotion render src/index.ts FilmMaster out/video.mp4 --props=timeline.fixtures.json

Fixture render (writes props then renders):

npm run render:fixture

Transitions

Timeline JSON may include transitionOut on each sequence except the last (Muse strips it on the final clip). type: "fade" with durationSec produces a crossfade via @remotion/transitions (TransitionSeries + fade()). cut or missing values are hard cuts. Durations are clamped to the shorter of the two adjacent clips and to a small minimum fade length in the Remotion layer.

Smart Edit / Remotion export: the editor LLM supplies transitionOut per scene; the backend normalizes it (fade duration clamped to 0.05–2.0 s, unknown types become cut).

ffmpeg-only Smart Edit master concat does not apply these transitions; polished Remotion export and the Muse Studio player preview do.

Environment summary

Variable	Where	Purpose
`MUSE_REMOTION_PACKAGE_PATH`	Python backend	Optional absolute path to this package if not under `<repo>/packages/remotion-film`.
`MUSE_VIDEO_HTTP_BASE`	Python backend	Origin where Muse Studio serves video URLs Remotion will fetch (default `http://127.0.0.1:3000`).

14. Publishing on GitHub

This repository is hosted at github.com/benjiyaya/Muse-Studio.

Do not commit API keys or secrets: use muse_backend/.env and muse-studio/.env.local (see .gitignore and .env.example files).
Generated media under muse-studio/outputs/ (draft images, videos, timelines, final cuts) is ignored so clones stay small; recreate outputs locally after install.
muse-studio, muse_backend, and packages/ (including packages/remotion-film) are all part of the same repo — clone once at the root and follow the setup sections above.

This project is licensed under the MIT License — see LICENSE.

15. Credits & acknowledgments

Remotion — Muse Studio’s polished film export and timeline preview build on Remotion, the React-based framework for programmatic video. Thank you to the Remotion team and community for the tools and documentation that make headless, code-driven rendering practical in this stack.

16. Security checks

Run the supply-chain guard script from repo root:

python scripts/security_dependency_guard.py

The guard scans for known IOC strings and denylisted dependency versions (including litellm==1.82.8), then exits non-zero on findings so it can be used in CI.

For remediation guidance, see SECURITY.md.

Changelog releases

Version history and detailed release notes (features, codebase paths, docs) live in CHANGELOG.md. For v1.5.0, open that file and paste the [1.5.0] section into the GitHub Release body for tag v1.5.0 if you want the web UI to mirror the repo.

Name		Name	Last commit message	Last commit date
Latest commit History 45 Commits
.github		.github
comfyui_example_apiworkflows		comfyui_example_apiworkflows
mcp-muse-studio		mcp-muse-studio
muse-studio		muse-studio
muse_backend		muse_backend
packages		packages
scripts		scripts
.gitignore		.gitignore
20260904-update.md		20260904-update.md
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
NOTES.md		NOTES.md
PlugIns Development Documentation.md		PlugIns Development Documentation.md
README.md		README.md
SECURITY.md		SECURITY.md
Setup steps.txt		Setup steps.txt
run-muse.bat		run-muse.bat
run-muse.ps1		run-muse.ps1
run-muse.sh		run-muse.sh

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

Muse Studio — AI Story & Video Workspace

Table of Contents

1. What's New

Release 1.5.5 (2026-09-04) https://github.com/benjiyaya/Muse-Studio/releases/tag/v1.5.5

Release 1.5.0 (2026-03-28)

Exporting your film with the Video Editor Agent (2026-03-13)

Local LLMs — LM Studio support

OpenRouter (cloud, OpenAI-compatible)

2. Prerequisites

Node.js (frontend)

Python (backend)

NVIDIA GPU + CUDA Toolkit (recommended)

Git

3. Clone / Open the Project

3.1 Quick Start (TL;DR)

4. Frontend Setup — muse-studio (Next.js)

Windows (CMD or PowerShell)

Linux / macOS

5. Backend Setup — muse_backend (Python / FastAPI)

5.1 Create the virtual environment

Windows (CMD)

Windows (PowerShell)

Linux / macOS

5.2 Activate the virtual environment

Windows (CMD)

Windows (PowerShell)

Linux / macOS

5.3 Install PyTorch with CUDA

CUDA 12.8 (RTX 50xx Blackwell — e.g. RTX PRO 6000)

CUDA 12.4

CUDA 12.1 (RTX 30xx / 40xx Ada)

CUDA 11.8 (older GPUs)

5.4 Install remaining dependencies

5.5 Configure environment variables

Windows (CMD)

Windows (PowerShell)

Linux / macOS

6. ComfyUI — Image & Video Generation

7. Running the Stack

Terminal 1 — Python backend

Windows (CMD)

Windows (PowerShell) for Agent Debug and In Browser

Linux / macOS

Terminal 2 — Next.js frontend

Windows (CMD or PowerShell) for Agent Debug and In Browser

Linux / macOS

8. Using Muse Agent (UI Overview)

9. Verify Everything Works

Check backend health

Windows (CMD / PowerShell)

Linux / macOS

Verify GPU in Python

Verify diffusers

Verify ltx-core upsampler

10. Quick Command Reference

11. Upgrading PyTorch

All platforms (replace cu128 with your CUDA version)

12. Troubleshooting

CUDA available: False after install

PowerShell execution policy error

ltx-core installs a CPU torch

nmake not found / CMAKE_C_COMPILER not set when installing llama-cpp-python

llama-cpp-python has no GPU support (CPU-only build)

Windows (CMD) — open a fresh terminal after installing Build Tools

Linux / macOS

ModuleNotFoundError when starting the backend

Frontend cannot reach the backend (503 Backend unreachable)

Port already in use

Windows (CMD / PowerShell) — kill process on port 8000

Linux / macOS

13. Polished export (Remotion)

Prerequisites

Installation

Paths and HTTP video sources

Dev — Remotion Studio

`CUDA available: False` after install

`ltx-core` installs a CPU torch

`nmake not found` / `CMAKE_C_COMPILER not set` when installing llama-cpp-python

`llama-cpp-python` has no GPU support (CPU-only build)

`ModuleNotFoundError` when starting the backend

Frontend cannot reach the backend (`503 Backend unreachable`)

Packages