AI Co-Pilot for Microsoft Flight Simulator 2024
MERLIN is a voice-interactive AI co-pilot powered by Claude that connects to MSFS 2024 via SimConnect. Talk naturally to control your aircraft, get real-time flight guidance, and manage procedures -- delivered with the personality of a Navy Test Pilot who has seen it all.
"Airdale" is Navy slang for a naval aviator. Fitting, because MERLIN flies right seat.
Voice-Commanded Aircraft Control -- Say "gear down" or "give me full flaps" and MERLIN executes immediately. 20 controllable systems, 72+ actions including flaps, gear, autopilot, throttle, lights, magnetos, fuel selector, trim, deice, and more. See Aircraft Controls Reference.
Real-Time Flight Awareness -- Automatic flight phase detection (preflight through rollout) drives checklists, response style, and proactive callouts. MERLIN adapts: terse during takeoff, conversational during cruise.
Aviation Intelligence -- 6 built-in tools for NOTAMs, METAR/TAF weather, ADS-B traffic, approach charts, performance calculations, and airspace info. RAG-powered manual lookup with aviation-aware semantic chunking.
Safety Layer -- Emergency fast-path detection bypasses the LLM for engine failures, fires, and decompression. V-speed cross-referencing catches hallucinated numbers. Telemetry sanity checks protect against bad data.
Low-Latency Voice -- Deepgram Nova-3 streaming STT (~300ms) with aviation keyword boosting. Cartesia Sonic-3 TTS (~90ms time-to-first-byte). Barge-in interruption support.
MSFS 2024 ──SimConnect──> MSFS Adapter (C#) ──WebSocket──> Telemetry Service
|
Browser UI <──WebSocket──> Web Server (FastAPI) <──────────────────┘
| | |
| Claude API ChromaDB
| (tool use) (RAG store)
| |
└── Deepgram STT Cartesia TTS
(streaming) (~90ms TTFB)
Commands flow bidirectionally: voice in through Deepgram, Claude decides and calls tools, aircraft control commands route through the telemetry service to the SimConnect bridge, and confirmations stream back through Cartesia TTS.
- Docker Desktop with WSL2 backend
- .NET 8 SDK (Windows, for the SimConnect bridge)
- Microsoft Flight Simulator 2024 with the SDK installed
- Python 3.11+
- API keys: Anthropic (required), Deepgram (STT), Cartesia (TTS)
git clone https://github.com/frontsidebus/airdale.git
cd airdale
cp .env.example .env
# Edit .env with your API keys (ANTHROPIC_API_KEY, DEEPGRAM_API_KEY, CARTESIA_API_KEY)
# Also set CARTESIA_VOICE_ID (browse voices at play.cartesia.ai)./scripts/start.shOr test without MSFS running:
./scripts/start.sh --mockNavigate to http://localhost:3838. MERLIN is ready.
./scripts/healthcheck.shTo stop:
./scripts/stop.shManual startup (without script)
# Docker services (ChromaDB + telemetry service)
docker compose up -d chromadb telemetry-service
# SimConnect bridge (Windows terminal)
cd adapters/msfs
dotnet run
# Web server (WSL)
cd web
source ../orchestrator/.venv/bin/activate
python run.py| Layer | Technology |
|---|---|
| Orchestrator | Python 3.11+ / FastAPI (async) |
| SimConnect Bridge | C# / .NET 8 (out-of-process, event-driven) |
| AI Inference | Anthropic Claude API with tool use |
| Speech-to-Text | Deepgram Nova-3 (streaming WebSocket) |
| Text-to-Speech | Cartesia Sonic-3 (~90ms TTFB) |
| Vector Store / RAG | ChromaDB with semantic chunking + cross-encoder re-ranking |
| Telemetry Hub | Python / FastAPI (adapter ↔ consumer routing) |
| IPC | WebSocket (JSON) throughout |
| Frontend | HTML/JS cockpit display |
| Config | pydantic-settings with .env |
Fallback options: Whisper (local STT), ElevenLabs (cloud TTS), Kokoro (local TTS)
airdale/
├── orchestrator/ # Python orchestration package (the brain)
│ ├── orchestrator/
│ │ ├── claude_client.py # Claude API with MERLIN persona + tool use
│ │ ├── tools.py # Tool implementations (sim control, airport lookup, etc.)
│ │ ├── aviation_tools.py # NOTAM, METAR, ADS-B, charts, performance, airspace
│ │ ├── emergency.py # Emergency fast-path detection and response
│ │ ├── validation.py # V-speed cross-referencing, telemetry sanity checks
│ │ ├── chunking.py # Aviation-aware semantic document chunking
│ │ ├── reranker.py # Cross-encoder re-ranking for RAG
│ │ ├── stt/ # STT backends (Deepgram, Whisper)
│ │ ├── tts/ # TTS backends (Cartesia, ElevenLabs, Kokoro)
│ │ └── ...
│ └── tests/ # 619 tests
├── web/ # FastAPI web server + browser cockpit UI
├── telemetry-service/ # Universal telemetry hub (adapter ↔ consumer)
├── adapters/msfs/ # C# SimConnect bridge (.NET 8)
├── data/
│ ├── checklists/ # YAML checklists (single-engine, jet)
│ └── prompts/ # MERLIN persona and emergency prompts
├── tools/
│ ├── mock_adapter.py # Mock MSFS adapter for testing without sim
│ └── ingest.py # Document ingestion into ChromaDB
├── scripts/
│ ├── start.sh # Start all components (supports --mock)
│ ├── stop.sh # Graceful shutdown
│ └── healthcheck.sh # Verify all subsystems
├── docs/ # Documentation
│ ├── ARCHITECTURE.md # System design and data flows
│ ├── AIRCRAFT_CONTROLS.md # Complete control reference (20 systems, 72+ actions)
│ ├── VOICE_PIPELINE.md # STT/TTS streaming architecture
│ ├── SAFETY.md # Emergency detection, validation, timeouts
│ ├── AVIATION_TOOLS.md # NOTAM, METAR, ADS-B, charts, performance
│ ├── RAG_SYSTEM.md # Semantic chunking, re-ranking, metadata
│ ├── CONFIGURATION.md # Complete env var reference
│ ├── TESTING.md # Mock adapter, test suite, command testing
│ ├── MIGRATION_V1_V2.md # v1 → v2 migration guide
│ ├── GETTING_STARTED.md # First flight walkthrough
│ └── INSTALL.md # Installation guide
├── .env.example # Environment variable template
├── docker-compose.yml # Production services
└── CLAUDE.md # Project conventions for AI assistants
| Document | Description |
|---|---|
| Getting Started | First flight walkthrough |
| Installation | Full setup with troubleshooting |
| Architecture | System design and data flows |
| Aircraft Controls | Every system and action MERLIN can control |
| Voice Pipeline | STT/TTS streaming, barge-in, latency |
| Safety | Emergency detection, validation, timeouts |
| Aviation Tools | NOTAM, weather, traffic, charts, performance |
| RAG System | Semantic chunking, re-ranking, ingestion |
| Configuration | Complete env var reference |
| Testing | Mock adapter, test suite, command testing |
| Migration v1→v2 | Breaking changes and upgrade guide |
| Project Conventions | Code style, architecture decisions |
- Fork the repository
- Create a feature branch (
git checkout -b feature/your-feature) - Ensure
ruff checkpasses for Python code - Run tests:
cd orchestrator && pytest - Submit a pull request with a clear description
MIT -- Copyright 2026 frontsidebus