docs: terminal architecture redesign spec (muxterm + ghostty-web)

2026-05-28 05:10:41 +00:00
parent f7ba0a29e5
commit 1a34ce75b3
1 changed files with 280 additions and 0 deletions
@@ -0,0 +1,280 @@
+# Terminal Architecture Redesign: muxterm + ghostty-web
+
+## Goal
+
+Replace the multi-hop terminal data path (xterm.js → Python WS proxy → ttyd → tmux) with a direct architecture (ghostty-web → muxterm Go binary → tmux) that eliminates the reliability failures and latency inherent in managing ttyd as an external process through a Python relay.
+
+## Background
+
+The current terminal data path has fundamental architectural problems causing reliability failures (blank terminals, race conditions, process lifecycle hell) and performance issues (~700ms-1.2s session switch latency, up to 2.5s worst case):
+
+```
+Browser (xterm.js) ←WS→ Python FastAPI (dumb byte relay) ←WS→ ttyd ←PTY→ tmux attach
+```
+
+Every keystroke and terminal output byte traverses two WebSocket hops through a Python async relay that adds zero value during active use. The relay is a 100% dumb byte pipe — zero inspection, zero logging, zero transformation of frames. It exists solely because Caddy was removed to make muxplex distributable as a single-port Python tool (`uv tool install`), and the Python proxy replaced Caddy's WS forwarding role. Auth became a load-bearing side benefit.
+
+ttyd is locked to one command per process, forcing a kill/spawn cycle on every session switch. This spawned increasing layers of complexity — process pools, port range allocation (7682-7701), PID file management, terminal instance caches, sleep delays, lsof fallbacks — all patching symptoms of the wrong architecture.
+
+The "must be pure Python" constraint that drove this design doesn't actually exist. Python packages can include platform-specific compiled binaries via wheels — ruff (Rust), cmake (C++), and the Zig compiler all ship this way via `uv tool install`.
+
+## Approach
+
+**Approach A (chosen): Go terminal multiplexer binary, Python keeps the dashboard.**
+
+A small, purpose-built Go binary (`muxterm`) owns the entire terminal data path. One process, one WebSocket hop, direct PTY ownership via `creack/pty`. Session switching is server-side: muxterm holds a PTY per attached session and routes the active one to the WebSocket. No ttyd. No proxy.
+
+Python keeps what it's good at: the dashboard API, session enumeration, settings, federation orchestration, poll loop.
+
+**Rejected alternatives:**
+
+- **Approach B (Python owns PTYs directly):** Single language, simplest distribution. But Python is still in every byte's path. `asyncio` PTY I/O works but isn't great under heavy output. Signal handling and PTY lifecycle in Python requires careful work. Fighting the language for a systems task.
+
+- **Approach C (expose ttyd directly, bypass Python):** Eliminates the relay, but ttyd is still one-command-per-process. Session switching still requires process lifecycle management. Moves the complexity rather than eliminating it.
+
+## Architecture
+
+Replace the terminal data path only. The Python dashboard/API stays.
+
+**What changes:** ttyd + Python WS proxy → muxterm (Go binary), xterm.js → ghostty-web  
+**What doesn't change:** Python FastAPI (dashboard, API, settings, federation, session enumeration, poll loop), frontend JS (app.js, sidebar, grid, views, settings, style.css), sessions.py, index.html (SPA shell)
+
+```
+UNCHANGED:  Browser ←HTTP→ Python FastAPI (dashboard, API, settings, federation)
+CHANGED:    Browser ←WS→   muxterm (Go)  ←PTY→ tmux attach
+```
+
+Full stack:
+
+```
+Browser                          Server
+┌──────────────────┐            ┌─────────────────────────────┐
+│  ghostty-web     │            │  muxterm (Go)               │
+│  (WASM parser)   │◄──── WS ──►  PTY pool (creack/pty)      │
+│  + Canvas render │            │  session routing             │
+│                  │            │  control protocol             │
+└──────────────────┘            └─────────────────────────────┘
+                                         ▲
+┌──────────────────┐                     │ unix socket
+│  app.js          │            ┌────────┴────────────────────┐
+│  (dashboard,     │◄── HTTP ──►│  Python FastAPI              │
+│   sidebar, grid, │            │  dashboard API, settings,    │
+│   settings)      │            │  session enum, federation,   │
+└──────────────────┘            │  auth, poll loop             │
+                                └─────────────────────────────┘
+```
+
+Python starts muxterm as a managed subprocess on startup (one process, started once). muxterm listens on a Unix socket (e.g., `/tmp/muxterm.sock`) — not a TCP port. FastAPI handles the initial WebSocket upgrade at `/terminal/ws` (auth check, then hands off the raw connection to muxterm). After the handshake, Python is out of the data path entirely.
+
+## Components
+
+### muxterm (Go binary)
+
+muxterm manages a map of sessions → PTYs. Each entry:
+
+```
+session "dev-server" → {
+    pty_fd:    file descriptor (master side of PTY pair)
+    process:   child process handle (tmux attach -t dev-server)
+    size:      {cols, rows}
+}
+```
+
+**Lifecycle:**
+
+- **Attach:** Client requests session → muxterm checks the map. If no entry, calls `pty.Start(exec.Command("tmux", "attach", "-t", session_name))` → stores fd + process. If entry exists and process is alive, reuses it. ~1ms cold start.
+- **Detach:** When the last WebSocket disconnects from a session, the PTY stays alive (tmux is still attached). Next connection reuses it instantly.
+- **Cleanup:** On tmux session death (process exits), muxterm removes the map entry and closes the fd. Detected via process wait.
+
+**Session switching over one WebSocket:** Binary frames for terminal I/O (hot path), text frames for control messages (cold path). When the browser sends an attach command, muxterm swaps which PTY fd the relay goroutines read/write. The old PTY stays alive in the map. Switch time: a pointer swap — microseconds.
+
+**Size:** ~500-1000 lines of Go. Dependencies: `creack/pty` for PTY management, a WebSocket library (gorilla or nhooyr) for the browser-facing endpoint.
+
+### ghostty-web (frontend terminal)
+
+Drop-in replacement for xterm.js. Same `new Terminal()` API. Uses libghostty-vt compiled to WebAssembly — the same VT parser / terminal state machine that runs the native Ghostty desktop terminal. Battle-tested, fuzz-tested, 3+ years of production use.
+
+**What it provides:** Terminal state management, VT/ANSI/XTERM escape sequence parsing, cursor position tracking, styles, text reflow, scrollback, Unicode/grapheme handling, Kitty keyboard protocol, mouse tracking.
+
+**What it doesn't provide:** PTY management, networking, WebSocket — those are muxterm's job.
+
+**Bundle size:** ~400KB WASM.
+
+### Python FastAPI (unchanged role)
+
+Keeps all non-terminal responsibilities:
+- Dashboard API (session list, delete, rename, state)
+- Settings management
+- Federation orchestration
+- Session enumeration and snapshot polling
+- Auth cookie/bearer/localhost management
+- Static file serving (SPA)
+- muxterm process supervision (start, restart on crash)
+
+## Data Flow
+
+### Terminal I/O (hot path)
+
+```
+Keystroke → ghostty-web → WS binary frame → muxterm → write(pty_fd) → tmux
+tmux output → read(pty_fd) → muxterm → WS binary frame → ghostty-web → render
+```
+
+One hop. Raw bytes. No relay, no framing overhead, no Python in the path.
+
+### Session switch
+
+1. User clicks sidebar tile → `app.js` sends `ws.send(JSON.stringify({attach: "dev-server"}))`
+2. muxterm receives text frame, parses JSON
+3. muxterm checks PTY map — spawns `tmux attach -t dev-server` if needed (~1ms), reuses existing PTY if present (0ms)
+4. muxterm sends SIGWINCH to the new PTY → tmux repaints the full screen
+5. muxterm swaps relay goroutines to the new PTY fd
+6. muxterm sends `{"attached": "dev-server"}` text frame
+7. Browser receives confirmation, then full screen repaint arrives as normal binary terminal data
+8. ghostty-web renders it — user sees current state of the session immediately
+
+No explicit "screen dump" mechanism. Lean on tmux's native redraw behavior (same as resizing a real terminal).
+
+### Auth flow
+
+1. Browser opens `ws:///terminal/ws`
+2. Python FastAPI runs `_ws_auth_check()` (cookie/bearer/localhost — same as today)
+3. If authorized, upgrades HTTP connection to WebSocket
+4. Hands the raw socket to muxterm via Unix socket passthrough
+5. Python is out of the data path from this point
+
+muxterm has zero auth logic. Trusts any connection on its internal Unix socket (not network-exposed).
+
+## Wire Protocol
+
+Two frame types over one WebSocket. WebSocket natively distinguishes text and binary frame types — no additional framing needed.
+
+### Binary frames — terminal I/O (hot path)
+
+- **Client → server:** Raw bytes from keyboard, sent directly to active PTY
+- **Server → client:** Raw bytes from PTY, sent directly to ghostty-web
+- No type prefix byte, no envelope. Raw bytes in, raw bytes out. Zero overhead per frame.
+
+### Text frames — control messages (cold path)
+
+**Client → Server:**
+
+| Message | Purpose |
+|---------|---------|
+| `{"attach": "session-name"}` | Switch active PTY |
+| `{"resize": {"cols": 120, "rows": 40}}` | Resize active PTY |
+| `{"detach": true}` | Detach (keep PTY alive) |
+
+**Server → Client:**
+
+| Message | Purpose |
+|---------|---------|
+| `{"attached": "session-name"}` | Confirm switch complete |
+| `{"error": "session not found"}` | Attach failed |
+| `{"exited": "session-name"}` | tmux session died, PTY closed |
+
+### Design rationale (informed by Zellij)
+
+Zellij uses two separate WebSocket connections for terminal data and control. They noted this separation "ended up not being necessary" because their application layer already batches messages efficiently. Our single-channel approach with binary/text frame types achieves the same logical separation without managing two connections.
+
+The SIGWINCH-based screen sync on session switch also comes from studying Zellij's approach — instead of building a screen dump mechanism, leverage tmux's native redraw behavior.
+
+## Error Handling
+
+| Failure | Response |
+|---------|----------|
+| `tmux attach` fails (session doesn't exist) | Send `{"error": "session not found"}` over control channel. Browser shows toast. |
+| PTY process exits (tmux session killed) | Send `{"exited": "session-name"}`. Browser returns to dashboard grid. Clean up map entry. |
+| muxterm crashes | Python detects child exit, restarts muxterm. Browser WS reconnect fires, reattaches. PTYs die with the process, but tmux sessions survive — new PTYs created on reattach. |
+| WebSocket drops (network) | Browser reconnects, sends `{"attach": "last-session"}`. muxterm reuses existing PTY. Seamless. |
+| Python crashes | muxterm keeps running (independent process). Terminal stays live. Dashboard API down until Python restarts, but terminal is uninterrupted. |
+
+**Key resilience property:** tmux sessions are the source of truth. Everything else (PTYs, WebSockets, muxterm, Python) is ephemeral and reconstructable. Nothing persists that can get stale or corrupt.
+
+## Distribution & Lifecycle
+
+### muxterm binary
+
+Built as a Go binary, cross-compiled for linux/amd64, linux/arm64, darwin/amd64, darwin/arm64. Bundled into the Python package as platform-specific wheels (same pattern as ruff). `uv tool install muxplex` gets you everything — Python dashboard + Go terminal server.
+
+### ghostty-web
+
+Vendored into `muxplex/frontend/vendor/` alongside existing vendor pattern. WASM bundle is ~400KB. xterm.js files get removed.
+
+### Process lifecycle
+
+- Python FastAPI starts muxterm as a managed subprocess on startup (one process, started once)
+- muxterm listens on a Unix socket — not a TCP port
+- If muxterm dies, Python detects it and restarts it (simple process supervision)
+- On shutdown, Python sends SIGTERM to muxterm, which gracefully closes all PTYs
+- PTYs are just file descriptors — tmux sessions survive muxterm restarts (tmux is independent)
+
+### What this replaces
+
+- No `kill_orphan_ttyd()` — no orphans possible (Unix socket, single process)
+- No port range allocation — Unix socket, not TCP
+- No PID file management — Python holds the process handle directly
+
+## What Gets Deleted
+
+- `ttyd.py` — all of it (process pool, PID files, port allocation, kill/spawn)
+- Python WS proxy in `main.py` (`terminal_ws_proxy`, `_ttyd_is_listening`, auto-spawn logic)
+- `POST /api/sessions/{name}/connect` endpoint — muxterm handles attach directly
+- Terminal cache in `terminal.js` (no need for multiple xterm instances)
+- ttyd as a system dependency
+- xterm.js vendor files (replaced by ghostty-web)
+
+## What Gets Added
+
+- `muxterm/` — Go module, small binary (~500-1000 lines)
+- ghostty-web npm package vendored into `frontend/vendor/`
+
+## Testing Strategy
+
+### muxterm (Go)
+
+- Unit tests for PTY map operations (attach, detach, cleanup on process exit)
+- Unit tests for wire protocol parsing (control message JSON, binary frame routing)
+- Integration tests: spawn real tmux sessions, attach via WebSocket, verify I/O round-trip
+- Stress test: rapid session switching, concurrent connections
+
+### Frontend
+
+- Verify ghostty-web drop-in compatibility (same Terminal API as xterm.js)
+- Test session switch flow: send attach control message, verify screen repaint
+- Test error paths: attach to non-existent session, handle exited notification
+
+### Integration
+
+- End-to-end: browser → Python auth → muxterm → tmux, verify terminal interactivity
+- Failure scenarios: kill muxterm mid-session (verify Python restarts it, browser reconnects)
+- Verify WebSocket drop recovery (disconnect network, reconnect, verify session resumes)
+
+## Migration Path
+
+Migration is incremental, not a big bang. Two independent phases:
+
+**Phase 1 — ghostty-web swap:** Replace xterm.js with ghostty-web in `frontend/vendor/`. Update script tags in `index.html`. API-compatible drop-in. Existing ttyd+proxy architecture stays. Immediate benefit: better VT parsing, fewer rendering bugs.
+
+**Phase 2 — muxterm Go binary:** Build the Go binary, add it to the Python package, update Python startup to launch it, wire the WS endpoint. Delete `ttyd.py`, the Python WS proxy, the terminal cache, the process pool.
+
+Either phase can ship and be tested independently.
+
+## Design Decisions & Rationale
+
+1. **Go for the terminal binary, not Rust:** Go's goroutines are purpose-built for the "copy bytes between two fds" pattern. `creack/pty` is mature. Cross-compilation is trivial. The binary is ~500-1000 lines — Go's simplicity is a feature at this scale.
+
+2. **Single WebSocket with binary/text frame separation, not dual channels (Zellij pattern):** Zellij uses two separate WebSocket connections for terminal data and control. They noted the separation "ended up not being necessary." Our frame-type approach achieves the same logical separation without managing two connections.
+
+3. **SIGWINCH for screen sync on session switch:** Instead of building a screen dump mechanism, leverage tmux's native redraw (triggered by SIGWINCH). This is exactly what happens when you resize a terminal — tmux repaints everything.
+
+4. **Unix socket, not TCP:** Eliminates the port range allocation, port conflicts, and orphan scanning that caused so much complexity. Python connects to muxterm via a well-known socket path.
+
+5. **ghostty-web over xterm.js:** API-compatible drop-in with better VT100 parsing via WASM. The terminal industry is moving this direction (Coder's ghostty-web has 2.5K stars). Reduces parser bugs that are outside our control.
+
+## Open Questions
+
+- **ghostty-web maturity:** How stable is the API surface? Are there xterm.js addon equivalents (search, web links, fit) or do those need custom implementation?
+- **WASM bundle caching:** Does the ~400KB ghostty-web WASM bundle cache well across page loads, or does it need explicit Cache-Control headers?
+- **muxterm socket path:** Should the Unix socket path be configurable, or is a well-known path (e.g., `/tmp/muxterm-{user}.sock`) sufficient?
+- **Federation implications:** Remote muxplex instances currently proxy through the Python WS relay. Does muxterm need its own federation-aware mode, or does Python remain the federation proxy (now forwarding to remote muxterm instances)?