---
title: "The Architecture Mistake I Made in Lage"
date: 2026-05-27
type: post
tags: [build-systems, lage, rage, rust, architecture, personal]
summary: "I co-created lage at Microsoft. It's used across the company. Its architecture has a fundamental flaw that I could only fix by starting over."
---

I built [lage](https://github.com/microsoft/lage). It runs task orchestration across JS monorepos at Microsoft. Hundreds of developers use it daily. Teams depend on it for CI, for local builds, for the "run `lage build` and it works" experience that makes monorepo development tolerable.

Its architecture has a fundamental flaw. I put it there.

This is the story of how I found the flaw, why patching it was impossible, and what I built to replace it. If you've been following this series, this is the post that connects the other five.

## Where lage came from

In 2020, the JS monorepo ecosystem had a gap. Bazel could orchestrate everything, but the ceremony was punishing: BUILD files, Starlark, a mental model designed for Google's infrastructure, not for a team of twelve shipping a React app. [BuildXL](https://github.com/microsoft/BuildXL) was powerful but pointed in the other direction: designed for 50-machine build farms, not for a developer on a laptop running `npm test`.

What JS teams needed was a task runner that understood the package graph. You have 200 packages. Package `api` depends on `utils`. Run `build` across everything in the right order, skip what hasn't changed, parallelize where possible. No Starlark. No pip declarations. No build farm.

That's what lage did. You wrote a pipeline declaration:

```js
module.exports = {
  pipeline: {
    build: ["^build"],  // upstream deps first
    test: ["build"],    // own build first
    lint: [],           // no deps, run immediately
  }
};
```

Lage resolved the package graph from `package.json` dependencies, built a DAG, scheduled tasks in topological order with parallelism, and cached results by hashing declared inputs. Teams adopted it. At peak, lage orchestrated builds across some of the largest JS monorepos inside Microsoft.

I was proud of it. The design solved a real problem for real teams.

But it had a gap, and the gap was load-bearing.

## File access was an afterthought

Lage's cache worked like every other JS monorepo cache: you declared your inputs, lage hashed them, and if the hash matched, the cache returned stored output. If you forgot to declare an input, the cache returned stale results and called them hits. I covered this failure mode in detail in [Post 1](/posts/your-build-cache-is-lying-to-you). The short version: declared-input trust does not scale. In a monorepo maintained by 40 engineers over 3 years, someone will forget a file. The cache will lie silently.

I knew BuildXL had the fix. Sandbox every task. Observe what files the process reads and writes. Use those observations to build a second-phase fingerprint that catches what declarations miss. The [two-phase fingerprinting scheme](/posts/two-phase-fingerprinting-buildxls-deep-magic) makes the cache correct without requiring developers to be perfect.

So lage got a BuildXL integration path. You could run lage on top of BuildXL, and BuildXL's file access monitoring would feed sandbox data back into the caching layer.

This was a hack. Lage reached into BuildXL's internal file access mechanism rather than having a clean API between the task runner and the sandbox. The coupling was tight: lage couldn't run with sandboxing unless BuildXL was present. BuildXL couldn't expose its sandbox to other runners cleanly. The result worked at Microsoft but was non-portable and impossible to ship as something anyone outside the company could use.

The architecture's original sin was not that lage lacked sandboxing. It was that file access observation was never a first-class primitive. It was grafted on from an external system, and the cache and sandbox lived in different codebases with different APIs and different assumptions.

<div class="breakout">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 840 400" role="img" aria-label="Diagram: The Architecture's Original Sin. Lage has file access bolted on from BuildXL as an external hack. Rage has file access as the foundation that cache and scheduling sit on top of.">
  <defs>
    <marker id="arr" viewBox="0 0 10 10" refX="10" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
      <path d="M 0 0 L 10 5 L 0 10 z" fill="#2B2421"/>
    </marker>
    <marker id="arr-d" viewBox="0 0 10 10" refX="10" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
      <path d="M 0 0 L 10 5 L 0 10 z" fill="#8B7D72"/>
    </marker>
  </defs>
  <rect width="840" height="400" fill="#F5F0E8" rx="4"/>
  <text x="420" y="32" text-anchor="middle" font-family="Georgia, serif" font-size="16" font-weight="bold" fill="#2B2421">The Architecture's Original Sin</text>
  <text x="200" y="64" text-anchor="middle" font-family="Georgia, serif" font-size="14" font-weight="bold" fill="#2B2421">lage</text>
  <rect x="100" y="80" width="200" height="48" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="200" y="108" text-anchor="middle" font-family="monospace" font-size="12" fill="#2B2421">Task Runner</text>
  <line x1="200" y1="128" x2="200" y2="158" stroke="#2B2421" stroke-width="1.5" marker-end="url(#arr)"/>
  <rect x="100" y="160" width="200" height="56" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="200" y="184" text-anchor="middle" font-family="monospace" font-size="12" fill="#2B2421">Cache</text>
  <text x="200" y="202" text-anchor="middle" font-family="Georgia, serif" font-size="10" fill="#8B7D72">declared inputs only</text>
  <line x1="200" y1="216" x2="200" y2="252" stroke="#8B7D72" stroke-width="1.5" stroke-dasharray="6,4" marker-end="url(#arr-d)"/>
  <rect x="80" y="254" width="240" height="60" rx="4" fill="none" stroke="#8B7D72" stroke-width="1.5" stroke-dasharray="6,4"/>
  <text x="200" y="280" text-anchor="middle" font-family="monospace" font-size="12" fill="#8B7D72">BuildXL</text>
  <text x="200" y="300" text-anchor="middle" font-family="Georgia, serif" font-size="10" fill="#8B7D72">File Access Reporting</text>
  <text x="200" y="340" text-anchor="middle" font-family="Georgia, serif" font-size="11" font-style="italic" fill="#8B7D72">Bolted on. External.</text>
  <text x="200" y="358" text-anchor="middle" font-family="Georgia, serif" font-size="11" font-style="italic" fill="#8B7D72">Non-portable.</text>
  <line x1="420" y1="56" x2="420" y2="380" stroke="#D4C9BB" stroke-width="1" stroke-dasharray="4,4"/>
  <text x="640" y="64" text-anchor="middle" font-family="Georgia, serif" font-size="14" font-weight="bold" fill="#2B2421">rage</text>
  <rect x="540" y="80" width="200" height="48" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="640" y="108" text-anchor="middle" font-family="monospace" font-size="12" fill="#2B2421">Task Runner</text>
  <line x1="640" y1="128" x2="640" y2="158" stroke="#2B2421" stroke-width="1.5" marker-end="url(#arr)"/>
  <rect x="540" y="160" width="200" height="56" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="640" y="184" text-anchor="middle" font-family="monospace" font-size="12" fill="#2B2421">Two-Phase Cache</text>
  <text x="640" y="202" text-anchor="middle" font-family="Georgia, serif" font-size="10" fill="#6B5F55">WF + pathset + SF</text>
  <line x1="640" y1="216" x2="640" y2="252" stroke="#2B2421" stroke-width="1.5" marker-end="url(#arr)"/>
  <rect x="520" y="254" width="240" height="60" rx="4" fill="#A0522D" stroke="none"/>
  <text x="640" y="280" text-anchor="middle" font-family="monospace" font-size="13" font-weight="bold" fill="#F5F0E8">Sandbox</text>
  <text x="640" y="300" text-anchor="middle" font-family="Georgia, serif" font-size="10" fill="#F5F0E8">syscall-level file access</text>
  <text x="640" y="340" text-anchor="middle" font-family="Georgia, serif" font-size="11" font-style="italic" fill="#A0522D">Foundation. First-class.</text>
  <text x="640" y="358" text-anchor="middle" font-family="Georgia, serif" font-size="11" font-style="italic" fill="#A0522D">Composable.</text>
</svg>
</div>

## Watching the ceiling arrive

The ceiling didn't arrive all at once. Monorepos grew. New teams onboarded. The number of undeclared dependencies grew with them.

Every few months, someone filed a bug: "my build is stale but CI says it's cached." Every time, the root cause was the same. Find the missing input declaration, add it to the pipeline config, move on. The fix was always one line. The debugging cost was always two days.

The pattern was unmistakable. The system's correctness depended on human discipline, and human discipline degrades as the repository grows. I tried partial fixes: better default globs, documentation about what to declare, tooling to detect common omissions. None addressed the structural problem. Lage's cache was only as correct as its input declarations were complete, and completeness required every developer to understand every file that every build step touched. That is not a reasonable expectation for a tool used by hundreds of people who didn't write the build configuration.

The right fix was the one BuildXL had known for years: observe instead of declare. Make the sandbox a first-class primitive. Let the system see the truth.

That fix couldn't be patched into lage. The architecture assumed file access was external. The cache assumed declared inputs were sufficient. Rebuilding the foundation meant rebuilding everything above it.

## Why Rust, not Go

TypeScript 7.0 Beta [shipped April 21, 2026](https://devblogs.microsoft.com/typescript/announcing-typescript-7-0-beta/). The compiler was rewritten in Go: 10x faster overall, 30x faster type checking, 2.9x less memory. Microsoft chose Go, not Rust. I covered why tsgo makes the build harness more important in [Post 4](/posts/typescript-7-makes-the-build-harness-more-important).

The language question for the harness was separate. If the primary workload is a Go binary (tsgo), does the harness itself need to be Rust?

Yes. For one specific reason.

The harness's most correctness-critical component is the [syscall sandbox](/posts/hooking-syscalls-without-a-kernel-driver). On macOS: `DYLD_INSERT_LIBRARIES` interposing. On Linux: eBPF tracepoints via [aya](https://github.com/aya-rs/aya). On Windows: DLL injection with inline hooking via [retour](https://crates.io/crates/retour). All three mechanisms involve unsafe memory manipulation, function pointer replacement, and cross-process injection. Rust's ownership model and explicit `unsafe` boundaries make these operations auditable. You can grep for `unsafe` blocks and know exactly where the danger lives. Go's garbage collector can move memory during a syscall hook. Rust doesn't have that problem because Rust doesn't have a garbage collector.

| Concern | Rust | Go |
|---------|------|-----|
| Syscall-level sandboxing (eBPF, ptrace, Detours) | Strong safety model | Possible but less idiomatic |
| Parallel task scheduling | Rayon, Tokio | Goroutines (simpler) |
| Single-binary CLI | Yes | Yes |
| Reusable library (`no_std` capable) | Yes | Runtime required |

Go would have been fine for a CLI tool. Rust was the right choice for a library of composable crates with a sandbox that needs to be correct at the syscall level.

## The velocity

The [spec](/posts/two-phase-fingerprinting-buildxls-deep-magic) was complete before the first line of Rust. Every crate boundary, every trait, every platform-specific feature flag was designed on paper. Then the implementation went fast.

17 crates. ~40,000 lines. Phases 1 through 11 in roughly 48 hours of implementation time. Not because the coding was fast. Because the spec was complete and the architecture was designed, not discovered.

```
rage/
├── crates/
│   ├── workspace-tools/         package graph, PM detection
│   ├── build-graph/             DAG, topological sort
│   ├── pipeline-config/         rage.json, policy resolution
│   ├── cache/                   two-phase fingerprinting
│   ├── scoping/                 git-diff affected packages
│   ├── scheduler/               wave-parallel execution
│   ├── sandbox/                 file-access monitoring
│   ├── sandbox-macos-dylib/     DYLD interpose cdylib
│   ├── sandbox-windows-detours/ Detours DLL injection
│   ├── plugin/                  EcosystemPlugin trait
│   ├── plugin-typescript/       TS detection, postinstall
│   ├── daemon/                  watcher, reconciler, HTTP/WS
│   └── telemetry/               Chrome DevTools traces
└── cli/                         thin binary, opinionated defaults
```

Each phase had a plan document, TDD throughout, and a verification gate. Phase 3 (local cache) landed with 73 tests. Phase 9 (two-phase cache) landed with 165. Phase 11 (daemon with HTTP status page) landed with 192. By the end of the postinstall caching work: 350+ tests across the workspace.

The speed came from the spec. When you know what every crate's public API looks like before you open your editor, you don't spend time discovering the architecture. You spend time implementing it. Design documents convert uncertainty into typing.

## What rage doesn't have yet

This is not a victory lap. Rage is a working system with real correctness guarantees. It is not a production tool that 200 engineers depend on tomorrow.

What lage still does better:

**Documentation.** Lage has years of guides, examples, and Stack Overflow answers. Rage has code comments and plan documents. A developer picking it up cold would struggle.

**Ecosystem maturity.** Lage has been hammered by real monorepos for years. Edge cases have been filed, triaged, and fixed. Rage has been tested against fixture workspaces and one production codebase. The long tail of weird things that real packages do hasn't been explored.

**Community.** Lage has users, contributors, and institutional knowledge. Rage has one author and an AI pair programmer.

**Remote cache.** Rage has local cache only. The `CacheProvider` trait exists and the Azure Blob and S3 provider specs are written, but no remote cache implementation ships yet.

**Linux sandbox.** The eBPF sandbox via [aya](https://github.com/aya-rs/aya) is designed and spec'd. The macOS and Windows sandboxes are implemented and tested. Linux is the gap. The fallback is unsandboxed execution with weak-fingerprint-only caching.

These are real gaps. Rage solves the correctness problem that lage cannot. Lage solves the adoption problem that rage has not yet faced. Both matter.

## The thread

This series covered cache correctness, two-phase fingerprinting, syscall sandboxing, TypeScript 7's implications for build tooling, and install caching. Six posts that look unrelated from the outside. They are one idea.

[Post 1](/posts/your-build-cache-is-lying-to-you) showed the failure mode: declared-input trust doesn't work at scale. [Post 2](/posts/two-phase-fingerprinting-buildxls-deep-magic) showed the fix: weak fingerprints find candidates, strong fingerprints verify them, the sandbox-derived pathset bridges the two. [Post 3](/posts/hooking-syscalls-without-a-kernel-driver) showed how to build the sandbox that produces the pathset, across three operating systems, without root. [Post 4](/posts/typescript-7-makes-the-build-harness-more-important) showed why a 10x faster compiler makes the harness more important, not less. [Post 5](/posts/node-modules-in-a-content-addressed-store) showed the unsexy problem nobody talks about: `node_modules` install caching, the CI bottleneck that persists even when everything else is fast.

The connecting thread: make correctness a property of the mechanism, not of human discipline.

Declared inputs are human discipline. Sandboxed observation is mechanism. Weak fingerprints are human discipline. Strong fingerprints from sandbox data are mechanism. Expecting developers to configure isolation flags is human discipline. Enforcing `isolatedModules` at the harness level is mechanism.

Every time you move a correctness guarantee from "someone has to remember" to "the system does this automatically," you buy reliability that compounds over the lifetime of the project. The 200th engineer to join the monorepo gets the same correctness as the first, without reading a wiki page about cache input declarations.

That's what [rage](https://github.com/kenotron-ms/rage) is for. Not a faster lage. A correct one.

<div class="breakout">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 840 290" role="img" aria-label="The Series: six posts covering cache correctness, two-phase fingerprinting, syscall sandboxing, TypeScript 7, install caching, and this meta narrative, connected by the thread of making correctness mechanical.">
  <rect width="840" height="290" fill="#F5F0E8" rx="4"/>
  <text x="420" y="28" text-anchor="middle" font-family="Georgia, serif" font-size="15" font-weight="bold" fill="#2B2421">The Series</text>
  <rect x="40" y="48" width="230" height="64" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="155" y="70" text-anchor="middle" font-family="monospace" font-size="10" fill="#A0522D">Post 1</text>
  <text x="155" y="88" text-anchor="middle" font-family="Georgia, serif" font-size="11" fill="#2B2421">Your Build Cache Is Lying</text>
  <text x="155" y="102" text-anchor="middle" font-family="Georgia, serif" font-size="9" fill="#8B7D72">the failure mode</text>
  <rect x="305" y="48" width="230" height="64" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="420" y="70" text-anchor="middle" font-family="monospace" font-size="10" fill="#A0522D">Post 2</text>
  <text x="420" y="88" text-anchor="middle" font-family="Georgia, serif" font-size="11" fill="#2B2421">Two-Phase Fingerprinting</text>
  <text x="420" y="102" text-anchor="middle" font-family="Georgia, serif" font-size="9" fill="#8B7D72">the fix</text>
  <rect x="570" y="48" width="230" height="64" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="685" y="70" text-anchor="middle" font-family="monospace" font-size="10" fill="#A0522D">Post 3</text>
  <text x="685" y="88" text-anchor="middle" font-family="Georgia, serif" font-size="11" fill="#2B2421">Hooking Syscalls</text>
  <text x="685" y="102" text-anchor="middle" font-family="Georgia, serif" font-size="9" fill="#8B7D72">the data source</text>
  <rect x="40" y="140" width="230" height="64" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="155" y="162" text-anchor="middle" font-family="monospace" font-size="10" fill="#A0522D">Post 4</text>
  <text x="155" y="180" text-anchor="middle" font-family="Georgia, serif" font-size="11" fill="#2B2421">TS7 and the Harness</text>
  <text x="155" y="194" text-anchor="middle" font-family="Georgia, serif" font-size="9" fill="#8B7D72">why it matters more</text>
  <rect x="305" y="140" width="230" height="64" rx="4" fill="#fff" stroke="#2B2421" stroke-width="1.5"/>
  <text x="420" y="162" text-anchor="middle" font-family="monospace" font-size="10" fill="#A0522D">Post 5</text>
  <text x="420" y="180" text-anchor="middle" font-family="Georgia, serif" font-size="11" fill="#2B2421">Install Caching</text>
  <text x="420" y="194" text-anchor="middle" font-family="Georgia, serif" font-size="9" fill="#8B7D72">the unsexy problem</text>
  <rect x="570" y="140" width="230" height="64" rx="4" fill="#A0522D" stroke="none"/>
  <text x="685" y="162" text-anchor="middle" font-family="monospace" font-size="10" fill="#F5F0E8">Post 6</text>
  <text x="685" y="180" text-anchor="middle" font-family="Georgia, serif" font-size="11" font-weight="bold" fill="#F5F0E8">Why I Rewrote It</text>
  <text x="685" y="194" text-anchor="middle" font-family="Georgia, serif" font-size="9" fill="#F5F0E8">the full story</text>
  <line x1="60" y1="240" x2="780" y2="240" stroke="#A0522D" stroke-width="2"/>
  <circle cx="60" cy="240" r="4" fill="#A0522D"/>
  <circle cx="780" cy="240" r="4" fill="#A0522D"/>
  <text x="420" y="268" text-anchor="middle" font-family="Georgia, serif" font-size="12" font-style="italic" fill="#A0522D">Make correctness a property of the mechanism, not of human discipline.</text>
</svg>
</div>