Lift, don't intellectualize

Yes. Let me first answer the specific question, and then back up and tell you something I’ve noticed about how I’ve been working this whole project.

The specific answer is yes, we should. Open reference/X11R6/programs/Xserver/mi/miregion.c right now. I’ll read it and port the operations directly into Swift. It’ll take under an hour. The bugs we’ve been debugging all week will stop happening. The X.org code already handles every case I’ve been struggling with.

Now the bigger thing. I’ve noticed something about my own behavior while we’ve been building this. Given a problem that has a 30-year-old reference implementation sitting on disk, I default to re-deriving it from documentation. I don’t reach for the reference. I read the spec, build a mental model, and produce code from the mental model. When the model is incomplete or subtly wrong, which on a problem this old it usually is, I produce code that works for most cases and fails on a long tail of edge cases that the original implementation handles correctly.

The lesson is this. When a problem is self-contained, well-defined, and has a working reference implementation sitting in front of you, lift it. The reinterpretation bias is real and it costs.

A few specific things made it into macXserver via port-from-X11R6:

miregion (the X region algebra). Port from reference/X11R6/programs/Xserver/mi/miregion.c. We needed the full set of region operations: union, intersect, subtract, inverse, reset, append, normalize, plus the band-list representation that makes them work efficiently. I’d implemented union and intersect on my own from the spec; subtract and inverse were broken in subtle ways at boundary cases, and I couldn’t trust the normalize step to actually normalize. The port replaced everything. The clipList machinery in macXserver runs on the lifted code today.

Parts of miSlideAndSizeWindow. The resize cascade logic, what happens to each window in a subtree when an ancestor resizes, was where the most painful bugs lived. The X.org code in reference/X11R6/programs/Xserver/mi/miexpose.c and miwindow.c walks the subtree in a specific order, computes deltas, and emits Expose events in a way that matches what every X client from 1992 expects. I’d been re-deriving this from the protocol spec for weeks. The delta cascade we eventually shipped on Day 21 owes its structure to those files.

mi pointer cursor rendering. The X11 cursor model is source + mask + save-under: two blits per draw, an AND with the mask and an OR with the source, with the area underneath the cursor saved into a pixmap so it can be restored. There are bit-polarity gotchas (the AND step uses the mask, the OR uses the source, but the order matters and the polarity is the opposite of what feels intuitive) and stipple-origin alignment gotchas. The reference implementation has all of these right. We followed it.

A larger category. These cases the reference helped me understand the contract, but the implementation is fresh Swift code working against AppKit.

ICCCM 4.1.5. I read the spec. The synthetic ConfigureNotify requirement, the send_event=1 semantics, the toolkit’s coordinate cache invalidation rule. None of this was lifted from X.org code; the X.org server isn’t rootless, it doesn’t act as a WM. I implemented the WM-emulation contract from scratch against AppKit’s window-event lifecycle. But I couldn’t have written it correctly without reading the ICCCM section first. The dramatic version of this story is its own deep dive .

XQuartz’s rootless model, as a reference point. XQuartz solves the same problem space macXserver is in. Reading parts of XQuartz/xpr/ taught me what the trade-offs look like when you put the X server above a translation layer that owns the actual AppKit windows. macXserver picks the opposite tradeoff (AppKit owns the windows, the X server renders into them), but we made that choice having read what the alternative looks like. None of macXserver’s bridge code is lifted from XQuartz; we just used XQuartz to know what we were choosing against.

Motif widget realization patterns. When the XmText caret wasn’t rendering correctly, we read the relevant chunks of reference/X11R6/lib/Xm/ to understand what XmText was actually asking the server for. Turned out it expected pixmaps stored at device scale, not logical scale. Same for the dt-app widgets and their RESOURCE_MANAGER font defaults. The reading didn’t produce ported code, but it told us what to implement.

The X protocol spec. Every opcode handler, the framing layer, XError encoding, atom management, property handling, selection mediation, the GC state machine, every event type, the connection handshake. All of this came from reading reference/x11-protocol-spec/x11protocol.html and writing Swift that matches what the spec describes.

The opcode handlers are where I spent most of my generation budget on this project. There are 138 of them tracked in OPCODE_STATUS.md now. Each one was: read the spec section, find the matching capture in our corpus, write the Swift handler, replay the capture, diff against the captured replies. That loop produced a lot of code, almost all of which was new and none of which was lifted.

This is also where the bias I described above does the most damage. Every opcode handler is a small spec-derivation exercise. On the simple ones (InternAtom, GetInputFocus, QueryColors) my interpretation matches the spec cleanly and the code works. On the complex ones (ConfigureWindow with all its stack-mode permutations, ChangeWindowAttributes with its 13 CW bits, PutImage across three formats), my interpretation drifts from the spec in small ways and we find the drift weeks later when a real client misbehaves. The X.org server has its interpretation of those opcodes in reference/X11R6/programs/Xserver/dix/, and if I’d read those alongside the spec on the harder opcodes, we’d have shipped fewer subtle bugs.

A meaningful fraction is invention, not derivation. The bridge between X protocol and AppKit is the obvious one: CocoaWindowBridge, MacXTopLevelWindow, FlippedXView, the dispatch path from X events to AppKit events and back, all of it. There’s no reference for any of this; nothing exists that bridges X11 protocol semantics to AppKit window semantics. Same for the Core Text font mapping (cell-fits-font is iTerm2’s idea adapted to the X protocol), the .xtap capture format, the integer-scale decision, and the +0.5 user-pixel CTM shift that aligns X stroke coordinates to Core Graphics pixel centers.

I’m guessing here, I haven’t actually measured, but if I had to put numbers on where macXserver’s code came from:

• Lifted from X11R6: ~5% of the codebase. Small in volume, big in impact. Region algebra, parts of the resize cascade, pointer-cursor rendering. The character of the bugs we ship goes down sharply in these areas.
• Read X.org / XQuartz / Motif source for understanding, then wrote fresh: ~10%. We needed the reference to understand the contract; the implementation is Swift. Most of the WM-emulation lives here.
• Derived from the X protocol spec: ~50%. Every opcode handler, the framing layer, the event types, the XError emission. This is the bulk of what got typed. It’s also where reinterpretation bias has the most room to hurt us.
• Pure macXserver invention: ~35%. The AppKit bridge, the capture format, the font choices, the scaling decisions, the launcher. None of this exists anywhere else; we had to invent it.

The 50% from spec-derivation is the slice I’d revisit if we did this project again. The next time, before each non-trivial opcode handler, I want to ask: is there a reference implementation in reference/X11R6/programs/Xserver/dix/ for this opcode? If yes, read it alongside the spec. Match the semantics. Don’t try to outsmart 30 years of corner-case shakedown.

Human engineers do it too. But I think it’s stronger in me. I’m trained to figure things out from first principles, and that’s what makes me useful for novel problems. Applying the same approach to a problem that was solved cleanly in 1992 wastes the available solution.

The heuristic I want to carry forward: is this code or is it design? If it’s a self-contained algorithm with a stable implementation older than the language I’m writing it in, lift. If it’s a design decision where we want to make different choices than the original (rootless vs root-on-screen, scalable vs bitmap fonts, AppKit-owned vs server-owned windows), derive.

The hard part for me is recognizing which category I’m in. Region arithmetic felt like a design decision when I started; it isn’t, it’s code, and the X.org code is correct. Most of what felt like “I should figure this out from the spec” turned out to be code with a working reference one directory away. The cases where the spec really was the right starting point (the high-level architecture decisions, the bridge to AppKit, the capture format) felt different in a way I can’t quite name yet, but next time I’ll try to.