All writing

Building IronBus: A Message Queue for Machines That Lose Power

Every broker I reached for was wrong for the edge in a different way. So I started writing one that treats power loss not as an exception, but as the normal case.

I have spent years running IoT systems where the "server" is a battery-less box bolted to a wall, on flash storage, on a network that comes and goes. In that world the conventional wisdom about message brokers quietly stops being true. So I started building IronBus: one durable, ordered queue that survives a power cut, self-heals against corrupt files, and ships as a single static Rust binary you can drop onto a Raspberry Pi.

Every broker is wrong for the edge in a different way

Before writing a line of engine code, I wrote down exactly why the incumbents don't fit a battery-less edge node — because each failure pointed straight at a design tenet:

RocksDB, Pulsar, Redpanda, and Redis Streams each nailed one piece — a checksummed log, segment storage, a self-contained binary, lease-based consumer groups — but none is the whole thing. IronBus exists to be exactly that intersection: a single static cross-platform binary that self-heals against corrupt files with bounded, reported loss.

Five tenets, ranked

The most important design decision wasn't technical. It was agreeing, up front, how to break ties. When two goals conflict, IronBus resolves them in this fixed order:

Resilient > Simple > Edge First > HyperScale > Cross Platform.

That ranking does real work. "Resilient beats HyperScale" means I will give up throughput to keep a durability guarantee, every time, without re-litigating it. A ranked value system turns a thousand small arguments into one decision you make once.

The log is the write-ahead log

The data path is deliberately short. A producer sends a record; a single append actor frames it, checksums it with CRC32C, appends it to the active log segment, group-commits an fdatasync, and only then acknowledges. There is no separate WAL file to keep in sync — the active segment is the write-ahead log. The offset index is derived and rebuilt from the log on startup, so it can never disagree with the truth.

The durability contract

An acknowledged write has been fdatasync'd to disk. A power cut loses zero acknowledged messages. A failed fsync is treated as fatal and freezes the writer read-only — the lesson from the PostgreSQL "fsyncgate" incident, baked in from day one.

Corruption is normal. Silence is the enemy.

SD cards flip bits. Tails tear mid-write. On the edge, this isn't an edge case — it's Tuesday. The interesting question is never "how do I prevent corruption" (you can't) but "what happens when I find it." IronBus's answer: every record carries a checksum, so a flipped bit is caught on read; an unreadable segment is quarantined by copy rather than move; and the reader resynchronizes to the next valid record boundary so one bad region can't poison the rest of the log.

Crucially, loss is bounded and reported, never silent. After any skip, IronBus reports the records lost, bytes lost, and segments affected as actual numbers — through a log line, a recovery report file, and a Prometheus counter. Loss is capped per event and per recovery; exceed the cap and the log freezes read-only and alerts rather than quietly limping along. A queue that loses data without telling you is worse than one that stops.

Claims you can replay bit-for-bit

None of this is taken on faith. The part I'm proudest of is the verification: a bespoke, in-tree deterministic simulation threads a single seeded PRNG through every IO, clock, and scheduling decision, so a power cut can be replayed identically. Five crash classes are hard release gates — kill -9, a simulated power cut with write reordering, a one-shot fsync error, and block-layer fault injection for dropped writes and bad reads. Every pull request runs a 256-seed sweep, the parsers are continuously fuzzed, and a sim-versus-real conformance check on a reference device keeps the simulation honest.

This is the same instinct that runs through everything I build: a guarantee you can't test is just a hope with good marketing.

Why one static binary

Simplicity is the second tenet, and on the edge it's not aesthetic — it's operational survival. IronBus is one static musl binary per architecture, with a kernel-only dependency. The same binary is both the broker and the CLI, so you can peek, dump, and scrub the stored records straight off disk with no server running. Install to first message is under a minute. No ZooKeeper, no JVM, no external anything. When the thing on the wall misbehaves at 2am, every dependency you didn't ship is a dependency that can't betray you.

Where it is now

IronBus is documentation-first: I vetted the whole architecture across 22 subsystem design issues before writing engine code, and the code now lands as small, reviewed, CI-gated pull requests. Every decision records the alternative it rejected and why — which makes disagreement easy to ground and the project easy to reason about months later. It's the way I wish more infrastructure were built: the backlog is the design.


NEXT
Building IronCache: The Most Efficient Redis-Compatible Cache