ESP32 + CM4/CM5: Our Hardware Architecture for Self-Hosted AIoT

In The Lean Stack I wrote about the software side of going back to fundamentals, and in the Rust article about the language we run on devices. This post answers the question underneath both: what hardware do you actually build on?

After several years of research and real deployments, our answer has solidified into two components: ESP32 at the edge, Raspberry Pi CM4/CM5 at the core. That's the entire architecture — for a home, a factory, an office building, a resort, a hotel, a school.

Where this came from: a balcony full of hot sand

This architecture was not designed on a whiteboard. It was forced into existence by the balcony sand battery experiment.

A sand battery is, at its heart, a data problem: temperatures at multiple depths, heater duty cycles, energy in versus energy out, logged reliably for weeks. My first need was embarrassingly simple — a lean, reliable channel of data collection that would not die overnight or silently drop readings.

Then the real world kept adding requirements. Commercial deployments meant interfacing with whatever was already installed: Zigbee sensors, Matter and Wi-Fi devices, Bluetooth meters on the residential side; SCADA systems and PLCs speaking Modbus on the industrial side. Every site is a museum of protocols, and none of them care about your preferred stack.

Year after year, the same shape kept emerging: cheap, low-power nodes collecting and controlling at the edge; one trustworthy local machine collecting, storing, and deciding at the center. By early 2024 that shape had solidified into a software and firmware architecture layer we called second.brains — the repo is public at github.com/AlphaBitsCode/second.brains, and it predates the current local-AI wave (pre-March 2024). We were building for self-hosted intelligence before it was cool. 😄

The edge: ESP32 everywhere

The ESP32 is, in my assessment, the best value proposition in the entire hardware industry.

For a few dollars you get Wi-Fi, Bluetooth, enough GPIO to speak to almost any sensor or relay, and power consumption low enough to run battery-powered nodes that sleep most of their lives. That covers the overwhelming majority of what a site actually needs:

Data collection and transmission — temperatures, power, water flow, occupancy, air quality — read, timestamp, transmit, sleep.
Basic control and automation at the local node — a factory line interlock, a farm irrigation valve, a pump schedule. The ESP32 is smart enough to make these decisions locally, so a network outage never stops the physical process.
Protocol bridging — an ESP32 with an RS-485 transceiver talks Modbus to a 20-year-old PLC on one side and MQTT to our core on the other. Legacy hardware gets a second life instead of a forklift replacement.

The discipline is knowing what not to put there: no databases at the edge, no business rules, no cloud credentials. Edge nodes stay simple, cheap, and replaceable — that is exactly what makes them reliable.

The core: CM4/CM5 as the local brain

Every site gets one central platform: a Raspberry Pi Compute Module 4 or 5 on an industrial carrier board — proper power input, real Ethernet, an M.2 SSD, a case that survives a dusty electrical cabinet.

This is where everything the edge collects lands, and — this is the important part — where it stays:

Centralized, reliable, secured storage. Every sensor reading, every event, every control decision, in a database on hardware you own, in a building you can walk into.
Nothing goes out to the cloud — unless you explicitly want an encrypted off-site backup. The cloud becomes an option, not a dependency.
Local dashboards and local AI. The CM5 in particular has enough compute to run the site's dashboards, automation logic, anomaly detection, and increasingly capable small models — right where the data lives.

A resort's occupancy patterns, a factory's production counts, a school's energy profile — this is operationally sensitive data. The honest question I ask every operator: why should any of it live on someone else's computer? This is the self-hosting governance argument made physical.

Why this two-tier split keeps winning

The division of labor is clean and it maps onto economics:

An edge node failing costs a few dollars and a five-minute swap. So we deploy many, cheaply, everywhere.
The core is the single component worth engineering seriously — redundant storage, watchdogs, monitored — because there is exactly one per site.
Both tiers are commodity hardware with a decade of community knowledge behind them. No vendor lock-in, no proprietary gateway that disappears when a startup pivots.

And it scales in both directions from the same architecture: a home runs one CM4 and a handful of nodes; a factory runs one CM5 and two hundred. Home, office, building, farm, resort, hotel, school — the topology never changes, only the node count.

Hardware is frozen, software keeps compounding

Here is the strategic payoff. By standardizing on total control of a boring, stable hardware layer, all of our innovation moves into software and firmware — the layer where AI coding agents give us compounding leverage.

The ESP32 firmware improves continuously. The core's storage, automation, and AI capabilities improve continuously. The hardware underneath doesn't need to change for years — and when the CM6 eventually arrives, it will slot into the same carrier boards and the same architecture.

Buy the hardware once. Own the data forever. Let the software compound. That's the whole strategy. 🔧