The SQLite Rewrite In Rust

TL;DR

Limbo is a Rust reimplementation of SQLite intended to preserve SQLite-level compatibility while adding memory safety and a modern architecture.

Briefing Cornell Notes

Briefing

A company-backed Rust rewrite of SQLite—called Limbo—aims to preserve SQLite’s reliability while switching to memory-safe code and a modern architecture built for deterministic testing and async I/O. The pitch is straightforward but ambitious: reimplement SQLite “from scratch” at the language and file-format level, keeping compatibility while raising confidence through deterministic simulation testing rather than relying on SQLite’s (largely opaque) proprietary test suite.

The motivation starts with how SQLite is governed and tested. SQLite is maintained by a very small group, and contributions are effectively constrained; even understanding how it’s tested is difficult because key parts of the testing infrastructure are proprietary or hidden. That limits the ability to confidently evolve behavior or verify large changes. In contrast, the project behind Limbo—LibSQL—grew as an open contribution effort and has added features such as native replication and Vector search, becoming the engine behind Turo’s platform.

Limbo’s development began as a personal experiment by Pekka, the CTO, after earlier attempts to build in Zig. The experiment, initially named “Limbo,” attracted organic growth to roughly a thousand GitHub stars and more than 30 contributors, leading to an official Turo-backed effort. The stated goal is a fully compatible reimplementation with “the same or higher reliability” than SQLite, but with full memory safety and a new architecture.

A central differentiator is testing. Instead of conventional nondeterministic runs that depend on timing and OS behavior, Limbo uses deterministic simulation testing—made possible by adding DST facilities into the database core and partnering with Antithesis, a company providing a deterministic hypervisor and system-level simulation framework. The approach targets hard-to-reproduce failures by controlling event ordering and simulating rare conditions, including partial writes and I/O edge cases. Antithesis is credited with helping find issues in iouring under partial rights—problems the team says their own DST framework wouldn’t catch because the real I/O loop is replaced by a simulated one.

Limbo also pushes toward async performance. SQLite’s public interface is synchronous, so async behavior typically requires helper threads, which can be inefficient when queries are fast but still need to scale under real workloads. Limbo is designed to be fully asynchronous from the ground up, using io_uring on Linux and targeting WebAssembly builds with a VFS implementation compatible with tools like Drizzle. Early benchmarks cited in the discussion show Limbo matching or beating SQLite on specific queries after tuning, but the emphasis lands on what matters operationally: tail latency under concurrency, not single-query speed.

The project is positioned not as a replacement for LibSQL but as SQLite “plus more.” Limbo drops some legacy or less relevant SQLite build features (like amalgamation and certain tuning defaults) while aiming to keep compatibility with SQLite’s file format and query language. If it succeeds, the codebase becomes LibSQL’s SQLite core—turning a liability (rewriting risk) into an asset (control, safety, and robustness) for a distributed, serverless-style database used at scale.

Cornell Notes

Limbo is a Rust-based reimplementation of SQLite aimed at matching SQLite’s reliability while adding full memory safety and a modern architecture. The project targets compatibility at the language and file-format level, but it also prioritizes async I/O and WebAssembly readiness. A major reliability strategy is deterministic simulation testing: Limbo integrates DST facilities and partners with Antithesis to run system-level deterministic simulations that can reproduce rare OS and I/O failures. The effort began as an organic personal experiment by Pekka and became an official Turo project after rapid community traction. If successful, Limbo is intended to become the SQLite core inside LibSQL, effectively making “SQLite plus more” rather than a separate competing database.

Why does the project treat SQLite’s governance and testing model as a constraint on change?

SQLite is described as having only a small number of maintainers, with contributions effectively blocked from most outsiders. On top of that, the testing infrastructure is portrayed as proprietary or hidden, making it hard to understand how correctness and reliability are validated. That combination limits confidence when attempting large, compatibility-sensitive changes—especially when the goal is to evolve behavior while preserving SQLite’s reputation for reliability.

What does “deterministic simulation testing” change compared with typical test runs?

Deterministic simulation testing controls timing and event ordering so repeated runs produce the same outcomes. That matters because database behavior often depends on file reads, OS scheduling, and other nondeterministic factors; running the same scenario twice can yield different interleavings. With deterministic simulation, rare failures become reproducible, letting teams debug and verify fixes reliably instead of chasing “works on my machine” timing issues.

How does Antithesis fit into Limbo’s reliability plan?

Antithesis provides a deterministic hypervisor and system-level deterministic simulation framework. In the discussion, Antithesis is credited with helping find an issue in io_uring under partial rights—an edge case where a write may succeed only partially (e.g., an attempted 50-byte write results in only 30 bytes written). The team claims its own DST framework wouldn’t have caught the issue because the real io_uring is replaced by a simulated I/O loop during testing, making certain rare conditions hard to trigger without Antithesis’s tooling.

Why is async I/O a key design goal, and what problem does it address?

SQLite’s interface is synchronous, so async behavior usually relies on helper threads. That can be acceptable when queries are slow (network round trips dominate), but it becomes inefficient when many queries are fast and the system needs to scale. Limbo is designed to be fully asynchronous from the ground up, using io_uring on Linux and supporting WebAssembly builds with a VFS implementation, aiming to handle both local and remote data patterns more efficiently.

What compatibility and feature stance does Limbo take toward SQLite?

Limbo is framed as a one-to-one reimplementation of SQLite at the language and file-format level, targeting the same or higher reliability. At the same time, it’s not presented as “SQLite only”: it’s also positioned as SQLite plus additional modern capabilities (async performance focus, wasm support, and VFS integration). The discussion also notes that Limbo intentionally drops some older SQLite build conveniences and less important features for modern environments, while keeping compatibility where it matters.

What performance claims are treated as less decisive than workload realism?

The discussion cites a benchmark where a specific query is faster on Limbo than on SQLite after tuning. But it warns that single-query microbenchmarks aren’t the main risk area; real systems often fail in the tail (e.g., 99th percentile latency) when many concurrent operations contend for resources. The practical goal is to understand behavior under concurrency and stress, not just best-case query timing.

Review Questions

What specific limitations of SQLite’s contribution and testing model motivate a Rust reimplementation effort?
How does deterministic simulation testing help reproduce failures that are otherwise hard to trigger in normal database testing?
Which design choices in Limbo target async performance and WebAssembly compatibility, and why do those matter for distributed workloads?

Key Points

1
Limbo is a Rust reimplementation of SQLite intended to preserve SQLite-level compatibility while adding memory safety and a modern architecture.
2
The project’s reliability strategy centers on deterministic simulation testing (DST), aiming to make rare failures reproducible by controlling event ordering.
3
Limbo integrates DST facilities and partners with Antithesis, whose deterministic hypervisor helped surface an io_uring edge case involving partial rights.
4
The effort began as Pekka’s personal “Limbo” experiment, then became an official Turo project after strong organic community traction.
5
Limbo is designed for fully asynchronous I/O from the ground up, avoiding helper-thread overhead that often accompanies async use with SQLite’s synchronous interface.
6
WebAssembly support is treated as a first-class goal, including a VFS implementation compatible with tools like Drizzle.
7
Performance comparisons should focus on tail latency under concurrency rather than single-query benchmark wins.

Highlights

Limbo’s core reliability bet is deterministic simulation testing: repeated runs should produce identical outcomes even when OS and I/O timing would normally vary.

Antithesis is credited with finding an io_uring issue under partial rights—an edge case described as partial writes that are difficult to test automatically.

Limbo targets full async behavior and wasm readiness, including a VFS that works with Drizzle without changes.

The project frames itself as “SQLite plus more”: compatibility at the file-format and language level, with modern architecture and async focus.

Topics

Mentioned

Pekka
DST
VFS
io_uring
MIT