BLOG · #Engineering

Local-first data: snapshots, deltas, and the two kinds of state

Most sync problems get hard because two kinds of data with opposite properties are pushed through one mechanism. Separate the slow-changing catalog from the fast-changing values, and the catalog becomes a solved problem: ship a snapshot, sync deltas by version.

· 6 min read · by kaguc

Also in 中文. Part 3 of Trading app architecture.

An app that deals with a large domain holds two kinds of data, and they have nearly opposite properties:

Reference data is the catalog of things that exist — instruments, exchanges, currencies, precision rules. It is large, changes slowly, and is mostly appended to. A user expects to search it instantly, including with no network.

Live data is the current value of things — prices, positions, balances. Each item is tiny, changes constantly, and is worthless when stale.

Most difficulty in “sync” comes from pushing both through one mechanism. Split them and each becomes tractable: live data is a stream (the previous part), and reference data is a replication problem with an unusually easy shape.

Why the catalog belongs on the device

The requirement “search must be instant, always” cannot be met by a network call. Not because a network call is slow on average, but because its worst case is unbounded — and search is judged on the worst case, in an elevator, on a train.

So the device holds the catalog and the network becomes an enhancement rather than a dependency. This is what local-first means in practice: the local copy is good enough to work from, and the network’s job is to make it fresher, not to make it exist.

That inverts the usual failure story. A conventional client is broken while offline. A local-first client is merely stale while offline, and staleness in a slow-changing catalog is often unobservable. The difference in perceived quality is large and it costs you very little, because the underlying data was never fast-moving to begin with.

Snapshot plus deltas, keyed by a monotonic version

The mechanism is small:

The server keeps a monotonically increasing version for the catalog. Every change bumps it. The client stores the version it has reached and asks a single question on launch: what has changed since version n? The server returns the deltas; the client applies them and records the new high-water mark.

That question is the entire protocol, and it has properties worth naming:

  • Resumable. An interrupted sync loses nothing. You resume from the last version you fully applied — so apply the batch and update the marker in one transaction, or you will eventually skip a delta.
  • Idempotent. Re-applying a range you already have is harmless if deltas are expressed as upserts and tombstones rather than in-place mutations.
  • Cheap to verify. Two integers tell you whether a client is current.

This is why the catalog is the easy side of cache invalidation. Invalidation is hard when you must guess whether a cached thing is still valid. Here you never guess: the version answers it exactly, and the data is append-mostly, so the answer is usually “add these few rows.”

Two versions must not be conflated: the data version (how fresh the contents are) and the schema version (what shape the local store is in). App updates change the schema; server changes bump the data. Store and migrate them separately, or one upgrade will make you discard a database you only needed to add a column to.

Shipping the snapshot inside the app

You can start every user at version zero and download the catalog on first launch — a few hundred thousand rows over a mobile network, before they can search anything.

Or you ship a snapshot inside the binary and start them at the version it was built at. First launch, first search, works offline, no spinner.

The trade is install size against first-run experience, and for a catalog that is genuinely required for the product to function, the snapshot usually wins. It is a deliberate exchange of bytes users download once, in the background, from an app store built for it, against a fragile network moment at the least forgiving point in the product.

The catalog then only ever gets deltas. A user who installs a year later gets a bigger delta, which is the one cost of this design and the reason you re-cut the shipped snapshot periodically.

What must never be local-authoritative

The device can own the catalog. It must not own the ledger.

Balances, positions, and order status are the server’s truth. Cache them for display, but treat the cache as a rendering of a remote fact, with two rules: reconcile on reconnect and cold start, and never let a local value participate in a decision that has consequences. Local state answers “what do I show,” never “what is true.”

The line is drawn by a single question: if this value is wrong, is the outcome a stale screen or a wrong action? Stale screen is a caching problem. Wrong action is a correctness problem, and correctness problems live on the server.

Keep the local surface read-mostly

Everything above is easy because the replication is one-way. The server writes, the client reads. There are no conflicts, because there is only one writer.

The moment the client also writes to the same data, you inherit the genuinely hard part of distributed systems: two edits to one record, no global clock, and a need to converge. Now you need conflict resolution — last-write-wins with a version vector, CRDTs, or a server-side merge — and you need to accept that some resolution will be wrong in a way users notice.

There is usually a way to keep local writes off this path: model user-generated state (a watchlist, a note) as its own small collection with its own sync, rather than as edits to the replicated catalog. One-way replication for the big thing, deliberate two-way sync for the small thing you actually need.

Scaling this down

The whole design is forced by three properties: the catalog is large, slow-changing, and needed offline. Remove any one of them and something simpler is correct:

  • Catalog small enough to fetch in one call? Fetch it and cache in memory.
  • Changes constantly? It isn’t reference data — treat it as a stream.
  • Never needed offline? A search endpoint, with the server doing the ranking it is better at anyway.

The failure mode worth avoiding is building version-tracked delta sync for a few hundred rows that would have been a single request. Snapshot-plus-delta is what you graduate to when the catalog gets big enough that re-downloading it is the thing users complain about.

References

Previous: Order reliability · Next: The cross-platform boundary

#Engineering #Architecture #Systems