When to adopt the heavy machinery: thresholds, not opinions

Also in 中文. The decision rules for Trading app architecture.

Every article in this series ends with some version of “adopt this when a measurement demands it.” That advice is worthless as stated. It names no metric, no threshold, and no device, which means in practice the decision gets made by whoever argues most confidently in the meeting.

So here are the numbers. Some are arithmetic and hold everywhere. Some are perception research. Some are conventions I’ll label as conventions. Calibrate them to your product — but calibrate numbers, not vibes.

Before that, the correction that matters more than any threshold.

Not everything waits for a threshold

The series implies one decision rule for everything. That is wrong, and it’s the more dangerous half of the mistake, because “we’ll add it when we need it” applied to the wrong category produces a rewrite or a loss.

Sort every candidate into one of two boxes.

Box A — cost of waiting is bounded. You can add it later at roughly the cost of adding it now. Performance work is almost entirely here: an isolate, a binary codec, native ingest, custom painting, feature packages. If you skip it and you were wrong, the symptom is slowness, which is visible, measurable, and fixable next sprint.

Box B — cost of waiting is unbounded. Retrofitting means touching every call site, or the damage has already happened silently by the time you notice. These get built on day one at any scale, and there is no threshold discussion:

• Idempotency keys on irreversible actions, persisted before the first send. Retrofit means auditing every mutation path, and until you do, duplicates are already reaching production.
• Server-side authorization for every request. A client-trusting endpoint is a breach that exists from the day it ships.
• Sequence numbers on any stream where a gap corrupts state. Without them you cannot even detect the corruption retroactively.
• Reconnect and resubscribe. Bolting this on later touches every subscription site, and its absence looks exactly like a working app.
• Decimal, not binary floating point, for money. 0.1 + 0.2 != 0.3 is a rounding error in a ledger. Migrating a persisted schema off floats is brutal.
• An append-only event log for financial actions. You cannot reconstruct history you never recorded.

The test for Box B: if this is missing, is the failure silent, or does it corrupt data, or does it require touching every call site to add? Any yes means day one.

Everything below is about Box A only. That distinction is the actual answer to “when do I use the heavy stuff.”

First, define the target

A threshold needs three things to be meaningful, and most teams state none of them:

A metric with a percentile. Averages hide exactly the problem you care about. One frame in twenty taking 40 ms is invisible in a mean and clearly visible to a human. Use p95 and p99.

A reference device. “Fast on my phone” is not data. Pick a specific mid-range device roughly three years old and make it the gate. A flagship will pass tests that a large share of your users would fail.

A realistic load. Production message rate, production catalog size, and a session at least 30 minutes long — leaks and cache growth are invisible in a two-minute test.

Written out, a usable target looks like: p95 UI frame under 16 ms, and p99 under 24 ms, with 200 visible symbols at the production feed rate, on the reference device, after 30 minutes.

Now a threshold can be violated, which is what makes it a decision rule instead of an opinion.

The frame budget, which is arithmetic

At 60 Hz you have 16.67 ms per frame for everything; at 120 Hz, 8.33 ms. Two threads must both fit: the UI thread (your Dart, build, layout) and the raster thread (painting into pixels).

Budget roughly half the frame for UI-thread work — so about 8 ms at 60 Hz. The rest is layout, paint, raster, and headroom for the spikes that any real system has. If your Dart work routinely exceeds 8 ms per frame, you are one hiccup from a dropped frame, and dropped frames cluster exactly when the app is busiest.

This is why the numbers below are expressed in milliseconds per frame. It’s the only budget that exists.

Thresholds, element by element

Move decoding to an isolate when parsing occupies more than 2–3 ms per frame at p95, sustained, in the DevTools timeline.

Below that it’s noise. Above it you’re spending a third of your UI budget on parsing. But apply the cheaper fixes first (see the ladder below) — and note that messages sent between isolates are copied, so a chatty isolate can be slower than no isolate at all. Send batches, use TransferableTypedData to move bytes without copying, or use Isolate.run, which transfers its result back rather than copying it.

Switch to a binary protocol when any of: JSON parsing still exceeds ~2 ms per frame after conflation; sustained payload exceeds ~100 KB/s on mobile data; or message rate exceeds roughly 1,000/s.

Under those, JSON’s cost is real but not your bottleneck, and it buys you debuggability that a binary format takes away.

Go native (FFI) when Dart cannot meet the budget after isolate and conflation — realistically above ~10,000 messages/second, or for numeric work over large arrays such as computing indicators across 100k+ bars where you need sub-millisecond determinism.

An honest calibration: at a few thousand messages a second, Dart is fine, and the conflation pattern means your render cost is already independent of feed rate. Most products that reach for native here have a conflation problem, not a language problem.

A counter-threshold applies: if nobody on the team can maintain C++ on two platforms, the answer is no regardless of the measurement. An unmaintainable optimization is a liability, and this is the one place where team capability legitimately overrides a number.

Ship a local database when the catalog exceeds roughly 5,000–10,000 rows or ~1 MB, or whenever search must work offline.

There’s a perception argument that usually settles it earlier: interaction feels instantaneous under about 100 ms (Nielsen’s classic thresholds: 0.1 s instant, 1 s uninterrupted flow, 10 s attention lost). Mobile network p95 round-trip rarely fits in 100 ms. So if search-as-you-type must feel instant, local is not an optimization — it’s the only design that meets the requirement.

Custom-paint your charts when a charting package can’t express what you need — synchronized indicator panes, drawing tools, precise crosshair semantics — or above roughly 2,000 rendered elements, or when the series updates at frame rate. Below that, use the library.

Extract feature packages when the team on one codebase reaches about 4–5 developers, or when incremental builds exceed ~60 seconds, or when the same area produces repeated merge conflicts (say 3+ per month), or when a layer needs testing in isolation.

These are conventions, not physics. The underlying variable is coordination cost, and it rises with people, not with lines of code. Conway’s prediction gives the sanity check: modules should number about what your teams number.

Adopt a shared cross-platform core when you have two or more independent client codebases and a body of domain rules where divergence would be a defect. If one team ships both platforms from one Flutter codebase, the sharing already happened; a separate shared core solves a problem you don’t have.

Add-to-app is not a threshold at all — it’s a situation. You adopt it when an existing native app must keep shipping while you migrate. If there’s nothing to embed into, the question is void.

Try the cheap fixes first, in order

Most threshold violations should never reach an architectural change. Work down this ladder and stop at the first rung that fixes it:

1. Do less work. Smaller payloads, fewer fields, LIMIT on queries, pagination, fewer visible items. Usually the biggest win and always the cheapest.
2. Do it less often. Conflate, debounce, cache, memoize. Frame coalescing alone converts many “we need native” situations into non-problems.
3. Do it in a smaller scope. Narrow the rebuild, split the repaint layer, invalidate less.
4. Do it elsewhere. Move to an isolate or to the server.
5. Do it in a faster language. Native, FFI.

Rung 5 is where most of this series’ “heavy machinery” lives, and reaching it before trying 1 through 4 is the most common way teams end up with a complicated architecture and the same performance.

When not to adopt, even past the threshold

Four counter-signals, each of which should stop you:

You can’t operate it. Certificate pinning without a rehearsed rotation and a kill switch is a self-inflicted outage waiting for a cert to expire. Native code without a maintainer is a bug you can’t fix. Capability to run a thing is part of its cost.

The trigger was a spike, not a level. One bad day at market open is not a threshold crossing. Require the violation to be sustained — over a week of real usage, not one profiling session.

A cheaper rung is untried. If you haven’t done 1 through 4 above, you don’t yet know whether you have a problem or a workload.

It’s a one-way door. Reversible decisions deserve fast, cheap deliberation; hard-to-reverse ones deserve more. A binary protocol you can revert in an afternoon is a two-way door. A persisted schema, a public API contract, or a native module three teams now depend on is not — those deserve the slow version of this process.

The procedure

1. Write the target before you need it: metric, percentile, reference device, load.
2. Measure under realistic conditions. Not a synthetic benchmark, not a flagship phone.
3. If violated, walk the ladder from rung 1.
4. If the ladder is exhausted and the violation is sustained, adopt — and check the four counter-signals first.
5. Record the decision: what triggered it, what you measured, what you’d need to see to reverse it. Six months later this is the difference between an architecture and an accretion.

Step 5 matters more than it looks. Most bad architecture isn’t a bad decision; it’s a decision nobody can re-examine because the reasoning was never written down.

So is the MVP the whole story?

No — and it shouldn’t read as the ceiling. The reference MVP is the correct starting point for its constraints, and the upgrade table there names the seams. This article supplies the numbers that fire each upgrade. Together they describe a path, not a destination.

Roughly, four tiers, so you can locate yourself:

Two things are worth noticing about that table. Every tier is a superset of the previous one, which is what makes the path walkable — nothing is thrown away, seams are filled in. And Box B from the top of this article applies at every tier, including tier 1. A one-developer app that places real orders needs idempotency keys and server-side authorization exactly as much as a tier-4 system does.

That’s the whole judgment framework: correctness has no threshold; performance does; and the threshold is a number on a reference device, not an opinion.

References

API and renderer claims checked against Flutter 3.47.0 / Dart 3.13 (August 2026).

• J. Nielsen, “Response Times: The 3 Important Limits”, 1993 — 0.1 s, 1 s, 10 s.
• J. Dean, L. A. Barroso, “The Tail at Scale”, CACM 56(2), 2013 — why percentiles, not averages, describe what a user experiences.
• J. Bezos, Amazon 2015 Letter to Shareholders, 2016 — Type 1 (one-way door) vs Type 2 (two-way door) decisions.
• M. E. Conway, “How Do Committees Invent?”, Datamation, 1968 — system structure mirrors the communication structure of the organization that built it.
• D. Goldberg, “What Every Computer Scientist Should Know About Floating-Point Arithmetic”, ACM Computing Surveys 23(1), 1991 — why binary floating point cannot represent decimal money.
• Dart docs, Isolates — messages are copied between isolates; Isolate.run transfers its result instead. See also TransferableTypedData.
• Flutter docs, Use the Performance view — UI and raster timelines, frame budget, and the rebuild counter.

Back to the overview · The concrete starting point: reference MVP