TLDR: If your data source takes more than ~500ms, don't block your UI on every read. Serve the last known value instantly, revalidate in the background. The user feels the speed. They don't feel the 9 seconds.

the setup

I've been building Apollo — my personal AI command center, a local Python dashboard that aggregates my tasks, emails, calendar, and system health into a cockpit-style interface.

One of the most important data sources is Things 3 (my task manager), which exposes no HTTP API.

The only way to read it from the dashboard is via AppleScript — macOS automation talking directly to the app.

And it's slow.

the wall

Every page load called into Things 3 and waited for the AppleScript response.

8 to 10 seconds.

EVERY. PAGE. NAVIGATION.

Click "Tasks." Wait. Click "Home." Wait again. The whole cockpit aesthetic — instant, mission-control, responsive — felt like a lie.

The dashboard runtime is pure Python stdlib, no async framework, no queue. It binds 127.0.0.1 only. So "make the call faster" had a hard ceiling.

what I tried first

The first move was a single-flight guard — if multiple page requests arrived while the Things read was in-flight, collapse them into one. No duplicate AppleScript calls. One caller waits; the rest ride along for free.

That helped with concurrent hits. It did absolutely nothing for the first load.

I was still staring at a spinner on every cold navigation. The root cause wasn't concurrency — it was the blocking read itself.

the fix that worked

Stale-while-revalidate, with a 25-second TTL.

The pattern:

  1. Cold start (no cache yet) → block on a single-flight read, masked by the boot screen.
  2. Cache fresh (under 25s) → return it instantly, no call.
  3. Cache stale but present → return the stale rows immediately, kick off one background refresh thread.

Here's the actual implementation from apollo/ranking.py:

_THINGS_CACHE = {"at": 0.0, "data": None}
_THINGS_LOCK  = threading.Lock()   # serializes cold/forced blocking reads
_REFRESH_LOCK = threading.Lock()   # single-flights the background SWR refresh
_THINGS_TTL   = 25.0

def _things_cached(force=False):
    c = _THINGS_CACHE
    if _fresh(c, force):            # under 25s → instant
        return c["data"]
    if not force and c["data"] is not None:   # stale-while-revalidate
        threading.Thread(target=_bg_refresh, daemon=True).start()
        return c["data"]            # stale rows, instantly
    with _THINGS_LOCK:              # cold or forced → blocking read
        if _fresh(c, force):        # another thread beat us
            return c["data"]
        return _do_read(c)

Two locks, not one — that's the key detail I missed on my first pass. _THINGS_LOCK serializes the rare blocking read (cold start, forced rerank). _REFRESH_LOCK uses try-acquire to make sure only ONE background refresh runs at a time — concurrent stale readers skip the refresh and serve what they have, rather than queueing behind a 9-second call.

One more thing: when I complete or cancel a task, the handler calls invalidate_things_cache() immediately. Otherwise you'd complete a task and it'd ghost back for up to 25 seconds. User actions need instant reflection. Background staleness is fine. Contradicting what the user just did is not.

why this matters

I've hit the same problem in production apps — a third-party API with brutal latency, a database query that can't be optimized further, an LLM call in the read path.

The instinct is always "make the read faster." But there's a ceiling on that.

The better question is: does this page actually need fresh data, or does it need to feel instant?

For a personal task dashboard, last-known-good is almost always good enough. My task list doesn't change mid-navigation. 25-second-old data is fine. What's NOT fine is making the cockpit feel like a 1990s dial-up page load every time I switch views.

Stale-while-revalidate is the right answer when:

  • Freshness lag is acceptable — users won't make an irreversible decision on slightly stale data
  • The read is slow — >500ms is a UX problem; >2s is a product problem
  • User mutations need instant invalidation — serve stale for ambient reads, but never contradict an action the user just took

All three were true here. And now the cockpit is actually a cockpit.

P.S. The one genuinely-cold first read — app boot — is masked by the boot overlay + a startup rerank that pre-warms the cache. By the time the boot screen fades, the first Things read is already done. The user never sees it.