Mobile Release Health And Observability

Backend observability rests on an assumption so fundamental it is rarely stated: you can get to the running system. Read its logs, attach a profiler, raise the log level, reproduce the problem in an environment you control, and when all else fails, restart it.

None of that is available on mobile. The running system is on a device belonging to a member of the public, on a hardware and operating system combination you have never tested, in a network condition you cannot simulate, running a version you shipped eight months ago. You cannot attach anything to it or ask it a question. You get whatever it told you before it failed, and if it failed badly enough, not even that.

Mobile observability is therefore not about collecting more. It is about deciding in advance, as you write the code, what evidence you will need when something goes wrong — because that is the only moment at which you can arrange to have it. Everything else is archaeology on whatever you happened to record.

And because you cannot roll back, the signals have a second job: deciding whether to keep rolling out a release or stop it, in a window measured in hours that nobody will be awake for. They must be reliable enough to act on automatically, a much higher bar than being interesting enough to look at.

The four signals that matter, and what they actually mean

There is a long list of things you can measure on a mobile client, and a much shorter one that should gate a release.

Crash-free session rate and crash-free user rate. These answer different questions and you need both. Session rate tells you how often the application fails; user rate tells you how many people experienced a failure. A crash on a rarely used screen shows up weakly in session rate and strongly in user rate; a crash in a tight loop does the reverse. When the two diverge, the divergence is the information.

Application-not-responding events, or whatever your platform calls unresponsiveness. A frozen application is often worse for the user than a crashed one — a crash at least restarts. These are typically main-thread work: synchronous input or output, a lock held across a network call, an over-eager query on startup. They are underinstrumented relative to their impact, because they do not produce the satisfying artefact a crash does.

Cold start time, from launch to the point at which the user can do something. The most visible performance property of your application, and the one that degrades most insidiously, because each release adds a little startup work and none is individually responsible. Measure it as a distribution against a stated budget, segmented by device class — the aggregate is dominated by whoever has the newest hardware.

Frame timing on the paths that matter. Not average frame rate, which hides everything, but the proportion of frames that missed their deadline and the severity of the misses, on your two or three most important scrolling surfaces. Users perceive the stutter, not the average.

Around these sit the signals that catch failures that are not crashes: error rates on critical network calls, completion rates for core journeys, and one business metric that would notice a broken funnel. A release that silently fails to load content is a bad release, and crash-based monitoring will say it is fine.

Every signal needs the same five dimensions

A number without dimensions is not actionable. Attach these at collection time, because you cannot add them retrospectively.

App version. The most important one. Nearly every investigation begins with "which versions" and ends quickly if you cannot answer. It matters more than people expect because every version you have shipped is still running, so any aggregate is a weighted average across your release history.

Operating system version. Platform releases change behaviour, and a crash confined to one OS version is a different problem from one spread across all of them.

Device class. Not just model — a grouping by capability, so you can see whether something only affects older or lower-memory hardware. Memory-pressure terminations are a device-class story.

Flag state. Which flags were active in the session. Without it, a crash caused by a feature revealed to ten percent of users looks like a crash affecting all users at a low rate, and you will spend a day on the wrong hypothesis. If you use flags seriously, this is not optional.

Locale and region. Catches defects from text expansion, date and number formatting, right-to-left layout and region-specific backend behaviour — easy to miss in testing, obvious the moment the dimension is present.

Add all five centrally, once, so every event carries them by construction rather than by the diligence of whoever wrote the call site.

Regression detection is harder than it looks

Here is the trap that catches almost everyone the first time they wire release health into an automated decision. You release to a small percentage. The crash-free rate looks different from the previous version's. You conclude something about the release, and you are probably wrong, because the population that received the new version is not the population running the old one.

Early updaters skew towards newer devices, better connectivity and greater engagement, and the rollout mechanism may itself select on properties correlated with device or region. The new version's numbers therefore come from a healthier population, which usually makes a release look slightly better than it is — so a genuine regression can hide inside a favourable population shift for exactly the window during which you could have stopped it.

Corrections that work, in rough order of value:

Compare within device class and OS version, not in aggregate. A regression confined to one segment is invisible in the total and obvious in the segment. This correction alone catches most of what the aggregate hides.

Compare against the previous version at the same point in its own rollout, not against its steady state. A version's early hours differ from its mature numbers for reasons unrelated to quality: the first sessions after an update include migration work and cold caches.

Set a minimum volume before any automated decision fires. Small samples produce dramatic percentage movements, and a rule that fires on the first twenty sessions gets switched off by the second week.

Watch for new crash signatures, not just the rate. A new signature in a new version is a strong, early, low-volume signal that survives all the population problems above, and often the first honest indication that something is wrong.

Wiring signals to the rollout decision

The point of all this is a decision, made faster than a human release manager reliably will.

Write the criteria into the release checklist before submission. Three or four comparative thresholds, the segments they are evaluated within, the minimum volume, and the observation window per rollout step. Criteria decided while watching a graph are decided in favour of continuing, every time.

Make the default action stop, not continue. A rollout that requires an affirmative decision to expand, and halts if signals are unavailable, fails in the safe direction. One that expands unless someone intervenes fails in the expensive one.

Automate the halt where the platform allows it, and alert a rota rather than a person. Capabilities for programmatic rollout control differ and change, so check current vendor documentation. Where automatic halting is unavailable, automate the alarm and make the manual step one action rather than an investigation. A criterion depending on one individual noticing works on weekdays, in one timezone, when that person is not on leave.

Pair the halt with the kill switch. Halting a rollout stops new users receiving the build and does nothing for those who already have it. The kill switch protects them: halt distribution, then disable the offending path via config.

Record the outcome of every halt — real, population artefact, or threshold set too tight. An unreviewed halt drifts towards being ignored, and an ignored halt is worse than none.

One consequence is worth reporting upwards. Time to restore splits in two on mobile: time to mitigate, which is a flag flip and can be minutes, and time to fully resolve, which needs a new binary through review and an adoption curve. The gap between those numbers is the investment case for everything in this article.

Signal	Compare against	Typical halt reason	Reaches existing users
Crash-free sessions	Previous version, same segment, same rollout age	Sustained regression beyond tolerance	No — needs a kill switch
New crash signature	Absence in previous version	Any signature with real volume	No — needs a kill switch
Unresponsiveness	Previous version, same device class	Regression, especially on older hardware	No — needs a kill switch
Critical-path error rate	Previous version, same region	Contract or parsing break	Sometimes, via server fix
Journey completion rate	Previous version, same segment	Silent functional break	Sometimes, via flag or server fix
Session volume	Expected volume for rollout share	Build may not be starting at all	No — halt and submit a fix

Debugging what you cannot reproduce

The signals tell you something is wrong. They rarely tell you why. The diagnostic problem on mobile is distinct enough to deserve its own practices.

Breadcrumbs beat logs. You cannot turn up the log level on a device in the field. You can record a bounded trail of significant events — screen transitions, state changes, network calls with outcomes, flag evaluations — attached to any crash or error report. Keep it small, free of anything sensitive, and designed deliberately rather than accumulated from whatever happened to be logged.

Stack traces are necessary and insufficient. They tell you where the process died, which is frequently not where the problem started. A crash on a null value names the line, not which of six code paths produced the null; the breadcrumb trail is what distinguishes them.

Instrument state, not just events. When something fails the questions are about state: was the user signed in, was there cached data, had the config fetch succeeded, was the device offline. Attach a small snapshot to error reports. This is the highest-value addition most teams can make, and it takes an afternoon.

Build a way to get diagnostics from a cooperative user. A support-triggered bundle — breadcrumbs, versions, flag state, an identifier for correlation — turns an unreproducible report into evidence. Beta and internal distribution channels help for the same reason. Device farms complement both, catching hardware-specific defects before release.

Write down what you could not diagnose. A crash closed as unreproducible is a gap in your instrumentation, and the most specific guidance you will get about what to instrument next. Work that list during release cycles, because in an incident you have only what you already had.

What to do on Monday

Take your last significant mobile incident and reconstruct the timeline: when the bad build started rolling out, when the first signal was available, when a human noticed, when it was mitigated, when it was resolved. The gap between "signal available" and "human noticed" is what automation is for, and it is usually the largest.

Check that every error and analytics event carries app version, OS version, device class, flag state and locale. If any are missing, add them centrally this week — they are the difference between an investigation and a guess.

Then write halt criteria for your next release and attach them to the release checklist: three comparative thresholds, the segments they are evaluated in, a minimum volume, and a rota that gets alerted. Review the first few halts honestly, including the false ones, and tune. An automated halt the team trusts is the closest thing to a rollback that mobile offers.