Reading Test Quality in Play Console — Crash-Free Rate, ANRs, Pre-Launch Reports, and the Vitals That Actually Matter
Your 12 testers have been using the app for two weeks. Installs look fine, nobody complained loudly, and the production access form is staring at you. Before you submit it, there is one question worth answering honestly: did the test actually prove the app is stable — or did it just prove 12 people managed to install it?
Play Console already has the answer. It is sitting in Android vitals and the pre-launch report, in numbers most developers glance at once and never read properly. Here is how to read them, which ones matter before you apply for production access, and which ones you can safely ignore.
Start with the two numbers that matter most
During closed testing, two rates tell you almost everything about stability. Both live under Android vitals, and both are measured per day, not per release — so check them across the whole 14 days, not just today.
- Crash-free sessions. The percentage of app sessions that ended without a crash. If this is 99% or higher across your test, your build is holding up under real use. If it is 97% or lower, something is crashing regularly — and your testers are the ones hitting it.
- ANR rate. The percentage of sessions that froze long enough for Android to show an "app not responding" dialog — the user-perceived ANR rate. Google treats this separately from crashes because a frozen app feels just as broken to a user, even though nothing technically crashed.
A test can look healthy on installs while both of these numbers are quietly bad. Twelve testers who installed the app, opened it once, and hit a crash on the second screen will still show up as 12 installs. The crash-free rate is what catches that.
What counts as good — and what counts as a problem
Google publishes bad-behavior thresholds for vitals, and the production access review cares about them even if nobody quotes them to you directly.
- Crash rate: Google flags an app when its user-perceived crash rate is above roughly 1% of daily sessions. During a 14-day test with 12 testers, even a handful of crashes a day can push you over that line, because your session count is small. One crash on a day with 20 sessions is a 5% crash rate for that day.
- ANR rate: the threshold is lower — roughly 0.47% of daily sessions. ANRs are rarer than crashes in most apps, but they are also easier to miss, because a tester whose app freezes for ten seconds often just force-closes it and never reports anything.
- Excessive wakeups and stuck wake locks also appear in vitals, but during a short closed test they are usually noise unless your app runs background work. Note them, fix the obvious ones, and do not let them distract you from crashes and ANRs.
Do not panic over a single bad day. One tester on an old phone with a full storage can produce a crash spike that says nothing about your app. What matters is the pattern across the 14 days: is the crash-free rate stable and high, or does it dip every time testers actually use a particular feature?
Reading crashes: the stack trace is the diagnosis
Vitals gives you crash counts. The crash reports behind them — in Play Console under Android vitals, and in the pre-launch report — give you the cause. Read them the way a doctor reads test results: group first, then diagnose.
- Group by stack trace, not by count. Fifty crashes from one null pointer in your login flow is one bug. Five crashes from five different places is five bugs. Fix the biggest group first — it is usually the one your testers hit most.
- Check which devices and Android versions crash. A crash that only happens on one device model with an old Android version is a different problem from a crash on every device. The pre-launch report tests your app on real devices in Firebase Test Lab, so compare its device list with your testers' devices: if the report crashes on a device none of your testers own, your test never covered it.
- Look at when the crash happens. Crashes on first launch are the worst kind — they mean a tester can never get past the front door. Crashes deep in a feature are still bad, but at least they tell you the tester got that far. Sort by "first launch" crashes and kill those first.
- Do not trust a crash you cannot reproduce — but do not dismiss it either. If the stack trace points at a real code path, it is real, even if it never happened on your own phone. Your phone is one device; your testers are twelve different ones.
Fix the top crash group, push the fix through the tracks as a new build, and watch the crash-free rate for a day or two. If it recovers, the fix worked. If it does not, you fixed the wrong thing — or there is a second crash hiding behind the first.
The pre-launch report: a free test you are probably ignoring
Every time you upload a build to a test track, Play Console can generate a pre-launch report: your app is installed and exercised automatically on a range of real devices, and the report comes back with crashes, ANRs, screenshots, accessibility issues, and security warnings.
- Read it before your testers do. The report arrives before your 12 testers even open the app. A crash it finds on launch is a crash you can fix before a single tester sees it — which is the entire point of testing in the first place.
- Check the screenshots, not just the crash list. The report includes screenshots from each device. A layout that breaks on a small screen or a button hidden behind a notch will not show up as a crash, but it will show up in a screenshot — and it is exactly the kind of thing a tester would quietly uninstall over.
- Mind the accessibility and security flags, but triage them. A missing content description on an image is worth fixing; it is not worth delaying your test for. A security warning about an exposed API key or cleartext traffic is worth fixing before anyone outside your team touches the build.
- Compare reports across builds. When you push a fix, the next report should show fewer crashes than the last. If it does not, the fix did not land — check that the new build actually reached the track before you celebrate.
The pre-launch report is not a replacement for 12 real testers. Automated testing never taps the button your users tap first, never gets confused by your onboarding, and never tells you a feature is pointless. It is a first pass: it catches the crashes your testers should never have to find.
The numbers that do not matter (yet)
Not every number in Play Console deserves your attention during closed testing. Some are useful later, at production scale, and misleading now.
- Total installs. With 12 testers, this number barely moves, and it says nothing about quality. A tester who installed and never opened the app counts the same as one who used it daily.
- Store listing views and conversion. Your app is not on the store for the public during closed testing. These numbers are either empty or meaningless until open testing or production.
- Ratings and reviews from testers. Testers on the closed track can leave private feedback, but it does not become a public rating. Read their words — they matter more than any star count at this stage.
- Battery and network usage details. Unless your app is draining batteries or eating data in a way testers notice, these are fine to check once and then leave alone until you have thousands of users.
The trap is spending your test watching the dashboard instead of the testers. Numbers tell you the app is stable. Testers tell you the app is worth using. You need both, and only one of them lives in Play Console.
Turning vitals into evidence for the production access form
The production access form asks what you tested, what you found, and what you fixed. Vitals give you the numbers to answer it properly instead of with adjectives.
- Record the crash-free rate across the 14 days. "Crash-free sessions held at 99.4% across the test, after fixing the login crash found on day 3" is evidence. "The app is stable" is a claim.
- List the top crashes you found and fixed, with dates. One line each: what crashed, which build fixed it, and what the rate did afterwards. Three fixed crashes with numbers attached is a stronger application than a test with no crashes at all — because a test with zero findings looks like a test nobody did.
- Note the ANR rate and what you did about it. Even if it was zero, say so with the number. "ANR rate 0.00% across 14 days" is a fact a reviewer can check.
- Attach the pre-launch report summary. The latest report, after your final build, showing no launch crashes on the tested devices. It is the one piece of evidence that does not depend on your testers at all.
- Be honest about the gaps. If one device model still crashes occasionally and you know why, say so and say what you will do about it. A known issue with a plan reads better than a perfect report nobody believes.
Keep this page of numbers with your tester log — who opted in, who stayed installed, what they reported. Together they are the application: 12 real testers, 14 days, a stable build, and proof you read the results.
The vitals checklist before you apply
- Crash-free sessions at 99% or higher across the full 14 days, not just the last day.
- ANR rate below the bad-behavior threshold, checked per day.
- Top crash groups fixed, with the rate recovering after each fix.
- Pre-launch report on the final build: no launch crashes, screenshots checked on small screens.
- Numbers written down — crash-free rate, ANR rate, crashes found and fixed — ready for the form.
- Tester feedback read alongside the numbers: stability plus real use, not stability alone.
If any item fails, fix it first. A week spent fixing a real crash is cheaper than a rejected application followed by another 14 days.
The part no tool does for you
Play Console gives you the vitals, the crash reports, and the pre-launch report. What it does not do is find you 12 real people with Android phones and different Google accounts, get each of them opted in on the right account, keep them installed and engaged for 14 continuous days, replace the ones who go quiet, and hand you a final report built for the production access form — so that the numbers you submit come from a test that actually happened.
That is exactly what VerifiedTesters handles: 12 real testers on their own Android phones, opted in on your Play Console closed track and engaged for 14 continuous days, with free dropout replacement and a final report built for the production access form.
Read the vitals. Fix what they show. Then let your 14 days count.