Getting 12 testers is the easy part: what 1,500+ Google Play closed tests taught us about tester activity
If you're running a Google Play closed test right now, the headcount is not the hard part. Getting 12 people to opt in takes a day or two. What decides whether Google opens production for you is whether those 12 people actually use the app during the 14 days.
I run TesterBee, a paid service that manages closed tests for Android developers, so I see the outcome of a lot of them. We looked at 1,500+ campaigns to find out what separates the apps approved on their first application from the ones Google sends back. Disclosure up front: the numbers below come from our own campaigns, not from Google.
What Google actually says
The rule, from Google's help page: personal developer accounts created after November 13, 2023 must run a closed test with at least 12 testers who have been opted in continuously for at least 14 days before they can apply for production access.
The same page lists why Google asks for more testing: "having fewer than 12 opted-in testers or insufficient tester engagement during the testing period."
That second reason is the one that catches people. Google doesn't publish a number for "enough engagement", so we looked at what the approved apps had in common.
The numbers
| During the 14 days | Approved on the first application |
|---|---|
| Testers opened the app on 10+ different days (campaign average) | 97% |
| Testers opened the app on 5–7 days | 41% |
| Developer pushed 2 or more updates | 89% |
| Developer pushed no updates | 53% |
For context: across all our campaigns, 72% are approved on the first application. Of the apps that get rejected, 91% are approved on the second attempt once the developer fixes what Google flagged. Overall, 98.4% eventually get production access.
Two caveats. These are correlations, not Google's thresholds: an app that keeps testers coming back for 10+ days is probably also a better-built app, and both things help. And an "active day" here means a day the tester opened the app at least once, not a day they spent an hour in it.
Still, a gap from 41% to 97% is too big to ignore. How active your testers are across the whole window matters far more than how many names are on the list.
When testers drop off
Day one is easy: nearly everyone installs and pokes around. The drop comes in the middle of the first week, once the novelty wears off. Friends and family are the worst for this; they mean well and have forgotten by day five. Testers who are still opening the app around day 11 almost always finish.
So the risk window is days 4 to 7, and that's where to spend your effort.
What to do with this
- Fix launch crashes before you invite anyone. A tester whose app crashes on day one rarely comes back.
- Plan two or three small updates. One around day 5 and one around day 10 is a good rhythm. Each update gives testers a reason to open the app again and shows that you're using the test to improve the app, which is what the test is for.
- Measure active days, not opt-ins. Play Console tells you who opted in, not who opened the app. Log a simple app-open event to whatever analytics you already ship and count distinct days per tester. If someone hasn't opened it in three days, message them.
- Recruit two or three extra testers. If someone leaves, you still have 12 who have been opted in the whole time.
- Give testers something to do. Google's own advice is to "provide beta testers with clear instructions on how to test your app and report bugs." A short list of things to try each week beats "please test my app".
- Describe the engagement honestly when you apply. The production access application asks about your closed test, including the engagement you got from testers. Say what they did, what they reported, and what you changed because of it. The updates from point 2 give you real answers here.
The short version
Opt-ins get you to day 14. Activity gets you approved. Aim for every tester opening your app on at least 10 of the 14 days, and push two or three updates while the test runs.
The full breakdown, including drop-off by day and the device mix our testers use, is here: How active do closed testers need to be? Data from 1,500+ campaigns.
If you're stuck in a closed test or got the "more testing required" message, ask in the comments and I'll answer what I can.