One of the most tedious—and depressing—chapters in my book App Savvy relates to the beta testing process for iPhone and iPad apps. As I indicate there, it’s a necessary evil because launching a new app or a new feature into the App Store without testing it is simply asking for trouble (more on that in a separate post). Aside from having to collect the beta testers’ device UDIDs, continuing to distribute new builds to those testers and needing them to continually reinstall the updated apps is a time-consuming, inefficient, and frustrating process.
In the past few weeks, however, there’s been an avalanche of new tools and frameworks that tackle these issues head on, by providing over the air (wireless ad hoc) updates of beta apps. The idea is simple: once a beta app is installed, the tester will automatically receive the latest build when the developer pushes it to them. Let’s take a look.
Of all these new tools, TestFlight is currently the most interesting. Used by Neven Mrgan during the development of The Incident, TestFlight appears to not only provide over the air updates of beta apps but also helps with, “streamlining what happens before and after the builds as well: signing up new testers, grouping them for A/B testing and different tiers of builds, managing tester feedback, and hopefully much more.”
Apparently, there was a cool iOS app companion that was recently rejected by Apple that could, “collect new users’ UDIDs and send out push notifications when updates [were] available.” That’s definitely a bummer but the good news is that 23Divide—the developers behind TestFlight—are working on both web and desktop apps that will offer similar functionality. Here’s the TestFlight workflow, both from a developer’s and tester’s perspective:
*A developer’s TestFlight workflow.
1. Invite testers via -email
2. Upload your IPA
4. Watch as TestFlight parses everything out of the IPA (version, name, icon, list of testers, expiry date, etc.)
5. Continue to watch as TestFlight notifies your testers that a new build is available for installation.
6. Monitor the dashboard to see who has installed the build, who has crashed the build, how long the testers have tested, etc.
*A tester’s TestFlight workflow.
1. Install the TestFlight app via the app store (free)
2. Sign up / in
3. Accept any invitations to test (when you accept the invitation, the developer is given your UDID)
4. Receive notifications for new builds
5. Touch install
From what I’ve seen, Andreas Linde looks to have been the first to discuss his over the air beta framework “Hockey” just about two weeks ago ( which I had linked to then). I’m assuming that put the pressure on others who were doing the same, causing this deluge of over the air iOS beta tools to launch.
A week after his initial announcement, he formally launched Hockey as an open source iOS framework:
Before Hockey, developers needed to inform the testers, give them a link to download the binary, instruct them how to add the files to iTunes, connect the device, sync, and if it didn’t work, start all over again.
With Hockey, this is all history, as long as the beta tester has iOS 4 installed on the device. The beta application can be installed by clicking on a link on a webpage and the application will notify the user automatically when it starts, if there is an update available and the user can install it from within the application. Wherever they are and whenever they want. No iTunes required, in fact, not even a computer is required any more! All that is required, is a webserver and some simple code to be integrated.
iOS Beta Builder
Finally, there’s iOS Beta Builder, a Mac-powered app. iOS Beta Builder is, “a simple MacOS X app takes your archived IPA file and creates the required manifest and HTML files for wireless distribution. It even zips up a copy of the app for folks on 3.x that need to install via iTunes.”
Since all of the over the air tools require iOS 4, it’s nice that iOS Beta Builder also provides a version for those still running on 3.x.
These are exciting updates and invaluable tools/frameworks for iOS developers. It at least appears that Hockey and iOS Beta Builder are open source, while I’d guess that TestFlight will probably have some sort of cost. But with the additional tools TestFlight might offer (e.g., collection of UDIDs and feedback), it would be well worth the investment to run a much smoother and more insightful iOS beta program. Note that it’s also likely that Apple will release similar tools / frameworks in the future…in the meantime, definitely take advantage of these options.