Testing Times: Making sure your apps work

I recently briefed an Android app to an agency to create and when it came over for testing, it broke in a matter of minutes. It wasn’t a particularly sophisticated app, and at first glance it appeared to work well, but when you actually used it for any length of time, it suffered from a bug that made it progressively slower, and data glitches became apparent as more of the content was revealed.

The developer had probably done enough testing to confirm his belief that it was working, but hadn’t thoroughly investigated that it did everything as it should. That’s the art of testing: it’s not about confirming things work, it’s about making sure that nothing is broken, a subtle but important distinction.

As well as checking for obvious bugs, there is a grey area, where an app is designed to work in a way that seems counterintuitive to users. If it feels ‘broken’ when someone tries to do something, it is.

Arguably, testing is the most important part of the whole process of creating Android apps. Apps that offer a bad user experience can get a real kicking in reviews, which deters further downloads. If an app is badly designed, it deserves everything it gets, of course, but it’s heartbreaking when a great app is spoiled by slipshod quality control.

THE EMULATION SOLUTION

Emulators can be a good starting point for weeding out obvious problems. Intel has published emulator system images that enable you to test your Android apps on a virtual Intel-Architecture based device, under the Jelly Bean, Ice Cream Sandwich and Gingerbread versions of Android. (Download Intel’s Android emulators here; you need to click on the Tools and Downloads tab.)

Emulators are a valuable tool, but they’re not enough in themselves. You should also test with as many real devices as possible. Independent app developers might struggle to test with more than a handful of real devices, but even the bestselling devices represent a relatively small proportion of the Android market, so it’s a good idea to test with as many as you can source.

In particular, now that Intel Android devices are entering the market – including the Orange San Diego, and the Motorola RAZR i handsets – it’s important to test apps on this new hardware architecture as well as devices based on other processors.

THE POWER OF THE USER

It’s a good idea to involve users in your testing as much as possible too. Experienced app developers have a good understanding of how users like to use devices, but there are usually a few surprises when any usability testing is carried out, especially if you’re creating apps for audiences outside the main base of young-to-middle-aged phone users. Children and older people can have different expectations of how apps should work, and can need additional guidance, because they don’t have the same reference points that more experienced app users might have.

As well as identifying usability problems, users can often uncover new bugs because they will try things that you might have overlooked in your exploratory testing. While developers must make a conscious effort to try to break the app by doing things it’s not designed to do, users will often try things because they don’t know they’re off limits, and can accidentally reveal bugs that more deliberate testing didn’t find.

Although an app development project can go through phases that are dedicated to testing, such as user testing (with wireframes initially, and then with the app in various stages of development later) and emulation (once there is code available for that), testing isn’t so much a process as a state of mind. The idea of proving everything works as users expect should be interwoven throughout the whole design process.

What tools and techniques have you found useful for making sure your mobile apps work as designed?

—

This blog post is written by Softtalkblog, and is sponsored by the Intel Developer Zone, which helps you to develop, market and sell software and apps for prominent platforms and emerging technologies powered by Intel Architecture.