Testing Mobile Phone Applications By André Vondran - Software Quality Assurance Analyst

Two months ago, I started learning about the joys and challenges of testing mobile wireless applications. This article is dedicated to the various tips and tricks I've collected along the way that may help you become productive much more quickly.
Reduce Setup Time

Find ways to reduce the time required to configure the phone, install the software, and learn about the underlying connectivity. For example:

Your carrier or handset manufacturer may enable you to download the Internet settings to your phone rather than trying to discover and then manually key in the obscure settings.

Often the software needs to be installed from a Web site. Use text messages to send long Web addresses. Keying a URL can take several minutes and one false move may mean starting again!

Learn how to use a computer to install the software. Many manufacturers provide free software that will enable you to add and remove software applications relatively painlessly from a computer.

Figure Out Connectivity

Mobile connectivity remains a challenge. But remember, a connection relies on at least four elements:

1. Configuration of the phone
2. The service provided by the carrier (and paid for by the user)
3. The connectivity between the carrier’s wireless network and the Internet (where gateways can filter, modify, convert, or even block communications for various reasons)
4. And the rest of the connection to the Web/application server, which may include more gateways, firewalls, etc.

Phones provided by carriers may have the network settings preconfigured. If you have to configure these settings, you will need to learn about Access Point Names (APN), WAP gateway settings, and, possibly, login details for the service provided by the carrier.

Understand Your Data Plan

Carriers may offer a range of data services, from very limited access to a small list of approved Web sites (called a walled garden in the industry) to full "Internet" access that may even allow Voice over IP, video streaming, etc. Some carriers provide clear information on which services are available for each price plan; for others, you may have to research what services and Web addresses work reliably.

Check how much you pay for data before embarking on data-intensive applications. I had a monthly data bill that was more than $300—even though I didn't use any of the installed applications on my phone during that time. However, one of the applications polled its server in the background while I was abroad. At $16/MB transferred, it was an expensive lesson to learn!

Understand the Impact of Transcoders

Transcoders help make a majority of the Internet more accessible to mobile devices; however, sometimes transcoders have unexpected consequences for the application you want to test. For browser-based Web content, these problems are often visible and therefore relatively easy to discover. However, for custom applications written to run on the mobile devices, transcoders may adversely affect the data communications and cause the application to report errors or even crash without telling you what's going wrong.

Find Ways to Understand and Simplify Problems

I have found diagnostic client software and diagnostic Web servers particularly useful for discovering and debugging issues with transcoders. Both the client and the server are designed to report the information that is sent and received. Find out whether the content is expected to be transcoded, and if so, how. If not, the data sent by one end should be received unchanged at the destination and vice-versa. The diagnostic software recorded all the data and made problems easier to detect.

If possible, run your diagnostic software on a personal computer with a wireless data modem to debug transcoder or carrier network issues. Generally this is much easier than trying to debug client software on a phone.

Use Complimentary Tools

Find complimentary ways to test using Web browsers for Web-based mobile sites. Firefox has numerous free plug-ins that emulate a phone’s Web browser and make manual testing easier. I use the following: WMLBrowser, Web Developer, User Agent Switcher, and Modify Headers.

Reduce the Number of Combinations

As there are thousands of permutations of phones and carriers, pick an exemplary subset of phones to test with. For instance, when testing Java software (written in Java 2 Micro Edition), I test on classes of phones that include: Nokia Series 60 second and third editions; Sony Ericsson’s Java Platform 6, 7, and 8 phones; and BlackBerry models based on the keyboard layout and operating system version.

Pick popular phones and phones with large and small screens and a variety of keyboards, including: T9 (where the alphabet is split across the numeric keys 2 to 9), QWERY, and other unusual keyboard layouts. Over time you may collect "interesting" phones that help expose application flaws. For example, one of my phone's core software has been highly customized by the carrier and has exposed limitations in applications that appear very quickly. By finding and reporting these issues early, the developers were able to revise their application software so it was much more flexible and robust.

Here's a site that details another way to classify your phones based on the operating system and UI: Using a Device Hierarchy.

Take Good Notes

Remember to use the recording tools, now! When you find a possible problem during testing, start by recording immediately what you see, as some messages are transitory and clear within a short period. Find ways to record UI issues, on-screen messages, etc. I use:

A good-quality, inexpensive digital camera with a sturdy, close-up stand

Free software that takes screenshots on certain types of phones, such as fexplorer (for Symbian Series 60 phones), and similar tools from Research-In-Motion (called JavaLoader) or from shareware vendor

Now document what relevant events preceded the problem, including things like signal strength, battery level, as well as actions you performed. Try to reproduce the problem on that device. Before considering the bug final, try similar and different devices, different carriers, etc. While this work may take anywhere from minutes to a few hours, the answers may help the developer hone in on the problem much more accurately and quickly. Be practical, and don't forget to ask the developer for help and advice before you expend too much effort.
Practice Makes Perfect

Testing mobile wireless applications effectively takes time and practice, particularly if you are new to the field. I hope you will find some of these tips useful. Get comfortable configuring and using a variety of phones, and you'll soon be much more capable and productive as a tester of mobile wireless applications.

The next lesson is how to automate some of the testing—a great subject with plenty of scope for you to develop your skills and make a big difference in the industry!

It is really hard to get hold of handsets and go through all the configurations and manual testing. There is service available on www.deviceanywhere.com which makes this job a lot easier by giving access to so many handsets out there in real networks. Above all they also provide tools for automating tests.

André Vondran SQA Analyst
www.aqasol.com.ar