Key Considerations in Testing Mobile Apps

The key points that need to be considered when testing mobile applications can be addressed with the following questions:

• Which mobile devices and operating system versions will this application support?
• How do we test applications to make sure they run on those platforms?
• What modifications must be made to accommodate the differences among platforms?
• How will industry innovations be supported going forward, since new mobile devices, technologies, and applications are constantly being introduced?
• How do we know how much testing is enough?

Hardware Diversity Compounds Complexity

In the PC test environment, testers have essentially only one central processing unit platform (x86- compatible microprocessors) on which they need to test applications. Most of the other hardware components that go into a PC or Mac, such as the disk drives, graphics processor and network adapters are usually thoroughly tested for compatibility with those operating systems and pose a relatively minor risk of problems. Their display formats also fall within a relatively narrow range of choices, and the input devices (mostly keyboards and mice) are well-known and familiar.

But mobile voice and data service carriers differentiate themselves by offering a wide range of handsets, each with unique configurations and form factors that can have unpredictable effects on the performance, security, and usability of applications.

Multiple handset options are built around a wide variety of processors, running at various speeds with widely varying amounts of memory, as well as screens of different sizes operating at different resolutions and in different orientations (landscape, portrait or both).

Many handheld devices rely on multiple digital signal processors (one to handle voice communications, the other to process the audio, video, and images associated with applications), as well as multiple input devices, such as a touch-screen and a keypad. Each combination of components interacts in different ways with each other, and with the operating system, and this creates potential compatibility and performance issues that must be addressed in testing.

PLUS Complexities from Software Platform Diversity

In addition to these hardware-based concerns, the tester must cope with the complexity of the software environment. To ensure that an application will work on most customers' desktops, a tester need only test it on the most popular current versions of the Windows, Apple Macintosh, and Linux operating systems.

To ensure performance on the same range of mobile devices, a tester must address all current versions of the iPhone, Windows Mobile 7, Symbian, Android, iPhone, and RIM Blackberry OSes, as well as the MeeGo platform developed by Nokia.

Applications must be tested for their compatibility with any of the networks on which any given device might run.The networks operated by different carriers provide various levels of bandwidth. Different carriers use different methods to tunnel their own traffic into the TCP IP protocol used by the Web, changing how applications receive, transmit and receive data. They also use different Web proxies to determine which websites their users can access, and how those sites will be displayed on their devices. 

All of these differences can affect the stability performance or security of a mobile application and must be tested to ensure that the the end-user experience works as intended.

Testing Mobile Apps on Device Emulators

Mobile application testing needs to take into account a wide range of target mobile devices to ensure that every possible interaction among hardware and software elements, as well as with the wireless carrier's network, is covered. However, acquiring every possible target device and performing manual testing on it is too complex, costly, and time-consuming to be feasible during every stage of testing.

Device emulators, which are software that simulates the performance and behavior of the physical device, are far easier to obtain and less expensive than samples of the physical devices. While they can be less accurate test platforms than the actual hardware, they can be a cost-effective alternative to testing on the physical device when used appropriately.

Emulators can be used to test Web applications using the software development kit for a browser or by packaging the application as a .jar, .apk or .sis (platform-specific) file, installing the application on the emulated device, and testing the application.

Since speeding time-to-market is often a critical consideration in app development, many mobile applications are developed using RAD (rapid application development) in which multiple versions of the software are quickly developed, assessed by end-users, and tweaked accordingly. This rapid-fire cycle of coding and re-coding makes it almost impossible to assess how each change affects the application's performance, stability, or security.

At Promet Source, we are adept at mobile application development, as well as the full range of processes and procedures required to ensure that apps perform as intended. While complexities continue to increase, the margin for error remains zero, and our commitment is is get it right the first time. 

Interested in help or consultation with application development or Quality Assurance of a mobile application?  Contact us today.