Challenges in QA automation in mobile cloud and hybrid mobile/web

The mobile phone has transformed from a simple tool of communication into a complex device called a smart phone. Besides communication it has now replaced many devices like cameras, GPS, Radio, Calculators and even the flashlight! It is today one of the primary methods of interaction for both consumers as well as business providers globally.

Mobile phones are slowly being integrated into the computing environment and its applications delivering complex functionalities on various platforms present unique challenges not only for development but also for testing.

In order to test these devices, testers must rapidly adapt the traditional testing methods like functionality, performance and usability testing processes for the mobile.

A well thought of automation strategy is essential when it comes to mobile cloud and hybrid mobile/web as it would enable both the hardware manufacturers and the software developers to reduce effort and accelerate time to market.

Before building a QA automation strategy, we must understand the unique challenges faced by the QA testing team.

Some of the major challenges for testing in mobile cloud and hybrid mobile/web are:

Real time challenges: Nearly 65 percent of organizations do not have the right tools to test the mobile apps on cloud or on web, and 52 percent cite lack the appropriate devices to test the applications on the mobile!

Standards: Mobile devices are less accessible and less open than standard desktop environments or web-based applications making it difficult for testing.

Diversity of devices: Mobile devices range from simple handsets to smart phones, tablets, iPads and wearables. Testing the mobile apps on all these is a logistic challenge, even for cloud infrastructure providers.

Device fragmentation is a big challenge for not just developers but also testers who would like to provide a consistent User Experience across platforms

Versions: Besides the hardware, the variety of mobile operating systems and ever updating versions poses a major challenge. Moreover, a mobile app can be a native app, a web app or a hybrid app.

The challenge of testing each such app type is unique, as their implementation in itself quite different from one another.

Ensuring that web applications and websites work well on all the different devices and with different screen sizes is itself a daunting & time-consuming task as seen by the QA community.

Network Challenges: Multiple network types (e.g., GSM/GPRS/Wi-Fi/Wi-Max) bring another dimension to testing effort. Added to this mix are the varying connection speeds across geographies and multiple network operators with customized network features.

There are differences in device communication protocols (e.g., WAP/HTTP) that need to be noted when testing.

Cloud based testing: Some cloud-based test automation suits are popular for achieving native and hybrid types of test automation. But these too lack sufficient coverage in terms of devices, locations, networks and the top User scenarios.

Other interoperability capabilities like cross-platform reusability, multi-browser compatibility, network traffic, and device vitals are required to perform optimizations and improve application performance. The cloud provider may not always provide these.

Automation Scripting: Given the presence of many different devices with different interface options, automated scripting needs to be abstracted away from the device to be of any real use.

For example, a script that works on an Apple iPhone may not work on another device because of different user interfaces. Luckily real device automated testing software does provide high-level scripting that operates on the text, image, or object layer. Most device emulators are not device dependent given their capability of automating test execution using a higher-level, abstracted scripting language.

When one employs automated scripting, the cost of setting up the script will no doubt be higher than the cost of a single manual test execution at first. But, when run on a periodic basis, the more time and effort is saved, and eventually will help recover the cost of initial scripting.

It is also pertinent to note that although automated scripting tools have the capability to “spider” or “crawl” a mobile website or application, which can test an entire site with a single command, there are no tools yet (at the time of this writing) that are able to perform complex transactions like that on a mobile app.

In summary, seek first to understand the above-mentioned challenges. Then build a QA strategy with the right tools for testing in the mobile cloud and hybrid mobile/web environment.

Do understand that there will be some compromises in coverage, primarily due to device, OS fragmentation.

At Agile Soft Systems, have vast expertise in all types of mobile QA including QA Automation for mobile and mobile/web devices. To know more about how we have tested hundreds of mobile applications for our Customers,