Are you testing your apps on the right platforms?
Guest Blogger Eran Kinsbruner, Technical Evangelist at Perfecto Mobile
As 2015 comes to a close, it’s a good time to reflect. Why not reflect about how you’re testing apps. Ask yourself: Are you testing the app for the whole digital experience, on the right devices, operating systems, and browsers?
This is the question that app developers, testers, and product managers ask themselves as app quality becomes a major focus for not only development and testing teams, but for the entire organization. The stakes are raised for delivering a high-quality end user experience across mobile and web apps. In short, the quality of digital experiences, across mobile and websites, matter more than ever. So how can you know what the right mobile devices, operating systems, and web browsers are for testing your app?
1. Start with what you know. As organizations seek to understand their end-users’ app usage, they obtain unique web traffic analytics and other insights coming directly from their customers’ specific devices. Such information is extremely useful for making decisions around the next app capability, enhanced quality for specific devices in specific locations, and more. With that in mind, I recommend you use this level of analytics for your mobile, web, and responsive web strategies. It only makes sense, organizations should test for the conditions, devices, and platforms that its customers are using.
2. Pair with industry data. There’s no better guide for your testing than real enterprise data. I’ve just published the third edition of the Digital Test Coverage Index to help you determine the right test matrix of mobile devices, operating systems, and web browsers that represents your customers, markets and geographies – you can get it here. We aggregated data from our customers on over 4,000 devices and combined it with mobile market sales data and our own analysis, to create a list of the most relevant devices, OS versions, and web browsers. It guides you in the right direction based on target geographies and market data so you can optimize your test coverage strategy.
For instance, did you know it only takes 32 mobile devices to achieve 80% test coverage for the mobile market? And did you know that these 32 devices vary widely between the US, Germany, and Australia?
Here are some key trends I found – there are more available in the index:
Devices: Legacy devices are still popular in major markets. For example, Samsung Galaxy S3, HTC One M7, iPhone 5S, iPad 2 and iPad Air remain important for various mobile and web apps and need to be part of organizations’ testing labs.
Operating systems: According to the Digital Test Coverage Index, three months after its release, iOS9 has stabilized with a 70% market share. Of users who are not on iOS9, 22% are using iOS8. It’s critical to test on both of these major platforms.
Regional trend: Also in the Digital Test Coverage Index, Apple continues to dominate in the U.S., Canada, and Australia markets with more than 40% mobile OS share in each region. In Europe, Android has a clear advantage, with an aggregated 72% market share.
Pair data like this with your own customer data to optimize your organization’s test coverage metrics.
3. Keep an eye out for the future. Test coverage planning is a continuous project. What was the top device a year ago is now totally irrelevant, and your test coverage plan must reflect these constant changes. In the Digital Test Coverage Index, we’ve included a calendar showing what mobile devices and OS versions are either new to the market or being released in Q1 2016, and a 2015 Mobile Retrospective that looks back at the year’s major releases. This should help you not only react when new devices, OS versions, or browser versions hit, but help you proactively prepare for the market updates.
Optimizing a test coverage strategy that matches end user scenarios, keeps up with the constantly evolving mobile and digital markets. Being prepared for what is yet to come is definitely a challenging part of mobile testing. Developing a test coverage strategy that addresses customer usage data, market data and the future of the market, however, will set your app apart from the rest and lead to the highest-quality end user experience. For help in aggregating data, download the Digital Test Coverage Index here and discover the best matrix of devices, browsers and operating systems for your test coverage goals.