AWS Device Farm mobile app testing excludes iOS—for now

Mobile app developers can turn to AWS Device Farm for testing on physical devices, but only on Android and Fire OS.

NEW YORK – Amazon launched a new mobile app testing option for developers, but it won't support Apple or Microsoft devices -- at least, not yet.

Next week, mobile app developers can start using Amazon Web Services (AWS) Device Farm to test Google Android and Amazon Fire operating system (OS) apps on devices in the AWS Cloud.

By uploading an app to Device Farm and testing it against real devices, developers can get results in minutes showing bugs or performance issues.

Tests can be run across many different physical devices of varying manufacturers, OS versions and form factors. Using physical devices as opposed to emulators for app testing takes into account memory, CPU usage, location and modifications by the manufacturers and carriers, AWS said.

Testing on an actual device instead of an emulator is the better way to go, said Adam Bookman, co-founder of Propelics Inc., a mobile strategy and app development company in San Jose, Calif.

"When you put [an app] on an actual device and you are able to utilize gestures and introduce interruptions (like going offline or getting a call) and you can get the camera to work and other device specific features, it’s a whole new ballgame," Bookman said.

Device Farm's service plugins and APIs are available to automatically initiate tests from continuous integration systems including Jenkins CI.

You'll never replace physical devices but you can't have thousands of people testing every device there is.
Reshat Hasankollimobile developer, News America Marketing

The move to introduce Device Farm for AWS comes a year after Google bought its way into the mobile app testing space with Appurify, a cloud service that also lets developers test apps on physical devices.

Fragmentation of the Android market is a key issue when it comes to development, said Reshat Hasankolli, mobile developer for News America Marketing, a division of News Corp. based in Wilton, Conn., here at the AWS Summit this week.

"You'll never replace physical devices but you can't have thousands of people testing every device there is -- and you can go out and buy your own, but as soon as new devices are introduced a service like this pays for itself," he said.

Instead of spending his time testing different physical devices, he could spend time writing good test cases and automation thanks to Device Farm, Hasankolli said.

No iOS for Device Farm yet

Device Farm has limits that might not make it the best testing option for all development shops. It won't support testing of iOS apps immediately, but adding more device platforms is planned for the future, said Trent Peterson, product manager for AWS here at this week's AWS Summit.

There was no mention of support for testing Windows Phone apps as part of AWS Device Farm's release.

In contrast, other similar platforms such as Perfecto Mobile Ltd. and Keynote LLC support iOS app testing on physical devices. *Google’s Apppurify, now in preview, will only support Android devices when it becomes available. Google said it has no immediate plans to launch iOS support.

Also, Device Farm will only be available in the Western region of the U.S. upon its release and Peterson didn't say when it will be available in other regions.

Developers can either use AWS's built-in test suite for Device Farm or use open-source test frameworks such as Calabash from Xamarin Inc., Appium from Sauce Labs Inc. and Espresso from Google. However, Device Farm doesn't support Android Studio, Peterson said.

Only app functions that can be done over WiFi are supported in Device Farm app testing. App upgrade flows and dependencies are also supported. Remote access to individual devices through Device Farm is not supported, Peterson said.

Pricing for Device Farm is based on device minutes, with the first 250 minutes free. After that, customers are charged $0.17 per device minute. There's also an unmetered testing plan for unlimited testing of $250 per month per device.

That $250 monthly fee is concerning for Kevin Felichko, CTO of, an online auction site based in Frederick, Md.

"Why not just go out and buy a low-end consumer device instead?" he said.

When asked this question, AWS chief of data science Matt Wood said customers would only have one low-end consumer device as opposed to a whole smorgasbord of devices for testing. In addition, to cover 80% of the Android market, you'd need to test across more than 100 devices, Wood said.

Despite his concerns over price, Felichko said the Device Farm idea is a good one.

"I've been in the situation of developing for different Android devices, and there are tough decisions you have to make," he said. "I like the idea."

*The story originally included Appurify in a list of platforms that support iOS.

Beth Pariseau is senior news writer for Write to her at [email protected] or follow @PariseauTT on Twitter.

Jake O'Donnell is the news writer for and He can be reached at [email protected]. Follow him on Twitter @JakeODonnell_TT.

Dig Deeper on EMM tools | Enterprise mobility management technology