As someone who works on day-to-day basis with usability testing and user experience design, my interest was piqued. Were these users just iPhone users who expressed dissatisfaction after a brief flirtation with Android, or was there something deeper going on?
I can’t honestly say I’ve had a lot of experience with Android – although I do own two iOS devices – so I couldn’t write off these concerns one way or another. But rather than basing it off a few, possibly biased opinions, it seemed the fairest way to compare the two was to set up a quick usability test.
A/B Testing and Preference Testing
This particular tests uses two different questions types; an A/B test, and a Preference test.
An A/B test is where you are presented with either image A or B (not both) to see how you interact with it in isolation. When looking at the results you get a fair comparison between A and B, and thus have a fair idea of how the two different designs or interfaces compare. It’s easy to judge which interface is the winner if it takes users a lot longer to interact with one than the other, or if they frequently select the wrong location or button for a simple task.
A Preference test has the two designs or interfaces side by side. This is a bit more of a popularity contest, but in the interests of fairness I’ve obscured the ‘status bar’ (the bar at the top) of both the iOS and Android apps in the preference test in order to try to eliminate people’s personal, preconceived preferences. When looking at the results, we can hopefully detect a trend about which interfaces or designs people prefer.
I know there are many people out there firmly on one side or other of this debate, but hopefully this test might help you to reconsider your preconceived notions.
The Test and Results
For each of the apps in the test, I tried to think of a very simple, common task that people might be trying to perform. If it works out that people take roughly the same amount of time to complete each task, or it works out that there is a huge difference between the two platforms or even from app to app, then we can start to form some judgements about what is going right or wrong, and which platform is more or less usable
If you are interested you can take the test yourself to add to our results.
On to the questions and results:
1) The Twitter App: How would you view your @ messages?
If you are a Twitter user, and even if you’re not, you are probably familiar with the @ message protocol. For those who aren’t, @ messages are the method you use for contacting other users on twitter. For example, if you wanted to contact me, you could send a message to @IntuitionHQ.
As a very simple, very common task, this should really take no time to complete. As it’s the first question in the test, we might expect people to take a few seconds longer as they are familiarising themselves with the interface.
The results for both for the Android and iOS versions were very similar; the most surprising thing here was the amount of people who clicked the direct message button instead of the @ message button. I guess Twitter still isn’t quite as popular as I may think.
Android 1 – iOS 1
2) Google Translate: How would you change the target language from German to another language?
Changing languages is obviously a core piece of functionality for a translation app, and should be quick and easy to do. Lets see what the results show:
Again, quite similar results. The Android version had a shorter response time at the time of writing by 2 seconds (possibly because of the addition of the arrows on the dropdown?) but also had slightly bigger spread of results. More or less a tie for these two.
Android 2 – iOS 2
3) Google Translate: How would you listen to the German audio pronunciation?
This is one of the really handy features of the Google Translate app – you can make it talk to you in a ton of different languages. Extremely useful when you are travelling or trying to learn a new language, and a common task within the app.
This is the first time (at the time of writing) that there is a bit more of a spread in the results. The Android app had a longer click time (average of 12 seconds), but an 80% success rate, whereas the iOS app had an average click time of 6.42 seconds but only a 52% success rate.
Regardless of click time, a 52% success rate isn’t nearly high enough, and so the Android version wins this battle. The Android app Speaker icon must be clearer to most users.
Android 3 – iOS 2
4) Soundhound: How would you share this song?
Soundhound is a handy app for finding the name of songs you are listening to but (obviously) don’t know the name of. The social aspect is obviously a big part of making a successful app in this day and age, so it’s essential that users can find and understand these task very quickly.
Interesting results; although the click success percentage is more or less the same (93% to 95%), the average click time is really very different. The iOS version took around 13 seconds longer in our test than the Android version. A win for Android.
Android 4 – iOS 2
5) Soundhound: How could you purchase this track?
Buying the songs that you’ve gone to the effort of finding is another handy feature of this app. How does the music purchasing stack up?
Android is slightly ahead here, with an average click time of 5.83 seconds to the iOS versions 7.64, and a success rate of 100% to iOS’ 90%. Still a win for Android, but I’ll give half a point to iOS for still a very strong results.
Android 5 – iOS 2.5
6) Pandora: Which interface do you prefer?
This is a preference test to see how users feel about the different designs of the Android and iOS apps. I’ve removed the status bar at the top of the apps in order to make people think a little more about which is the Android or iOS version of each app.
And now the results for the first of our preference tests; it couldn’t be any closer with both the Android and iOS versions of the app getting 17 clicks each. A draw.
Android 6 – iOS 3.5
7) Pandora: How would you pause this track?
A very common task in any music listening application; I’d expect a very strong response on this question.
The results here (perhaps unexpectedly) are both very strong. Both apps had a 100% success rate, and with just over half a second between the average click times (3.64-2.86 seconds) I’m going to give both apps a point for this test.
Android 7 – iOS 4.5
8) Geocaching: Which interface do you prefer?
Another comparison between an Android and iOS interface, this time for the official Geocaching app. Geocaching – for those who don’t know, is a process where you navigate to certain GPS coordinates in order to find a hidden ‘cache’.
The designs of the two are further apart than some of the designs we’ve seen, so it will be interesting to see what the results show up.
Another overwhelming result, but this time to the iOS versions – 91% of respondents preferred the iOS version of this app.
Android 7 – iOS 5.5
9) Geocaching: Please select the cache you’ve previously found:
Once you’ve found a certain cache, you tag it in the app and it then shows up as previously discovered. This is a quick test to see if that is easily noticeable in the Android and iOS apps.
An interesting result here. The click time for both apps was very similar, but the success rate for the iOS version was only 40% compared 100% for the Android version. Quite surprising to me that there was such a big difference between these two, but there we are. A win for Android.
Android 8 – iOS 5.5
10) Facebook: Which interface do you prefer?
Another preference test with an app that most people would be familiar with. Facebook has made a couple of changes between these two apps, and it will be interesting to see what the results show about the differences
Definitely the majority support the iOS version of the Facebook app here. I wonder how much of this is because they are already familiar with the design? The final results was 74% for iOS and 26% for Android. A win for iOS.
Android 8 – iOS 6.5
11) Facebook: How would you view your friend requests
Facebook is all about friends, so accepting requests is core functionality for this app. It may only make a few seconds difference, but every part of the user experience counts.
An extremely close result here, and in fact any of the selected areas allow you to view your friend requests. Considering the results are within a quarter of a second, I’m going to call this one a tie.
Android 9 – iOS 7.5
What does it all mean?
So the final result (as of writing) is Android 9 and iOS 7.5; you can also view the most recent usability test results yourself to form your own conclusions.
What we can conclude though, is that Android and iOS are very similar in terms of usability, at least in the apps we have tested. Of course, there are apps on both platforms that are really very well designed, and apps which should never have been let into the store/market. Neither platform is immune to poorly designed apps.
One difference that also needs to be considered as part of the user experience is the hardware, and this is where some issues come to the fore. Although you can buy very high quality Android phones, you can also buy very cheap, very poorly designed phones which can cause a terrible UX. One thing you have to admit about Apple is that their hardware is of a very high (and well designed) quality.
Regardless, this test shows that with the right hardware and software combination, both of these mobile operating systems can shine, and really which you choose can come down to personal preference, without worrying about a huge gap between the two platforms. Either way, you should be a happy camper.
What do you think about the results of this test? Why not create your own test to see what you find? Be sure to let us know your thoughts in the comment section below, and let us know if there is anything else you’d like to see us test. Thanks!
