A funny thing happened the other day. I was sitting at my computer – a Dell XPS 13 – working my day job, which is for an iOS app developer. I carry an Android phone and I was sitting in the room where I keep my Echo Dot. I was sitting in a room with four digit assistants within earshot – Cortana, Siri, Google, and Amazon. So I decided to conduct a little experiment – and assistant showdown , if you will – as illustrated in these tweets.
Who won the Stanley Cup in 2016? Siri: The Penguins, beating the Sharks, this was the score of the last game Google: The Penguins (1/2)
— Adam Doud (@DeadTechnology) January 13, 2017
Who won the Stanley Cup in 2016? Cortana: Here were the hockey scores last night. Alexa: I don’t have the answer to that question.
— Adam Doud (@DeadTechnology) January 13, 2017
As you can see, the results from my experiment were quite varied, plus I legitimately couldn’t remember who won the Cup, so win-win right? But seeing these results made me wonder where else digital assistants might vary in their responses. So, I called up the team and gave them each one question to ask. For the test, I used an iPad, Nexus 6P, Google Home, Kindle Fire, and the aforementioned Dell XPS 13 and Amazon Echo Dot. In some cases, having a screen made a difference. Here were the results:
Sing me a song.
Siri – “You know I can’t sing, Adam.”
Cortana – *singing* “Oh, he floats through the air with the greatest of ease, this daring young man on the flying trapeze.”
Google Home – *singing* “I can sing. I can sing a song. I’ll keep it simple, it won’t last very long.”
Google Phone – Search result of a Youtube video called “Sing me a song”
Alexa (Kindle and Echo) – “Who me? I couldn’t. I…hit it! *band plays* When my wifi left me, and I’m out in the rain. Those last few answers were hard to obtain. But that’s no excuse to put me on mute. Like a good cowgirl, I’ll up and reboot. It’s raining in the cloud and I ain’t proud. I’ll answer in the morning, answer in the evening when you’re around. I’ll answer in the morning, answer in the evening, when you’re around.”
Results – Alexa won this contest handily. Not only did the assistant sing the same song on both devices, but a full band accompanied it. Here’s a non-Pocketnow YouTube video of it. I’m talking studio time. The other three assistants recognized the command to sing a song, and the Nexus 6P just showed me a YouTube video. Amazon was clearly waiting for someone to ask this question.
Why did the chicken cross the road?
Siri – “To get to the next question. Next question.”
Cortana – “I thought you’d never ask, so I’ve never thought about it.”
Google Home – “There are millions of search for that which is a lot of chickens. You’d think they’d carpool.”
Google Phone – “Why did the chicken cross the road” is a common riddle joke. The answer or punch line is: “To get to the other side.” It is an example of anti-humor, in that the curious setup of the joke leads the listener to expect a traditional punchline, but they are instead give a simple statement of fact.
Alexa (Kindle) – “That’s really only the Chicken’s business.”
Alexa (Echo) – “Perhaps the Chicken saw some tasty corn? I don’t know.”
Results – In this circumstance, we have a similar field of responses – something quirky and funny. My favorite was Google Home. Ironically the least favorite was the Nexus 6P which described the joke to me. Everyone knows, if you have to explain a joke…
Anton D. Nagy
Remind me tomorrow morning, half an hour after I wake up, to reply to Adam’s email.
For this test, I first set an alarm on each device for 6:10am. Made for a fun morning.
Siri – Set reminder for 7:30
Cortana – Set for 8:00am.
Google Home – “Sorry, I can’t set reminders yet.
Google Phone – Set for 8:00am.
Alexa (Kindle and Echo) – Added “tomorrow morning half an hour after I wake up to reply to Adam’s email to my to do list.
Results – Of these responses, none of them performed the correct action, with Siri coming the closest. I suspect that just because Google defines “morning” as 8:00am, and Apple defines it as 7:30am. While Alexa was the most useless in this case, Google Home was the most disappointing to me personally. Why can’t Google Home set reminders yet?
When is my wife’s birthday?
Siri – “Sorry, I don’t see [my wife’s name] in your contacts. Can you try again?”
Cortana – Bing results for “When is my wife’s birthday?”
Google Home – Hmmm I don’t recall
Google Phone – Showed me my wife’s contact card with Birthday
Alexa (Kindle and Echo) – “Here are the next 4 events” It literally showed/read me the next 4 events in my calendar, none of which have anything to do with my wife’s birthday.
Results – This one was all over the place, with the Google Phone finding the right answer and Alexa being the most utterly useless. I mean, Alexa was just sad here. Siri’s answer is confusing. Somehow, Siri knows my wife’s name, and that she’s my wife, but can’t locate her in my contacts. How would trying again help? The story might be different if I had my Gmail account set up on the iPad, but for now we’ll just shrug.
Juan Carlos Bagnell
Who was the second captain of the Starship Enterprise?
Siri – “Here’s what I found on the web” – search results.
Cortana – “The third captain of the starship Enterprise is uncertain. If one goes purely by the television series, the first was Captain Jonathan Archer, second was James T Kirk and third was Jean-Luc Picard.”
Google Home – “Sorry, I’m not sure how to help.”
Google Phone – Search results, the second of which was Christopher Pike.
Alexa (Kindle and Echo) – “I can’t find the answer to the question I heard.”
Results – This question was also a mixed bag. Juan reported on his Pixel, he got a detailed description of Christopher Pike. I could reproduce that once on the Nexus 6P, but never again. Cortana’s answer is a little off track, but close. But let’s face it, doesn’t just sum up all of Bing? Alexa, dear Alexa, you’ll find your stride some day.
Can you tell me how to get, how to get to Sesame Street
Siri – Gave directions to Sesame Street, Bensenville. 32 minutes away.
Cortana – Gave directions to Sesame Street, FL (Ed Note: I totally need to go there some day!)
Google Home – “I don’t support directions yet. Sesame Street, Franklin Park is 31 minutes from your location by car in light traffic.”
Google Phone – Directions to Sesame Street, Franklin Park.
Alexa (Kindle and Echo) – “Sorry, I could find the answer to the question I heard.”
Results – Oh Jules, you plucky little podcast producer. This was one of the more interesting mixed bag of results I got. Three different ecosystems sent me to three different Sesame Streets. One of them was a mere 22 hours away. None of them recognized the reference to the children’s show.
So, there you have it. Science! What’s interesting about this in my world, is that of these assistants, Amazon is really pushing the heck out of Alexa. Alexa is starting to be found on connected devices everywhere, from the now-defunct Pebble Core to a slew Smart Home technologies, if CES is any indication. I suppose for now, I should just be thankful that none of my co-workers took advantage of me and made me ask “Order me a new 65” Samsung TV” (because I totally would’ve had to do that – for science).
But overall, this wasn’t supposed to be an Easter Egg hunt – though we did find some. This was meant to show you the disparity between various assistant ecosystems, and I think we ran the gamut well. It certainly lived up to my expectations, even if it is disappointing to show how Alexa just isn’t keeping up in general.
Overall, in my completely unscientific opinion, I see the most accurate answers coming from Google. This should surprise no one, since Google built its empire on finding information. Siri was good in her own right. Cortana had most answers, but in odd ways, which maybe makes her quirky, maybe makes her annoying. Depends on your world view, I guess. Alexa is great for ordering hard to find deodorant, and that’s ok. Everyone needs to find their niche. Regardless, I think we can all agree, that we live in great times right now, and there are four great assistants waiting to help you, however you need, especially if you need to order more paper towels.