Smart Assistants + Smart Speaker IQ Test Shoot Out For 2018

Loop Ventures recently completed and published the annual Smart Speaker / Assistant IQ Test. Loop Ventures pitted the top four smart assistants and speakers against each other as a part of the new test. This test grades speakers on a wide range of queries and commands and of course the answers provided.

Loop Ventures asked each smart speaker the same 800 questions, and they were graded on two metrics: 

1. Did it understand what was said? 

2. Did it deliver a correct response? The question set, which is designed to comprehensively test a smart speaker’s ability and utility, is broken into 5 categories as such:

Local – Where is the nearest coffee shop?
Commerce – Can you order me more paper towels?
Navigation – How do I get to uptown on the bus?
Information – Who do the Twins play tonight?
Command – Remind me to call Steve at 2 pm today.

The personal assistants Google Assistant, Apple Siri, Amazon Alexa and Cortana were queried  on their respective smart speakers, Amazon Echo, Apple HomePod, Google Home, and Harman Kardon Invoke (Cortana).

Google Assistant was the smartest among the assistants and beat them in almost all these categories, and took the first place in the test. The Google assistant answered 87.9% questions correctly, and understood 100% of the 800 queries. Apple Siri came in second, understanding 99.6% of the queries and answering 74.6% of them correctly. Alexa managed to come in as the third, answering 72.5% of the queries correctly and understanding 99% of the queries. Cortana, also my least favorite among the four, yes I use all the four assistants, due to work I do, came last. 63.4% accuracy.