2018's most improved smart speaker: Apple's HomePod

Google’s Home is still the brainiest, says Gene Munster, understanding every one of his 800 questions and answering nearly 88% correctly.

From Loup Ventures’ Annual Smart Speaker IQ Test:

Over a 12-month period, Google Home improved by 7 percentage points, Echo by 9 points, Siri by 22 points, and Cortana by 7 points in terms of questions answered correctly. We continue to be impressed with the speed at which this technology is making meaningful improvement. The charts below show the results of our past tests. Sequential drops in the navigation or commerce categories were caused by changes made to our questions set to reflect changing abilities and ensure our test is exhaustive.

Click to enlarge.

Aside from HomePod’s 22 point increase due to the enabling of more domains in the past year, Alexa had the most noticeable improvement. The largest advancement came in the Information section, where Alexa was much more capable with follow-on questions and providing things like stock quotes without having to enable a skill. We also believe we may be seeing the early effects of the new Alexa Answers program which allows humans to crowdsource answers to questions that Alexa currently doesn’t have answers to. For example, this round, Alexa correctly answered, “who did Thomas Jefferson have an affair with?”

My take: A really smart speaker might politely point out that it’s “whom did Thomas Jefferson have an affair with,” not “who.”

See also:

• Apple’s HomePods ‘at the bottom of the totem pole’
• What’s right and wrong with HomePod
• What the analysts are saying about Apple’s HomePod