Consumer Electronics

Apple's Siri Leads in Answers That Are 'Simply Wrong'

Paul Ausick

Published: May 31, 2017 9:41 am

Last Updated: January 12, 2020 11:31 am

Personal assistants such as Alexa from Amazon.com Inc. (NASDAQ: AMZN) or Google Assistant from Alphabet Inc. (NASDAQ: GOOGL) get a lot of attention as a new market for consumer electronics devices that are going to change the way we live. That may be worth looking forward to if the devices worked as advertised.

And it’s not just a problem for the digital assistant devices like Amazon’s Echo or Google Home. Personal assistant applications like Apple Inc.’s (NASDAQ: AAPL) Siri and Microsoft Corp.’s (Cortana) are not a lot better.

This is important because 60% of digital assistant and voice-controlled devices ask general questions of the personal assistants. Nearly as many — 57% — ask about the weather and more than half — 54% — use speech to control music streaming.

Digital marketing agency Stone Temple collected a set of 5,000 different questions related to everyday factual knowledge and asked those questions to the personal assistants and noted how the various assistants answered.

Here’s how the Alexa, Siri, Cortana, and Google Assistant fared. The first percentage is the number of questions answered and the second is the percentage of answers that were both complete and correct.

• Google Assistant: 68.1% answered; 90.6% complete and correct
• Cortana: 56.5% answered; 81.9% complete and correct
• Siri: 21.7% answered; 62.2% complete and correct
• Alexa: 20.7% answered; 87% complete and correct

As a kind of baseline, Stone Temple also tested Google Search, which answered 74.3% of the questions with responses that were complete and correct 97.4% of the time.

And when it came to answers that were simply wrong, Apple’s Siri was the runaway winner (loser?), getting the answer wrong nearly 3% of the time.

Stone Temple concluded:

Google still has the clear lead in terms of overall smarts with both Google search and the Google Assistant on Google Home. Cortana is pressing quite hard to close the gap, and has made great strides in the last three years. Alexa and Siri both face the limitation of not being able to leverage a full crawl of the web to supplement their knowledge bases.

The full report is available at the Stone Temple website.