In late July of this year, it was announced that the company would test four digital assistant services: Amazon Alexa, Microsoft Cortana, Google Assistant, and Apple Siri, by asking 7 different questions.IntelligenceLater, market research firm Loup Ventures used the same method againtestFour digital assistant services, but this time the test environment was changed to four smart speakers: Amazon Echo, Apple HomePod, Google Home Mini, and Harmon Kardon Invoke, which is somewhat different from the previous app-based testing.
Loup Ventures also tested 800 questions across five categories: local information, business consumption, location navigation, information content, and operational instructions. Google Assistant's correct answer rate was over 85%, roughly similar to previous test results. Its voice comprehension rate also reached 100%, while Siri's correct answer rate was 74.6%, a slight decrease from previous figures. This may be because Siri for HomePod required retraining, unlike the previous iOS version, which had already accumulated extensive learning experience. Its voice comprehension accuracy also remained around 99%.
Amazon Alexa's correct response rate reached 72.5%, and its comprehension accuracy rate also reached 99%, a significant improvement compared to previous versions. Microsoft Cortana also saw significant improvements in both correct response and comprehension accuracy.
Google Assistant performed very well in the local information category, followed by Siri. Google Assistant also had a high accuracy rate for business and consumer content, with a significant gap between Siri and Alexa. Cortana also had a significant gap between Siri and Alexa. For navigation, Google Assistant also achieved the best performance, followed by Siri. Alexa and Cortana performed similarly, but still significantly behind Siri.
In terms of information queries, Google Assistant maintained its lead, followed by Alexa. Siri, on the other hand, lagged behind, even performing worse than Cortana. In terms of operation commands, Siri clearly performed better, while Google Assistant lagged behind, but still outperformed both Alexa and Cortana.
