Siri report card from analyst Gene Munster indicates distinct improvement over the years

BY Evan Selleck

Published 22 Jul 2014

Siri Now report card

When Apple unveiled the iPhone 4S, they included Siri to make the “S” variant of their popular device stand out. Since then, the digital personal assistant has seen plenty of improvements to make the feature more helpful. A new report wants to show just how much better Siri is doing since 2012.

Analyst Gene Munster of Piper Jaffray has put together a new Siri report card, the results of which were originally published by AppleInsider, showcasing results taken from December, 2012, all the way up to July, 2014. As one can imagine, Siri has improved quite a bit since its days on iOS 6, especially in terms of interpreting questions (when heard correctly), and an overall improvement to Siri’s ability to answer questions in general.

Google Now report card

Munster put Siri to the test side-by-side with Google Now, Google’s own voice-enabled search and digital assistant, to compare the two, and see which one is improving the fastest. He found that while Siri has indeed improved, Google Now was able to barely nudge Siri’s 82% to the side with its own 84% of questions answered correctly. However, Siri wins in several other categories, but more than that shows a stark development from years prior.

Specifically, in December of 2012 the report shows that Siri’s ability to interpret questions heard correctly was at 88%, but in July of 2014 that number has effectively shot up to 96%. Compare that to Google Now, which was at 81% in December of 2012, and managed a score of 93% in July, 2014. In an uncontrolled environment, Siri was at an 84% in December, 2012. The service saw a sharp increase to a score of 93% in July, 2014. Google Now increased from 78% to 88% in the same uncontrolled environment test.

However, it should be noted that while Siri was able to interpret questions more often than not, it still managed to answer some questions incorrectly despite that fact. Munster found that Siri answered questions incorrectly 16% of the time when heard correctly. Google Now, comparatively, managed a score of 14% of incorrect answers when heard correctly.

Other key elements to point out include Apple’s decreasing dependency on Google search results, with responses coming from services like Wikipedia, Microsoft’s Bing, or even Apple’s own Maps app. Additionally, Munster noted Siri’s ability to ascertain answers from more than one source at a time. When he asked about the location of Mount St. Helen, Siri responded with information from not only Wikipedia, but also Apple Maps.

While there is obviously room to improve, from both Siri and Google Now, it’s pretty clear that Apple has put in the requisite work to make the digital personal assistant better by a large margin since the end of 2012. With iOS 8 right around the corner, and Siri getting her own improvements from the updated software, it will be interesting to see how she fares by the end of 2014.

Do you find yourself using Siri often?

[via AppleInsider]