Just how accurate is Siri, Apple's virtual and voice-activated assistant, anyhow?
We suppose you could just ask it. That almost feels like cheating, but it's exactly what Piper Jaffray analyst Gene Munster did in a quasi-scientific survey of just how well Siri performs against a common Google search.
The results? Siri replied accurately to Munster's tests anywhere from 62 to 68 per cent of the time, given the conditions in which Munster was asking his questions (we'll get to that in a bit). As for Google, the search engine was able to answer his (typed-in) queries 86 per cent of the time.
"In order to become a viable mobile search alternative," wrote Munster in a research note published Friday, "Siri must match or surpass Google's accuracy of B+ and move from a grade D to a B or higher."
Here's how Munster ran his tests: He asked Siri a series of 1,600 questions, split between 800 asked in a quiet room and 800 asked in a more rambunctious, everyday environment (the streets of Minneapolis).
In total, Siri understood approximately 89 per cent of his questions in the quiet room and 83 per cent of his questions in the louder environment – not bad, especially since the "control" group consisted of typing questions into Google (which Google understood 100 per cent of the time, go figure).
"Apple right now gets a 'B' in comprehension and a 'D' in accuracy," Munster wrote. "There's a big difference between comprehension and her actually doing what you want her to do."
That said, Munster believes that Apple will be able to increase Siri's overall accuracy once iOS 6 goes live later this year. Paradoxically, Apple is expected to simultaneously reduce its reliance on Google-generated results from approximately 60 per cent to 48 per cent, Munster said.
"Breaking down Siri's reliance further," Munster wrote, "Google provides 100% of navigation results, 61% of information results, 48% of commerce results and 42% of local results. Among other result aggregators, Yelp provided the most local results (51%) and commerce results (51%), while WolframAlpha provided 34% of information results."
According to Munster, Siri is roughly two years behind Google for delivering accurate responses to asked questions.