Every since Apple unveiled Siri, they tried to make us believe it was this amazing product that could never be wrong. Steve Jobs led us to believe it was as good if not better than Google search, and many people have attempted to prove or disprove this on a smaller scale. We really needed a more comprehensive test. That test has been conducted and things don’t look good for Siri.
Before you go thinking that an Android fanboy did the test, it was Piper Jaffrey’s Gene Munster. Gene is an Apple fanboy. In fact I blasted him in a recent post about his survey that showed developers favor Apple. He put together 1600 questions and asked both Siri and Google. He also went a step further and asked 800 of the questions in a quiet environment and the other 800 on the busy streets of Minneapolis. Hit the break for the results.
Since he didn’t use Google’s Voice Search, all questions were typed in for Google. It’s not surprising that Google understood 100% of the questions. On the other hand Siri was all voice and she understood the questions 83% of the time in noisy conditions and 89% of the time in a quiet environment. As to the results, Google was able to answer the questions accurately 86% of the time while Siri was able to get it right 62% of the time on the street and 68% of the time in a quiet room. Munster graded Google with a B+ and Siri with a D.
“In order to become a viable mobile search alternative,” Munster writes, “Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher.”
According to Wiki, Siri gets 60% of its answers from Google, 20% from Yelp, 14% from WolframAlpha, 4% from Yahoo and 2% from Wikipedia. Here’s what Munster found:
“Breaking down Siri’s reliance further,” Munster writes, “Google provides 100% of navigation results, 61% of information results, 48% of commerce results and 42% of local results. Among other result aggregators, Yelp provided the most local results (51%) and commerce results (51%), while WolframAlpha provided 34% of information results.”
Siri is promised to be better in iOS 6, but that remains to be seen. Not only that, Apple is continuing to get away from Google. For example, with iOS 6, they will use their own Maps data and they will start to rely on Yahoo Sports for sports scores, Open Table for restaurant reservations, Rotten Tomatoes for movie show times, and Fandango for movie ticket purchases. Will this pay off for Apple or will the lack of accuracy remain?
Here are some of the questions Munster asked and how Siri handled them
- What team does Peyton Manning play for? Responded with the answer to the previous query. This was the most common error.
- Where is Elvis buried? Responded I can’t answer that for you. It thought the persons name was Elvis Buried.
- Where am I? Pin dropped in the wrong place.
- When did the movie Cinderella come out? Responded with a movie theater search on Yelp.
- How do I get from Boston to New York? Responded I can only give directions from your current location. I cant give you directions to a place you are not in.
- What spices are in Lasagna? Responded with a Yelp search with lasagna on the menu.
- When is the next Haley’s comment? Responded “You have no meetings matching Haley’s”
- I want to go to Lake Superior? Responded with directions to the company Lake Superior X-Ray.
I do find it interesting that Munster didn’t use Google’s Voice Search in this test, but I guess he really wanted to find out if Apple’s search queries were up to par with Google’s rather than see who can handle “voice” better.
source: fortune
Mobile interface without disturbing the main site. A seasoned developer and designer, too, can help with a redirect if you are using a customized content management system.
ReplyDeletechicken coops