Worth mentioning is Yelp's long (alleged) history of using review filtering as a de facto racket to 'encourage' businesses to sign up for their paid services.
A company I used to freelance for found that after they stopped paying Yelp for a premium account, they started seeing more negative reviews and a number of their positive reviews were filtered out. They got almost daily phone calls from Yelp marketing types essentially suggesting that if they renewed their subscription, the negative reviews would 'disappear'.
This was a few years ago, so maybe things have changed since then. Either way, I wouldn't hold your breath for a better review verification system.
This happened to a driving instructor I know just 2 weeks ago. He had a 4 star average rating on Yelp, receives a call from Yelp sales pitching some kind of a premium service. He declined. Within a week most of his positive reviews were suddenly filtered out dramatically reducing his score.
I use Yelp mostly for its metadata, such as restaurants' '$/$$/$$$' price categories, or a quick, easy-to-read menu sampling. I give the reviews approximately zero weight at this point in Yelp's evolution.
Especially complicated when you realize that their mapping system is completely broken for numerous addresses. I used Yelp to try to find a good local cafe to eat at in Irvine. Drove right to the spot indicated on the map and found an industrial complex. Had to look up the business name on Google Maps to find that it was about five miles South of where Yelp told me the cafe was.
Same thing happened again in Vegas, looking for a nice place to eat, find something that looks interesting and then see that Yelp places it in an industrial complex nearby. This time I don't expect to find the place, but I drive by and find myself somewhere in a dark alley full of closed-for-the-evening auto shops.
I'll still use Yelp to find the name of a business, but then I have to switch to Google Maps and do a reality check to actually see if the business is near by and how to get there. A real PITA.
That's the only thing I use them for. It's more convenient to search for "bar" and see a list of actual boozeries, vs. searching Google Maps and finding metal manufacturers, and various businesses that happen to have "bar" in their name.
This is supposedly happening all the time, yet no one has managed to catch it on tape for some reason. I can't think of a credible explanation for the lack of hard evidence (and lawsuits).
"Worth mentioning is Yelp's long (alleged) history of using review filtering as a de facto racket to 'encourage' businesses to sign up for their paid services."
EXTREMELY worth mentioning. If you've yelped Yelp, so to speak, you know to take their recommendations with a grain of salt and then some. Caveat emptor.
While I'm on the topic, the idea of a single company managing reputation for any given space is silly beyond words. Downright moronic, even.
Reputation is an aggregation of multiple opinions. No aggregating entity can aggregate without their aggregating principle spinning the results. Yelp is notorious for a corrupt and indeed outright criminal aggregating principle -- "we'll suppress bad reviews if you pay us and good reviews if you don't" -- but the same is true for any aggregator, whether they're aggregating based on good principles or bad, because aggregation is a form of authorship. I mean look at Hacker News vs Reddit vs Digg vs Slashdot. They all aggregate, and each has a distinctive voice.
Because of this, yes, Yelp is a useful resource, but no, don't use it for a serious purchase without going on Google. If you use Google and Yelp, you're at least polling multiple sources, so you're doing something which can access the fundamental aggregating dynamic which powers reputation in the first place. A single point of failure for reputation is a contradiction in terms!
That made me think of an interesting idea. No problem is not solvable with some level of indirection!
A review aggregator website where the value of reviews from various sources is weighted by how reliable they are. For example, if there is Yelp, Facebook and Google, it can weight a restaurant 0.7 Yelp, 0.2 Facebook and 0.1 Google.
we do. if you use Google as well as Yelp, every other site on the Internet which mentions the business you are curious about effectively acts as a separate review site.
It's not worth mentioning. Almost every case has been a coincidence and they've gone out of their way to explain the review filter (which functions similarly to Google's PageRank algorithm and attempts to filter out spammy reviews using various factors).
Each court case related to this has been thrown out.
If you trust Google to be fair with PageRank then there hasn't been any evidence that should make you more concerned about Yelp.
Business owners complain about getting demoted in Google's PageRank algorithm quite a bit, too.
I don't recall ever hearing people receiving calls from Google salespeople who imply that a failure to buy advertising on Google may result in a reduction in PageRank, or that purchasing advertising may restore it.
> Business owners complain about getting demoted in Google's PageRank algorithm quite a bit, too.
Google rankings fluctuate over time, but it is never the result of a direct targeting of an individual company or URL by google, whereas there is ample evidence that Yelp directly targets individual businesses depending on whether or not they are paying customers.
I don't know if he's just being coy by saying this is an accident on Yelp's part.
The actual reviews are here: http://www.yelp.com/biz/royal-transportation-moving-and-stor.... It's pretty obvious there is something fishy about the unfiltered reviews. Each reviewer first left a one line review "Amazing!!! one of a kind..one of the best sushi place" and "the best of the best" before their very detailed, glowing review of the movers. If that's what it takes to fool the spam filter then something is very wrong. By contrast one of the filtered reviewers has ten measured, thorough reviews but his review has been banned for ToS violations.
Yelp really needs to disclose which vendors have a business relationship with them. They should also make the filtered reviews easier to see (place the link next to the unfiltered review count) and remove the damn CAPTCHA. I regularly try to check the filtered reviews but can't because the CAPTCHA is too difficult to read.
really really really makes me suspicious. Just did a check to make sure that google does not integrate yelp reviews (it does not) since that is what I mostly use.
also worth noting that "bob" liked the company so much he cross-posted his review to google.
Rather interesting to see his google review history, as long as we are speculating . . .
Gotcha. Damn, I see what you mean. If that's all it takes to game the system, it's amazing that their system isn't completely overwhelmed by fake reviews...
That's literally jawdropping for me. I would have thought it a fantastic business... and then to find that all the filtered reviews are 1 star... unreal.
They found that there were two patterns of content contribution: people who wrote one review and people who wrote lots. For people who wrote one review, anonymous contributions were higher quality than signed, and those single reviews were higher quality than people who contributed multiple reviews.
Chances are good your most valuable reviews are coming from the long tail and your least accurate reviews are coming from people who are posting lots, but potentially with sockpuppet accounts. Tossing out your most accurate reviews in an attempt to suppress the latter seems counterproductive.
Yelp is the poster child of the disingenuous Internet problem.
In a nutshell the problem is that people who are not verifiable by "the service" provide information or data which is essential to the operation of the "the service." Further the beneficiaries of the service (customers) are not paying for the service, and the people who can be harmed or benefit from the service (establishments) have every motivation to game the system to increase business and little downside.
So your business which has modestly thin margins to begin with, grows slowly, and the more popular it gets the more it tries to get gamed and the more margins are squeezed if you're paying people to filter those reviews.
There aren't really any 'good' answers. But there are other examples of businesses like this that have done well. The ones that do well however use the 'review' portal to push more customers to the 'value' area where higher rates of returns are to be had.
I hope someone writes up Yelp, Ripoffreport, TripAdvisor, Expedia, and AngiesList up for a Harvard Business Review group study. There have been a lot of experiments in this space taking different tacks and it would be interesting to compare their effectiveness.
I swear by AngiesList for services like moving companies.
They just called us up a couple weeks ago and sat on the phone for over a half hour (because we were willing) taking reviews from us on a bunch of companies that we had used.
This is why it costs money; they actually provide a service. If they don't have enough reviews, they hire people to call you up and write up the reviews for you.
It's a pretty expensive undertaking. You have to be a subscriber (at non-trivial cost) to be a member/reviewer. The vendor would have to create multiple puppet accounts at $20/pop to shill up reviews. But, what of all the other actual reviewers that will write bad ones? You tend to see patterns in the reviews and, when a shining review of a vendor leaves me skeptical, I always look at the members' other reviews to see if they left at least 2 or more. If they did, then I suspect less that they're in cahoots with the one vendor.
If you're always double checking reviewers, it's easy to do the same thing on Yelp, or anywhere else for that matter. Most people don't, which is why Yelp filters and Amazon ranks.
I don't defend their decision to advertise with Rush Limbaugh. He said some pretty unforgivable things, and frankly I was surprised when they returned to advertising for him.
But they provide a good service, and they also support NPR. I am under no illusion that every company I do business with will agree with me politically in all ways.
Disclaimer: I don't work for Yelp but I have created other user-ratings-based sites before.
It's not entirely fair to criticize Yelp for under-emphasizing negative reviews.
One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting.
You work harder to get positive reviews than negative ones. If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied.
This should be the motivation for preferencing good reviews over lots of one-off bad reviews.
BUT: Yelp made the cardinal sin of "objective" ratings sites: their profit comes from the entities they are rating. Ergo everything on their site is questionable.
> One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting.
This is the risk we take listening to ANYTHING depending on voluntary responses. Burying negative reviews to attempt some intrinsic rating dismisses the value in actual user input, with the paternalistic view that the review aggregator somehow knows better. Who decides which and how many negative reviews to hide? You? How about letting the user figure it out, knowing that trusting any opinion is a risk and learning from experience to calibrate reviews he reads to himself? It's the best we can hope for without resorting to the review site, in effect, clobbering consensus views to get something it "knows to be more accurate." In that case the users no longer review, but the site, which is not a user, does.
> If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied.
You absolutely do not know this. It could be that 70% were dissatisfied. You can't assume this without taking the role of a reviewer, which you cannot legitimately do without experiencing the item being reviewed.
I'm not sure I can agree with that. Negative reviews are almost always far more helpful than positive reviews. Review sites should be careful about referencing positive reviews. The gold standard is Consumers Reports which is much more balanced than the average interent review site (where everything gets 4 stars) and thus much more helpful.
Speaking as an app developer, sometimes people get furious over things that are clearly stated before they download it. It really is easier to get complaints, I can see usage stats that show many more people using my apps daily.
So I read the complaint, and that thing that was clearly stated before is now reiterated. If it doesn't bother me, I go ahead and buy your app.
Good reviews are worthless to me as a consumer. What I want are details, good and/or bad, that I can use to evaluate my purchase ahead of time. "This is great!!!", "I would use it again!" are devoid of information.
If negative reviews are more informative, then I assure you, you should welcome them over a bunch of five-star "great!"s.
Besides being informative for me, the user, they should be valuable to you, the business owner. Nobody maintains an "attaboy" database, but you better keep track of bugs and fix them if you expect to grow or even survive in this economy.
I do agree that a lot of valid reviews are being thrown out with the bathwater. There should be a better mechanism to show all unfiltered reviews.
The thing that Yelp that makes it stand above other review sites is that they incentive positive reviews. Like Yelp or not, the idea of "Yelp Elite" is actually a great idea, because it is able to draw out a lot of medium to great reviews, stuff you would rarely see on other sites. When you incentive reviews, either good or bad, then you do tend to eliminate this "one-and-done" bias that tends to focus on the negative.
That being said, my wife is/was a Yelp Elite for several years with over 800 reviews, but quit this year because she thinks that Yelp as a whole has started to go downhill, and doesn't like the pretentiousness of most people on there now.
That's absurd. Google's not going to see the filtered reviews unless they start doing a lot more JavaScript execution than they've been doing, and why would Yelp care if a browser extension or spammers see filtered reviews?
Google's not going to see them because they have purposefully made them difficult to scrape...
I'd imagine Yelp would rather not have a "Yelp Unfiltered" extension made and their filtering becomes less important. Same for helping spammers automate their processes by checking up on their reviews.
It's a race to the bottom though, the spammers will eventually win.
If this bias occurs across all businesses on Yelp, then still, wouldn't some places will be rated higher than others (even if "higher rated" comes to mean "has fewer bad reviews")?
Like you say though, Yelp is just making the playing field uneven in favor of those who pay to play, so none of this matters anyway.
"people are most motivated to write reviews when they are outraged. ...your site runs the risk of becoming a soap box for ranting."
There's a great big gap between letting all your angry customers (which hopefully you don't have a lot of) leave rants and creating sockpuppets so you have nothing but great reviews. The space between is the basics of word-of-mouth marketing: ask your happiest customers to talk about you.
No one is stopping American Eagle Movers from asking their happy customers to head on over to Yelp and leave them a review. They just don't seem to have any happy customers. So they have to fake it. And then Yelp steps in with their [seemingly] shady selection practices. Hence Justin's disastrous experience.
People are way more willing to take the time to write a bad review if they feel that something wrong was done to them than take the same time to leave a positive review, even if you ask them to.
The only one that will leave a good review will be Yelp users that rates everything.
And maybe people that you did them a big favor and they feel they have to repay you someway.
Maybe a shamless plug, but I work on mobile apps at Fodor's Travel. We've been doing print travel guides for 75 years. We now have apps, in addition to books.
We're professional curators. That means every restaurant review (and shops, hotels, etc) has been written by an actual human professional who's opinion we hope you trust (we're a little more upscale than Lonely Planet, but we have our hip moments). So in a way, we're the anti-UGC. When you have crowds of people, all the ratings become a noisey 3-4 stars. Including Burger King. That's why I think Fodor's content is a good alternative. We're in the travel space, not local, but, it's a refreshing alternative.
Places have good days and bad days (or even hours). Your reviewers could get made. While the reviews may be spot on, they are a one point in time random sample.
The other problem is that not everyone is the same. For example I absolutely hate "attentive" staff in a restaurant (let me eat in peace and if I want something I'll ask for it), but want the check as quickly as possible on finishing while others like to slowly wind down. A regular reviewer probably won't mention either of these because that is normal to them.
The existence of multiple reviewers means one will be more like me and would mention these issues which would make me go somewhere else while others who enjoy that kind of experience would be more likely to come. This doesn't make anyone right or wrong, but does show how different reviewers help and why the lowest review scores have the most informative reviews.
As a counterpoint, I don't like the same movies Roger Ebert likes. But his opinions are consistent, and I know well enough where we are different, that I can usually estimate from the text of his reviews how well I'd like a given movie.
Reviewers may prefer attentive staff, but the good reviewers (which exist) know that not everyone shares their preferences. That's the difference between a good review and a bad review: you can translate a good review into your own viewpoint and decide for yourself.
When I think of noisy one star reviews, I mostly think of the one star review for a microprocessor where the guy didn't use a heatsink and the motherboard melted. There are plenty of folk who come across well in writing but are monstrous in person, I'd never expect to trust their restaurant reviews, and I might not be able to identify them either.
The question is, are you going to find that one reviewer who is like you in the sea of 4-star-giving morons?
I love reading Ebert's movie reviews. I often disagree with him, but he is a mostly fixed point. I can tell if I'm going to like something because, over time, I have developed an understanding of where his tastes and mine align and where they differ. This kind of thing is a lot harder to do online—it requires real work and attention.
In a way, we're disagreeing about where we want to have our statistical anomalies. Do we want the reliable reviewer caught on a bad day, or perhaps recognized by the staff? Or do we want the sea of one-time reviewers, a few of whom had a statistically unlikely bad or amazing experience?
I just want to say, my mother has been using Fodor's Forums for years now to help her with planning her trips. So thank you for creating such a useful resource!
Huh, interesting, and I'm surprised I hadn't heard of Fodor's. I work at http://www.oyster.com/ -- we're kind of the anti-TripAdvisor as all hotels are visited and photographed by our pro photographers and reviewers. We've found the same thing with UGC in the hotel space. For the popular hotels on TripAdvisor, there are 1000s of reviews, and the aggregate rating is always 3-4 stars. In contrast, on Oyster.com there's one review written with the same guidelines as all the other hotel reviews (as well as 100s of high-quality, non-staged photos).
Man, I'm 30 and this makes me feel old. :) In the print travel guides market, it's one of the big names. It comes from the generation before the more backpacker-oriented guides (Lonely Planet, Rough Guides, etc.) and targets more of an upper-middle-class family vacation type of audience, but it's still fairly popular, within the print-travel-guides space.
It does seem that their move into digital guides is quite recent and incomplete, though.
Ya, I get that a lot.... I guess with less bookstores around, we have a lot less notoriety. You're right, high number of crowd ratings regress to some mediocre mean, less absolute dump 1 stars, less stellar 5 stars, i guess.
But not in the "back to square one" meaning, it is rather "next level of a spiral staircase". I believe there's still room for improvement (definitely there is over the '95 low tech and probably over what we have now) in the way we can combine internet and the '95 solution to find good <mover / restaurant / hairdresser ... etc>
Time will tell - lots of money goes into social, someone will do it right or at least we'll know what obstacles prevent this from happening.
This point doesn't apply very well to this case. The problem is that there was a lack of verifiable people on Yelp posting negative reviews. Yelp has many more semi-verifiable people on it (i. e. people who've left multiple reviews) than the typical person has friends with experience with a given rarely-used service.
If in this case the problem were likely to be solved by acquaintances, it would also be likely to be solved by Yelp, under its assumption that people need to leave multiple reviews to be considered credible.
No need for the 1995 solution, Yelp is just doing a very bad job at detecting bad reviews. It's usually rule #1 that in any type of "bad" user filtering that many people that are good users may not engage your site.
Q: Internet solution doesn't work. How do we fix it?
A: Use the 1995 solution but because it's not 1995 anymore you'll use Facebook and Twitter to ask your friends which is still using "the Internet".
Over and over and over again I read negative articles about Yelp. These kind of places are ripe for scammers. These businesses know that you're randomly searching for a service, don't know you and don't plan on ever seeing you again. Yelp's system is, apparently, very easy to game. They've been accused of extortion. And yet, billion dollar valuation. It's laughable.
We're so out of touch that people are talking about how this space is ripe for disruption and a new startup. Umm, Yelp is an extremely young company. I guess once you're public you're seen as old hat? Pump and dump, and move on to the next thing. Unbelievable.
The hiding of reviews has made Yelp virtually useless. I like the identity verification idea. I implemented the Equifax version and it's very simple to code. I believe it cost us in the neighborhood of $1/verification. That may outweigh the value of a verified Yelp user though, but I have no idea of the Yelp economics.
I have a friend that works as a veterinarian. Yelp called his business often to try to get an advertising deal. After he declined, many of his 5 star reviews disappeared from the site. After losing a lot of business, he signed up for the advertising. The next day all of the 5 star reviews returns. This was just a few months ago.
Then he should sue. Every case brought against Yelp has been thrown out so far. I'd imagine that if this racket stuff happened, at least -one- case would come to fruition.
The proposed solution sounds incredibly easy to game - as a moving company, I could create a bunch of fake, one-off profiles with complaints about one of my competitors and destroy their Yelp reputation. Unfortunately, I don't think it's possible for an algorithm to detect the difference between a bunch of angry one-time posters and a coordinated reputation attack.
This is why I've gone off sites like Yelp for recommendations, and instead stick to places like Foursquare which rely much more heavily on the behaviour of the people that I know. Seeing that several of my friends have been somewhere more than once is a much stronger recommendation signal than a bunch of strangers giving something five stars.
You need a valid, fully qualified, unique credit history for each new account. How is that easy to game? Are you assuming that the moving company will get all their employees to to signup to Yelp and certify their credit history against fake sock puppet accounts? They maybe able to do it for a few accounts but pretty soon they would run out of real people to use up... in the mean time real reviews will be submitted by an endless pool of real people with valid credit history.
It will be quite easy to tell which reviews are real and which are not. At the very least it will make the company look questionable because 50% of reviews will be 5 star and 50% will be one star. The main issue is we need to get valid reviews out of the filter so that Yelp users can more easily make their mind up.
As it stands a lot of real reviews are hidden and a lot of fake reviews are exposed. Simply by exposing real reviews you level the playing field.
Say what? Yelp is pulling credit histories of people who join the site? If they are, that is way out of bounds for what users typically expect when joining an online review site. As a simple reviewer, I'm not engaging in any business with them, they're not offering me credit, and pulling a credit report on some anonymous internet user would not only be expensive but would require some serious disclosure (as having credit reports pulled actually impacts one's credit score). Not only that, but if they are doing it, it doesn't seem to be working very well.
Besides, I never supplied sufficient information for Yelp to connect me to my credit report without doing some seriously ethically questionable data mining.
Why do you believe that Yelp does this? If so, that would be more surprising to most users than the pay-for-play behavior we're seeing here.
Did you read the article before commenting? The article proposes that Yelp should provide people the option to use their credit rating to prove they are not a sockpuppet in return for Yelp displaying their review. It is only an option, credit ratings are only used because they are a pre-existing identification and authentication source and most importantly YELP IS NOT DOING THIS YET.
As a matter of fact I did, and you're right: I mis-read the parent comment as present tense vs. potential future tense. My apologies.
That said, I think some of the problems with the credit report solution still stand; namely that pulling credit histories has a cost (both to the reviewer and to Yelp). I would worry about what they would do with that information beyond merely authenticating me as a "real person" and I would expect that fewer people would be willing to review a local donut shop if doing so required a credit pull.
It would be interesting to see a crowd-sourced recommendation site based only on positive recommendations. Something like a Net Promoter score to tell me where is good. i.e. you can only leave feedback for a place if you would actually recommend it to someone.
Every business can have bad days, and a negative review can put a prospective customer off even if actually 99% of the time that's a good business.
This isn't really a comparative recommendation site like yelp, the scenario would be something like a user saying what they want to do (like eat burgers on friday) then people would only recommend place to go. Kind of like Quora for places.
Eventually you could build a decent picture of business that are recommended multiple times to start making suggestions beyond the crowd sourced suggestions.
Personally, I use the negative reviews much more than the positives. Especially on business reviews on Angie's List or book reviews on Amazon, I get a lot more info from the negatives. The company itself is usually able to tell me the positives. I don't let a few negatives rule out my business, but it allows me to compare more accurately between alternatives.
If I'm in planning mode and I don't know where I should go eat burgers on Friday, I don't want to know where _not_ to go, I want to know where I should go.
Of course, if I want to look a specific place or product, pos and neg reviews are useful.
But if you're in a "I'm moving across the country" mode, a negative review of a moving company may be worth much more than a positive review. (Possibly $2,000 more, in this case.)
FWIW I watched the video (all the women have moustaches!) but am no more informed as to how it works. I also looked at the business sign up page but couldn't find any thing there to help work out the mechanics of the process. It looks like you integrate with FB closely?
Thanks for your feedback! I have gotten so little. :)
I was unsure of the etiquette here at HN about pimping one's own site in the body of a post. I saw that others chose to put their site URLs in their profiles, but very few were posting them in comment text.
As 'iamgilesbowkett mentions above, maybe it would be more informative to search over an aggregation of different opinions rather than hash mountains of conflicting reviews into a single byte digest.
For the consumer, our site offers a way to casually discover nearby businesses based on how well they are liked by specific circles of people. All activity is contextualized by location; the target of attention is geo-tagged by lat-long, not the user.
We try to reduce the effort required to browse through large amounts of business listing and business review data. The user should be able to quickly explore a specific area for businesses that match a search term, or are physically close to another (perhaps unrelated) business.
Please feel free to surf around using the business search.
On the site, business search itself is open to the world, but the core social parts require an account to participate. You can enter an object term plus a location. Search URLs can look like this: http://spotlikes.com/discover/places/noisebridge/san-francis...
Due to the founders' interest in Napa and Sonoma counties, we have lots of comments describing San Francisco North Bay Area small businesses. One entry point for browsing might be someplace like: http://spotlikes.com/business/benziger-family-winery-glen-el...
During after-release QA, sometimes I try to do a kind of major metropolitan area hopscotch, visiting all the largest US cities using the fewest clicks.
For the SMB owner, our site offers businesses a place to make a simple listing - a page where they can post their basic details (phone number, URL, logo, etc) - and then sit back and watch the users stream that small referral link out in their activities.
Yes, the site is tightly integrated with Facebook. User comments on business listings can flow automatically into social media sites. If you authorize it, your "likes" will flow onto your FB wall/timeline. We have similar Twitter and Foursquare integration.
We have probably overlooked all kinds of stuff. We move pretty fast. If you would like to call out anything in particular, I am keenly interested in the feedback.
Personally I just Yelp would have some sort of collaborative filtering engine. They kind of have a social one, but my friends' opinions are often as bad as a random stranger's.
What people review on Yelp is subjective and Yelp makes it hard to filter out people's biases because of their one dimensional rating system. When people review a restaurant, they review the service, the food, the atmosphere and the overall value. For some people, paying $200 on a meal is outrageous regardless of how epic the food might be. For others, they don't even look at the check when they pay and value doesn't factor into their ratings.
Yelp doesn't ask you to rank the businesses you like on a scale of best to worst, so people rate things based on some internal weighting scale that probably shifts slightly from review to review. I may have a 5-star taco trunk right next to a 5-star review for the French Laundry. They aren't equal.
What people review is also temporal. God help a business if there is say, some construction going on across the street, the patron was in a bad mood and becomes argumentative or the chef is sick and food quality dips for a night. Or maybe the waiter just broke up with his girlfriend. Maybe they run out of the one thing the patron really wants to eat. Any of these things can make someone walk away and feel the business is poor.
Businesses and people also change over time, I have reviews on Yelp for 5 stars from 5 years ago for places that have either gone downhill or simply don't match my current taste.
People also tend to only review things they either really liked/loved or hated which results in most businesses (especially in SF) as being rated 3.5-4 stars making Yelp pretty useless without looking at the underlying reviews.
A collaborative filtering engine could fix a lot of these problems. At the very least I'll match people who have my own biases and normalize people's ratings. Of course it still doesn't fix the temporal issues, but I don't think that's insurmountable.
That's an interesting idea. Is the main motivation of your suggestion to find a way to enable you to better discover things you might like without relying purely on reviews and ratings? In general, I agree that ratings and reviews have inherent biases that are hard to smooth out. There are so many more dimensions to it, such as how much weight you apply to a review written by someone that is trying a certain type of food for the first time.
Do you think that you would see better results if Yelp was able to use a collaborative filtering approach to somehow personalize their results such that they more closely match your tastes and preferences - based on other users' data?
What's most shocking at this point is that this story has been up for seven hours and Yelp hasn't bothered to fix the page in question, the wrongness of which could be observed by any human with eyes: http://www.yelp.com/biz/royal-transportation-moving-and-stor...
Wow, those 1-star reviews are pretty scathing too. I had no idea that negative reviews were filtered like that; that pretty much ruins my trust in the review component of the site. From now on, I'll be using Yelp for contact info. and price points...
For the record: the sales organization at Yelp has a read-only copy of business data in Salesforce. Salespeople are not able to fiddle with reviews in the main database. They're two completely different systems. There's no possible way a salesperson can hide/show reviews to try and sell advertising. The salesperson can _claim_ to be able to do it, but they're lying.
Any review on Yelp that says "We" or mentions the name of the person that assisted them at company is most certainly fake.
Example:
"We were really worried about where to stay on Las Vegas, but then we called hotel X and Margo sounded so nice and welcoming on the phone that we just knew it was the right choice."
Question based identity verification can be very painless.
For you, but not the vendor. See PayPal.
Question based identity verification processes are already used and supplied under the hood by agencies such as TransUnion, Equifax & Experian.
And that leads to friction. Yelp got it's massive user base from frictionless technology.
I don't like Yelp's business practices just as much as the next person, but this is a human interaction issue that isn't solved by a simple algorithm, nor is it solved purely by (mind you, costly) human being based committees.
This isn't really Yelp's fault. Sites like this always have fake reviews mingled in with those that are real. The solution?
Assume people with a 4-5 star review are competent. GET MULTIPLE QUOTES. Doesn't matter if they have 5*, if you feel something is off with them they you avoid.
You also will find out when a quote is too cheap/too expensive and reduce your risk to getting conned.
Don't blame the internets or Yelp. Blame youself for not doing due diligence.
Is anyone surprised that a recommendation site that gets its money from the businesses is less than honest? Is anyone surprised that when they get popular they turn into a protection racket?
Say it with me. "If you're not paying, you're the product." Or in this case, the club.
Wow. Surely Yelp can pick up on a continuous, consistent pattern of negative reviews.
The hard part is staying one step ahead of less-legitimate accounts. There's always somebody trying to cheat the system, and stopping these users often results in a tradeoff.
> Surely Yelp can pick up on a continuous, consistent pattern of negative reviews.
But a continuous, consistent pattern of one-shot negative reviews is also not particularly difficult for a competing business (or anyone with a grudge) to reproduce.
This is a hard problem, and I don't think the credit check-style identity verification questions suggested in the post are going to solve it. For one thing, I think an overwhelming number of people would find it creepy. They're trying to tell people about a bad moving company, not get a car loan.
Putting more of your "valuable" online identity at stake seems likely to help. I hate to say it, but "Log in using your Facebook account" suddenly gives Yelp a lot more data to exploit to decide just how much trust they should put in your reviews.
I agree it's easy to reproduce. Not an easy problem by any measure.
Putting in my credit card (or revealing any information I don't deem relevant to the company / form) is always an immediate deal breaker. For Yelp, a critical mass of reviews is needed, I'm not sure they can afford that.
Why would they find it creepy? You wouldn't need to say your using credit history. It would be as simple as asking the user a question "Please validate your a real person by answering these few questions".
As far as the user is concerned it's just a few questions and answers.
Have you seen the questions they use? They range from "which of these addresses have you previously lived at" to "which of these companies do you have a loan with" to "how much is your loan payment" among others. They're pretty creepy and made even more so by the fact that these are the same questions you answer to prove your identity to, say, the credit bureaus or your creditors. It's not the sort of stuff one should be answering lightly.
> Why would they find it creepy? You wouldn't need to say your using credit history. It would be as simple as asking the user a question "Please validate your a real person by answering these few questions".
As far as the user is concerned it's just a few questions and answers.
If the questions were about my credit history then this is a deal breaker.
There is no way a site like yelp can get me to authenticate by answering questions about my credit history and I suspect that the vast majority of people would agree.
Now, facebook credentials, that's a different story.
As far as the user is concerned it’s just a few questions and answers.
Just a few questions and answers that could hint at all kinds of things including relationship status, orientation, race, financial status, etc. Imagine if some stranger on the street, or even the cashier at a trendy retail store, asks you about some detail they have no business knowing. You would find it creepy. Now imagine it's some random web site (which is all Yelp is to most people).
"just a few questions and answers" that I would be, frankly, shocked, that Yelp knows out of nowhere without the explanation that they're hitting my credit file... and then with that explanation, it's equally creepy that they want to hit my credit file for me to leave a review of a pizza place on the Internet.
The problem is, a continuous, consistent pattern of negative reviews can just as easily come from someone who has it out for a business.
I have a friend who is currently a target of such a vendetta. He got into an edit dispute with someone who was trying to censor stuff on a wiki. This other person responded by flooding listings of my friend's previous employer (which has been out of business for a few months) on nearly every review site out there with a continuous stream of utterly fabricated reviews. These reviews range from calling him a pedophile (easy enough to get deleted) to pretending to be a client who was screwed by his incompetence. I'm not sure there is any way to differentiate the latter kind of reviews from genuine negative experiences.
Despite all its flaws, Yelp is the best solution I've had for finding good restaurants and other services. I love being able to read reviews by locals I might randomly see, browse through their profiles and see what else they like. In the vast majority of cases the crowdsourced reviews are awesome. Unfortunately their verification and filtering system is weak, and needs major improvement (an identity verification program would be awesome). Yelp is still a great service overall, and the fact that I use it over ZAGAT or anything else tells you a lot about how effective it is.
The problem with the proposed solution of verifying ID is that people who leave honest bad reviews often enough get harassment sued by angry dishonest business owners, who it turns out are more likely to harass and sue companies than the honest companies are.
The main problem here is that Yelp is a racket as well, this has been known for some time. Those using it, well caveat emptor.
There are other review sites out there that are not as morally compromised as Yelp due to their business model being modeled on a mafia protection racket of "Pay us and we can help these bad reviews go away."
I agree, though the example gripe is problematic as the moving industry is largely unregulated. It is full of outfits that are subcontracted by the big movers. Extortion is commonplace (moving truck is near your new home. they stop and call on phone asking for $2k more before they arrive and unload your property), and is considered a contract dispute and the police will not get involved, citing such.
Yelp has been having problems for a long time, especially with small business owners. The site is killing small business owners, just do a Google search and you will find endless complaints about the site. Truth is Yelp cannot control the reviews, they can't guarantee them and their Filter... if it works only 50% of the time then it is worthless.
There's no technical workaround for having good references from trusted source for something as important as movers, a dentist, a skilled tradesperson, a doctor, et al. (edit: I don't care, it's true. Even Angie's List isn't great)
I stopped using Yelp after I noted that reviews I submitted were only visible to me, if I was logged into the site.
That defeats the purpose of an online review site -- if the review is only visible to me, why am I posting to Yelp, and giving them valuable data they just will sit on, as opposed to logging in my own local host `reviews.txt`?
I noticed also that a number of favorable reviews, seemingly written by authentic reviewers were also suppressed in like fashion.
See Amazon for a site that gets posting reviews mostly correct.
I use Yelp to get ideas about what places to eat, but recommendations and confirmations from friends is usually the final say. I really like Yelp, but things like this keep coming up and it's hard to trust that their heart is in the right place. Not to mention how slow they are at staying competitive with other services. Check-in feature seemed to take forever, and I still can't believe they haven't tried to implement customer appreciation cards a la Stampt.
Let's talk about the mobile only world: would a native mobile app be self-verifying in that only a mobile phone could download and use the app - so every review requires a real smartphone to post the review. Although "reputation" companies could game this situation by buying 1000 smartphones, but then you would have 1000 smartphones with the same pattern of up-posting such that it would be easier for an algorithm to recognize these "planted" phones.
I generally defend Yelp against a lot of mis-informed opinions but this is pretty bad. It's clear Yelp is hiding a lot of legit reviews. There's a decent case Yelp should be erroring on the side of showing not hiding. At a minimum, Yelp clearly needs to improve its algorithms. As the OP describes, a bad experience with a service provider could easily lead to a 1 star review from a first time reviewer. And these are very helpful reviews.
Honestly the author is over thinking his solution to the problem. The way to solve this problem is to give people the option of viewing reviews that Yelp believes are of dubious origin. Add another section to the reviews page, and let the users decide how to process the extra information but frame it as possibly being untrustworthy with an explanation of why it was tagged as untrustworthy.
They already allow you to click through to the reviews they have filtered out. How about having two average ratings, one for only "filtered in" reviews and one for all reviews?
And the font is about 10px and 30%K buried in the bottom corner. I can understand that would be desirable if the algorithm was more accurate in filtering bogus reviews, but that's clearly not the case so it just seems unscrupulous.
Remember, everyone, if you aren't paying for it, you might be the product. In this case, a Yelp user is the product that Yelp sells to businesses by featuring 5-star reviews and burying the 1-star review.
Featuring the 1-stars and burying the 5-stars would also be quite the incentive for a business to pay Yelp, but it would be cynical to assume they are doing that, too.
Like what everyone is saying, Yelp review system is very easy to game. Just create a one off profile, and hype the business. Yelp algorithm only allows good reviews getting posted while the bad ones are buried. Why? Because good reviews generate traffic to Yelp which in turn help Yelp getting money from advertiser.
Would anyone be interested in disrupting this space? My friend and I have been working on something we quite like that deals with subjectivity/ethics concerns, but it's a tough space to be in, and we'd love some help. If you want to get in touch, email us at questionasker[at]liveDotCA
True, Yelp got the filtering completely backwards in this instance. But, do you base any economic decision on two 5-star reviews from two Yelpers with 0 friends and 4 reviews combined?
Not everyone is a sophisticated Yelp user with an understanding of social networks. A lot of people will simply see two 5-star reviews, zero negative reviews, and think that looks like a safe bet.
Are you saying it's Yelp's responsibility to protect unsophisticated users from themselves? Also, I am not quite sure "a lot of people" will behave as you describe.
If one has to become a sophisticated Yelp user and do a substantial amount of detective work in order to determine if Yelp reviews are legit in order to get any value out of it (or at least not get actively misled by it), then what value does Yelp provide?
Is there a startup opportunity for an identity verification service whereby you do the ID verification once and then each service that wants to use it gets charged a smaller amount?
Example: consumer goes to idverificationsite.com, registers there for free- they ask out of wallet questions to verify identity, potentially do a minimal background check too.
You then use this login with Yelp, AirBnB, Getaround, etc.
Yelp, Getaround, Airbnb pay idverificationsite.com some small amount per verified person. Removes some of the friction of ID verification/spreads cost around.
There was an interesting article in Inc. magazine a while back about a company that tried that. They found out that no sites wanted to use the service, even though they saw how it might solve their problems, because it would immediately make their non-verified users less valuable.
Google actually used to do verification on people with Knol; you'd type in a credit card number and it would look up your name on the CC and match it against the name in your profile. I think that functionality got retired.
It would be nice if this service was available more globally.
There are at least two startup opportunities for something like this. One is the creation of the credential/verified identity certification and the second is the transmission/exchange of the credential. These are two very disparate set of competencies.
Several years ago, I was working with Honesty Online, doing exactly this identity verification startup prop and where it became very evident that this is really two businesses. Now we, in another startup, sigkat.com, are very early stage in the second business - creation/exchange/transmission of any sort of credentials so that vertical apps like identity verification can focus on their domain.
Identity verification and such are only part of the problem. Even if it is a bona fide person writing the review (and ignoring the problems w/Yelp filtering practices), a truly valid review only comes from one who the readers can be assured really was a customer of Royal Movers in addition to some of the other, lower identity bars.
I have no pity for this guy...seriously he only went by Yelp's reviews and did no further research.
This article just proves how stupid consumers can be when making important decisions.
When I read the article I didn't see the guy mentioning looking up the company on the BBB or even simply Google'ing "[Company] scams" or "[Company] scammers".
Yes, Yelp could implement better solutions to protect consumers but ultimately the consumer needs to do adequate research before purchasing.
So there's a resource out there that claims to let the customer do a good portion of that research by aggregating real customer reviews and you berate him for using that research? Calling the guy "stupid" for believing a bunch of supposedly satisfied real customers while at least 10 extremely negative customer experiences were hidden from him is bullshit.
A company I used to freelance for found that after they stopped paying Yelp for a premium account, they started seeing more negative reviews and a number of their positive reviews were filtered out. They got almost daily phone calls from Yelp marketing types essentially suggesting that if they renewed their subscription, the negative reviews would 'disappear'.
This was a few years ago, so maybe things have changed since then. Either way, I wouldn't hold your breath for a better review verification system.