That particular topic about Yelp and their reviews have been discussed many times on HN.
Yelp is a major scam company that absolutely do not care about the business owner. They are in it for the money -doh- and what matters for them is Page Views, and regular users posting more reviews, thus more page views.
Their filtering system is a total mess but since it is a secret on how it works they can say whatever they want... As long as their filtering system goes into promoting page views.
Nowhere in there they care about the small business that are badly hurt by their lack of common sense.
Simply don't use Yelp for posting reviews or reading reviews, this is the best thing anyone can do.
PS: I speak from close personal experience with my wife small business being hit hard by Yelp algorithms for filtering review, and Yelp scammy sales people.
Out of curiosity, what is the alternative? Just allow twenty five-star or one-star reviews to show up and say hey, if business owners want to scam the system by putting in fake reviews, then great? Or just open-source their algorithms?
I guess to me it seems infinitely easier to criticize the way someone else does it than to present a more efficient way of doing it.
One possibility would be to allow users to manually flag reviews, instead of trying to do it algorithmically. They could also have some sort of confidence rating for reviews, or possibly a reputation system for reviewers. From looking at the hidden reviews for a couple of stores while I've been trying to find a place to get new glasses, they seem to be hiding a good number of legitimate reviews with their current system.
You are correct. Such simple change will be already going in the right direction.
It is amazing to see all the reviews hidden that are legit: you can see if the reviewer use Yelp often by checking the number of reviews posted.
The current filtering system seems to ignore that most of the times, but reviewers with no previous review prior giving 1 star will see that review stick.
Unfortunately, this takes the current problem(where legitimate reviews are removed), and replaces it with the exact same problem, where legitimate reviews are flagged to oblivion.
Instead of having the owner's friends post reviews, they post reviews and then flag every poor review, attempting to get them taken off. Or idiots with an axe to grind flag all the good reviews.
I think it was a useful, informative article. Why do you say she runs a shady company? Electrolysis, as far as I know, is a legitimate hair removal technique.
I understand Yelp filtering out what they may see as astroturfing reviews, but on the other hand I can certainly understand a business owner getting customers to submit positive reviews to counter negative effects of Yelp. Finally, is paying $350 a month to Yelp now considered a normal cost of running a bricks and mortar business these days? Unbelievable.
Agreed. Not to say there isn't a problem with Yelp's practices (as have been documented elsewhere), but this article does not make a good case against them.
From an algorithmic standpoint, filtering out reviews from new users makes sense. Otherwise you'll get the owner and friends stuffing in good reviews.
Just look at what the data might look like for the 17 people.
-They all sign up in a period of a few weeks.
-They all review this one place and nothing else.
-16 of the 17 never sign back in again.
It's pretty clear that it's a blatant rating stuff, so filter it out. Again, I'm not saying that the algorithm is perfect, but it makes sense to at least filter out these reviews.
I don't think that really has been documented elsewhere. There have been lawsuits and allegations, but no proof, as far as I know, that Yelp has offered to compromise their reviews for cash. Yes, they give business owners willing to advertise some editorial control over their listing, but the extent of that control is fairly well-understood and not much more nefarious than Google's text ads. Correct me if there's documented evidence otherwise.
Instead, we get article after article of business owners complaining because their Yelp reviews suck and their obvious attempts to game them have been thwarted. But that's exactly what Yelp should be doing. They're useless if they can be gamed. They're also useless if it becomes well-known that they're pay-to-play, but again, I don't think there's actual evidence of that. They'd have to be pretty stupid to take that course.
>> Just look at what the data might look like for the 17 people. -They all sign up in a period of a few weeks. -They all review this one place and nothing else. -16 of the 17 never sign back in again.
Looking at the 17 filtered reviews, 16 of them are 5-star reviews from people with 0 friends and less than 5 reviews (mostly only have 1 review). There's also a 1-star review from a person with 0 friends and 1 review.
It seems like to me that their filtering algorithm is working pretty well.
Linkbait titles combined with the lack of ability to downvote stories. Even though I think restricting the downvote is a good idea, it contributes to this problem quite a bit.
And don't even get me started on the lack of ability to vote or comment on YC companies job posting spam (the reason for which I also understand)... I've spoken with a handful of these companies that end up listing multiple times, and even they have expressed interest in votes/comments simply to get feedback as to why their listings don't seem to work.
The article says she asked her customers to post genuine reviews. I don't know if that's true but if it is, I see nothing shady about asking your customers to post reviews. There is definitely a conflict of interest on Yelp's side, but I don't see a good solution to the spam problem.
Both things can be true: that Yelp has and exploits a conflict of interest, and that this article chose a pretty bad example of a business harmed by that practice.
Yelp specifically mentions that this would lead to blacklisting of comments in their help, for the reasons of resulting bias (you only ask customers who are happy, you might provide a discount or preferential treatment etc).
The unfortunate thing for Yelp is that there is a cornucopia of stories of similar things - but often it's because folks say they didn't pay Yelp for further advertising.
I'd never use Yelp for reviews. Either they are glowing when they shouldn't be, or they are rubbish when they should be glowing. Who would want to use a site like that?!?
It goes beyond that, though. I reviewed a local deli around a year ago. I had been using Yelp for a few months, and it was probably my tenth review. I hadn't been asked to provide a review. None of the reasons they cited in the article applied to me, yet my review was buried, and negative reviews from newer users were left unfiltered.
Yelp has a pretty shady history with this. The class action suit from a year or two ago was dismissed, so I assume what they are doing is legal in some sense.
I really don't get it. I mean, could Zagat have gotten away with this in the pre-internet era? "Hey man, love your restaurant, but each star costs $100."
I've been running a business review website for about 4/5 years and I absolutely hate it.
If I had $1 for every time a business owner writes a review for themselves (signing up for an account with their business email address no less), our system removes it and then they email me within minutes complaining that a genuine customer had written them a review, where has it gone...
There are lots of variables our algorithms track to try and make a decision on whether to filter a review or not and of course it will get things wrong but if there wasn't any filtering in place, the amount of spammy reviews would be overwhelming.
I feel somewhat for Yelp, even with all their millions of dollars to throw at the problem as there is no easy solution, ignoring any potential pay for stars type nonsense. That is something I would never do and am very proactive when it comes to protecting small businesses.
Like I said though, after years of seeing people try to pull a fast one, the warmth in my heart fades.
I keep seeing articles like this. I read the article, then I find the company on Yelp and read through the reviews.
Every single time it's been very clear that Yelp is telling these businesses that they have a problem. But rather than acting on this extraordinarily valuable feedback the owners/managers react by blaming Yelp for the bad reviews or accusing Yelp of hiding the good reviews.
This behavior frequently exposes the characteristics about which customers are complaining: there's a problem, the owners have been told about it, and they're killing the messenger.
EDIT: I have no connection to Yelp whatsoever. I'm communicating the scope of my experience and the conclusions I've drawn. These conclusions may be grossly flawed. It wouldn't be the first time today. :)
Clearly you have not been running a small business.
Some reviews are posted from time to time by people that are not even customers or stopped by the business; for some dumb reason they post a 1 star review.. And that review usually stick and doesn't get filtered out...but other reviews from true customers posting 4 to 5 stars reviews disappear usually in less than 24 hours after they were posted.
Regarding your ad hominem statement, I've been running small businesses since my first in 1996.
I've not witnessed this scenario you present and I have no relationship to Yelp that might give me some greater insight than anyone else. I have no idea why this might happen; I'm limited to communicating based on my own observations.
Spam filtering is hard? You are a programmer, imagine trying to find a way to objectively and fairly filter reviews. Unlike with emails you don't know whose labels to trust. There are good business owners who will only report spam reviews as spam. Then there are bad business owners who will report negative ham as spam to try and boost their reviews. Finally, there are bad business owners who will use shell accounts to submit positive spam and then mark it as ham.
In general the labels from business owners are not trust worthy which gets them into these situations. Thus they try and create other ways of identifying spam: like user engagement on the site. However, engagement doesn't always work as a metric as spammers can cultivate spam accounts by posting lots of "legitimate" reviews on unrelated businesses and then sell reviews from the accounts to attack (boost) specific businesses. Finally, users who have a tremendous experience at a business and go on Yelp to give them 5 star reviews may have no history with yelp except as a passive browser. The engagement metric will weight poorly for their review.
So what is to be done? I don't have a good answer for that. This seems more like a human problem than a technical one. The best course seems to be: act ethically, serve your customers with compassion, write responses offering assistance to negative reviewers, and if you feel Yelp (and similar review system are a bad idea) put your advertising dollars elsewhere.
They are various ways to solve the problem but again it is not in Yelp's interest. Yelp's business is about selling Ad and they don't really care about the underlying issues.
One simple thing that could be added to their system was a way to verify that a reviewer did really have a business relation for posting a reviews. They are various ways for solving that.
Even a simpler approach: why hide those reviews. Keep them here and introduce a way for other viewer to rate the reviews aka 'X people found that review helpful'. That way all reviews are visible and the random person reading the Yelp page can decide for themselves.
Unfortunately Yelp and their filtering system have a different approach and instead usually end up filtering real customer reviews and leave the more 'spammy' visible. Then after a few weeks like that, you will receive a phone call from Yelp sales folks telling you that if you purchase 'ads' then your potential 5 stars review (if any are left visible) will be pushed to the top.
There is no relationship between the sales calls and the filtering algorithm. That is what the dismissed class action lawsuit was about. All "evidence" people have is purely anecdotal.
Verifying a "business" relationship may work but it still won't solve all of the problems. You can still pay people to show up at your restaurant, pay with their credit card and write a five star review. You can still pay people to show up at a competitors restaurant, pay with their credit card and write a one star review. It simply increases the cost of the spam (which is a good thing). They should probably create a way to do this but it won't solve the fundamental problem: the reviewer pool is an unreliable source of information.
"X people found that review helpful"
A helpful review is not the same as an accurate review based on experience. I would find a well written review helpful. However, the review could still be inaccurate or fake just well written. This is simply another data point for spam detection and one with many confounders at that. One has to simply google around about the state of Amazon reviews to see the flaws in this approach.
Once again. I stand by my assertion, detecting spam reviews is extremely difficult because it is a human problem.
I can vouch that this exact scenario has happened to one of my small businesses. Multiple reviews from actual customers are filtered out, but a negative review from someone who has never interacted with our organization is displayed normally.
Were your actual customers regular users of Yelp who have reviewed other businesses actively?
We can pick apart Yelp's model and likely should, but it does appear to discount "drive-by" Yelpers who only sign up to give a good review to a single business.
Yes, in my experience I have seen reviews from regular Yelp folks (aka posting maybe a review once or twice a month) see their 5 stars review vanish in less than 48 hours... While at the same time a person posting a 1 star review without any other review posted prior and even after stick forever.
This is a classic 'Yelp' business practice and many business owners you can talk to will have many of such reports.
No, I believe most of the reviews were from first (or second) time yelpers. I can understand where they're coming from on a macro scale, but at the individual business level, it can be irritating.
If their ranking algorithm is pushing reviews with high page-views to the top, then perhaps it's as simple as negative reviews getting more attention, either because negativity draws people in or because people are more interested in trying not to have an awful experience than in having a great one. On the other hand, if the algorithm is optimized around other values (credibility, for instance), then it wouldn't make sense for reviewers with no history to get their reviews pushed to the top. Hard to say, given that it's a trade secret.
You presumably work at Yelp? This isn't a case of killing the messenger. This is a case where the company wants business owners to conform to its way to dealing with spam rather than have its engineers come up with a better solution to the problem
Read this very well written article on the problem and possible solution and maybe you'll have a better idea of the problem at hand.
No, I don't work at Yelp. I don't think I even know anyone who knows anyone who works there or works for their VC investors.
And I'm not saying that a problem doesn't exists: I'm saying that *within the scope of my experience( the complaints about Yelp have been more about owners/managers not wanting to deal with something that sucks about their business.
I'm on the opposite side of the equation: I quit using it after I discovered it was hiding my positive reviews of establishments I frequent. I refuse to trust anything on that site now.
This does demonstrate the value of third-party "professional" reviews (e.g. Consumer Reports, Zagat) and a big reason why Google bought Zagat -- the trust factor.
You never know if personal reviews on various sites were driven by ulterior motives. Although it's likely that the majority of personal reviews are genuine, you should try not to be swayed by just a few reviews without further research.
Professionals have a reputation, which should reduce bias or influence. If Zagat was known to take bribes, they wouldn't be around for very long.
It may happen, but is less likely to happen than non-professionals. You have a much larger issue with selection bias for reviews, and a much easier way for vendors to bias them. I've been in shops that insist, rather persistently, that I write a good review for their store on their computer, at the shop.
Great points! The issue of selection bias is very real on many review sites. Especially with the volume of reviews being so low for most products and services.
I don't really see a good logistical way to deal with this kind of problem. Any algorithm is going to see a bunch of sudden reviews of extreme positive or negative opinions as possible spam, and for users who never review more than once, I'm not sure how the filter is supposed to know what is and is not legitimate.
Yelp does seem to have a conflict of interest with advertisements, but considering the only alternative seems to be a subscription-based model, and people don't like paying for things on the Internet, I suspect that this is still the best possible case at the moment. Plenty of room for innovation, though.
There's a well known solution for this and it's called a chinese wall. You segregate the ad team from the other teams and restrict the data flow between them.
It's what Google and many others use to stop ad revenues biasing their product.
Simply don't use Yelp for posting reviews or reading reviews, this is the best thing anyone can do.
PS: I speak from close personal experience with my wife small business being hit hard by Yelp algorithms for filtering review, and Yelp scammy sales people.