Science Figures Out How To Identify Groups Of Fake Online Reviewers

Anyone who has sifted through anonymous “user” reviews of products is likely aware that there’s a good chance some of those comments were posted by shills trying to game the system to make the product look much better or worse than it is. While there are already a number of common-sense ways to suss out a bogus review, science has found a way to identify entire groups of review spammers.

A recent study by researchers at the University of Illinois at Chicago, with some help by Google, looked at ways of identifying the behavior of both individual review spammers and groups of spammers.

“Although labeling individual fake reviews and reviewers is very hard, to our surprise labeling fake reviewer groups is much easier,” write the researchers in their report titled Spotting Fake Reviewer Groups in Consumer Reviews.

Here is an example from the report:
spamreviewsss.PNG

Figures 1, 2, and 3 show the reviews of a group of three reviewers. The following suspicious patterns can be noted about this group: (i) the group members all reviewed the same three products giving all 5 star ratings; (ii) they posted reviews within a small time window of 4 days (two of them posted in the same day); (iii) each of them only reviewed the three products (when our Amazon review data was crawled); (iv) they were among the early reviewers for the products (to make a big impact). All these patterns occurring together strongly suggest suspicious activities. Notice also, none of the reviews themselves are similar to each other (i.e., not duplicates) or appear deceptive. If we only look at the three reviewers individually, they all appear genuine. In fact, 5 out of 9 reviews received 100% helpfulness votes by Amazon users indicating that the reviews are useful. Clearly, these three reviewers have taken total control of the sentiment on the set of reviewed products.

Check out a PDF of the entire report here.

Google-sponsored research identifies groups of bogus product reviewers [The Verge]

Comments

Edit Your Comment

  1. dolemite says:

    I find this study about fake reviews to be excellent. It’s probably the best study about fake reviews that is available right now. If I had to do it all over again, I’d definitely select this study again, and I’d have no regrets. If I could give it 6 stars, I would.

  2. DevsAdvocate says:

    Holy crap… a Consumerist article which actually helps consumers!

  3. Weighted Companion Cube says:

    Let me tell you, this has to be one of the coolest PDF’s about fake reviews ever on the internet. I can read it anywhere I have a computer the has Adobe installed on it. I would give it 10 starts if it would let me.

  4. consumed says:

    Am I the only one who sees those mathematical formulas and my brain starts shutting down? It all looks like I’m reading a Greek alphabet…

    Can’t they just explain this in a 1 or 2 page paper that a normal human being can read?

    • maxamus2 says:

      I prefer not to dummy things down. Perhaps if you don’t understand it is you that should change and go and learn the math???

    • huadpe says:

      Many studies come with an “executive summary” that goes over the findings minus the math. But really, you can’t evaluate the merits of the study without the math. Especially in a case like this where the study essentially IS math.

      And the reason it looks like a greek alphabet is ’cause it is. Sucks if you’re an ancient greek mathematician though.

      • FatLynn says:

        It’s called an abstract, and if you click through to the study, it’s right there on the first page.

    • Mr. Fix-It says: "Canadian Bacon is best bacon!" says:

      But but… Mathi s power!

      Between Observer Effect, Quantum Superpositioning, and the EPR Paradox, I think I could mathematically prove the existence of ‘magic’!

    • ripoffnation says:

      The math is there largely to scare off most people and becomes the
      basis for a patent for a future product/service that google will introduce.
      This will make sure google can drag its competitors to court when they follow suit.

  5. giax says:

    And how about Urbanspoon or Yelp comments?
    “I’m a real Asian, this is not how they cook in China/Japan/Korea/India” – as if all the home cooking in all China was just of one kind? For every Asian place I’ve eaten in there’s always a few of these “I’m a real Asian” commenters. Who perhaps have a not so successful, competing restaurant of their own?
    I’ve been craving to use the same card some day for reviewing some place that serves European or European style food (Italian, German, Scandinavian – pick any). “I’m a real European, and this is not how they cook in…”

    • LadyTL says:

      I’ve seen reviews like that for greek and italian places where I live. It’s kinda funny to see honestly.

  6. caradrake says:

    I get an email every time a new review is posted at an apartment complex I moved out of (horrible horrible experience there). Almost every day, and sometimes many times a day, a shill review has been posted for the last month or two. Whereas before one review would come in every few months, and then it was a toss-up between positive or negative, they’ve had dozens of positive reviews, all saying almost the same thing.

    It would be amusing if it wasn’t so pathetic.

  7. thisisit says:

    For any product/place you should only read the negative reviews.

    • Cat says:

      I do that, too, and I find sometimes the negative reviewers are just morons. Other times, I wonder if they are “Reverse Shilling” for competitors.

    • vastrightwing says:

      Actually, that’s what I do. I don’t care for the positives ones. I want to know what issues a product or service has. If I see nothing but perfect reviews, I figure they’re all fake. On the other hand, when I read, “don’t buy this, it sucks!” That means nothing to me either. I want to know why.

    • Elizabeth B says:

      I always read the 2, 3, and 4 star reviews. Generally 1-stars hate it and won’t consider any remedy, 2 stars explain what went wrong, 3 stars explain what went wrong and how they fixed it, 4 stars say how to use it since the 1 and 2 star people used it wrong, and 5 stars are fanboys who won’t say a bad word against it, ever, even if it ate their mom.

    • BorkBorkBork says:

      Yeah, the middle ratings are where it’s at.

      I browse them looking for common comments, either positive or negative.

    • Fubish says: I don't know anything about it, but it seems to me... says:

      I check the 3-start reviews. They usually have the good and the bad about the product without being snarky or gushing.

    • Outrun1986 says:

      I try to read a mix of all reviews, and I look for trends across different sites. If 15 people on Amazon talk about the same flaws of a given product then its probably true, especially if that information is corroborated by visiting other sites and finding the same.

      The problem with 1 star reviews is they often don’t say anything about the product themselves, it will be a person who just hates the product, or someone who never got the product, or someone who had problems with the way the product was shipped or delivery wasn’t quick enough for the buyer. Neither of these reasons tell me anything about the actual product I am looking to buy.

  8. axiomatic says:

    Paging Harriet Klausner… you article is here…. Harriet Klausner… please pick up the white Comsumerist telephone…

    (Harriet Klausner is Amazon book reviewer shill known for giving great reviews without ever reading the book.)

  9. Corinthos says:

    The only reviews I trust are the ones with some cons or why it could be better. Those fake reviews don’t usually go into detail about some of the things that aren’t great about a device or what it needs to include in the future to be better.

    • caradrake says:

      And, alternatively, if a review is nothing but bad, it can also be a shill from a competitor.

      I trust reviews that are a mix of both.

  10. gman863 says:

    On Amazon, look for the phrase “Amazon Verified Purchase” under the review title. This is especially true on the early reviews (although unscientific, a bunch of 5-star reviews from non-verified purchasers that are posted soon after a product becomes available rasises my suspicions of shills).

    Newegg.com has a similar feature where you can filter out reviews from non-Newegg buyers.

    • Fubish says: I don't know anything about it, but it seems to me... says:

      On Amazon also beware of reviews from “Vine” reviewers and “top 1000″ reviewers. These people post an incredible amount of reviews and they really are all garbage. Take a look at the ‘Top Reviewers’ forum on Amazon and you’ll see how a lot of them admit to posting and then re-editing reviews to garner positive reviews of their reviews! Apparently, the actual product doesn’t enter into the equation – it’s the REVIEW that counts for these people. You have to disregard these reviews.

      • GoldVRod says:

        I’m an Amazon Vine reviewer and your post is inaccurate.

        Do we get the items for free? Yup.

        Are we asked in anyway to only write positive reviews, or inhibit negative opinions? Do the companies have any say in what we write? Are negative reviews edited or removed by Amazon or the manufacturer? Are we removed from the Vine program for receiving a certain amount of ‘not helpful’ votes? Do we gain ANYTHING extra by receiving lots of ‘helpful’ votes?

        Nope.

        There is no benefit for a Vine reviewer in only writing positive ‘shill’ reviews. In fact I’ve received more positive helpful votes when I’ve been very critical of an item than I ever have when I gush over a product needlessly.

        Vine reviewers for the most part spend countless hours or even days using then reviewing items that are sent to us – often far longer than the value of the item could justify in terms of hours. And since two items a month (of the four you get) are specifically targeted products that we’re familiar with, often we can give a much more accurate expert opinion on the widget.

        Are there exceptions to this? Sure – some people like climbing the reviewer position ‘ladder’, but writing only positive reviews doesn’t guarantee only positive votes – if anything, as I said, it’s the opposite.The program as a whole is probably the most honest one out there. There’s no hiding a Vine review and no gain or loss the the vine reviewer if you don’t find the review helpful.

    • joako says:

      NewEgg censors reviews so the ones they don’t want posted simply don’t show up.

  11. yaos says:

    What’s so surprising that it’s easier to predict the outcome of a group vs. an individual? It’s always easier to predict the outcome of a group, why this is the case is the interesting the part.

  12. doctor_cos wants you to remain calm says:

    How about fake online commenters?

    “I find the Consumerist to be useful and entertaining, and more enjoyable than chocolate.”
    ****

  13. Clyde Barrow says:

    I always thought it was fairly easy to know when a review was fake. The posts are loaded with colorful adjectives and words are that nothing close to what a normal person would describe. Customers’ don’t usually say things “rock”, or “it is the best that I have every scene”. Most posts will always have something positive and negative and a somewhat balanced comment to something that they find personally attached too. Every post should be different. If it’s “too good to be true, then it’s fake”.

    • Happy Tinfoil Cat says:

      The ones I love are the ones that use a fancy word but misspell it, under several account names each of which only had reviewed less than 3 products/services. Their graph has a bunch of 1-star reviews balanced by an equal number of 5-star reviews.