How Do You Get The Most Out Of Online Reviews?

Products don’t advertise their drawbacks leaving shoppers to rely on online reviews as one of the only ways to determine a product’s true worth. Salon argues in an article heavy on fluff and light on content that reviews are just a meaningless muddle of questionable opinions. We disagree, but the article does raise one good question: how do you judge the value online reviews?

Salon has trouble navigating the marketplace of ideas:

But what of Xinyi Yang “lilyyxy,” who claims the DeLonghi DCH2590ER Ceramic Heater with Remote Control was so noisy it kept her up at night? I have no way to tell whether or not Xinyi Yang “lilyyxy” is a light sleeper, or if he/she used the heater on an outrageously high setting, or if he/she placed it right next to his/her head. I can’t even tell what gender Xinyi Yang is!

And what about dinner tonight? Should I cancel my reservation at an Indian restaurant because mzt wrote a post about it titled “Bland, Blah and Expensive”? Who is this mzt, anyway? Is his bank account as small as his taste buds?

Much as with critics — the ones who do this for a living — personality and personal preference play major roles in the review process. But while professional critics are held to certain standards, the average consumer is not. A spilled drink, a lost wallet, a bad fight or oversalted popcorn can ruin any experience, but full-time critics are paid to look past these details and speak to the larger experience. Roger Ebert and cornpops116 can both give “I Am Legend” three stars, but only one of them would dare use (and this is an actual quote) “Best cheesesteak I ever had” as justification.

We avoid Salon’s troubles by immediately discounting reviews of low-dollar items like milk and ballpoint pens. Devoting the time to review tissues is evidence enough of questionable judgement. We mostly look at the volume of reviews and for repeating themes. If several people claim that an air conditioner is noisy, we’re going to trust that they don’t have super-sensitive hearing. Amazon is great for dredging broad consumer sentiment, but we are more inclined to trust sites that follow a niche market.

What are your tricks for determining the worth of online reviews? Share in the comments.

Was this review helpful to you? [Salon]

Comments

Edit Your Comment

  1. just_paranoid says:

    online reviews can be hard to decipher truth. not too mention some people just expect too much from the products/services they pay for.

    i take them with a grain of salt. i use to sell kirby vaccuums and we’d tell folks good news travels fast, bad news travels faster. so if you like what you see, you’ll tell a few people. if you hate what you see, you’ll tell everybody.

    so my point is, you will usually see more bad reviews than good. and for every good review you see, you could probably multiply that by 10 or more, to make up for the other satisfied people who didn’t leave a review.

  2. timmus says:

    Isn’t most of Salon heavy on fluff and light on content? I think that sentence technically requires a dangling modifier.

  3. bohemian says:

    I take them with a large grain of salt. Some people are just going to complain or are looking for the negative when they write reviews. But you can find recurring themes on products. We were interested in finding out more on a certain brand of vacuum. They were fairly low in cost so we were worried that they were not well made. In looking at reviews online we found out that just about everyone was impressed with the suction and cleaning. Many people listed the back release as way too hard to get it to release. Hotel reviews are kind of the same, you can find running themes and those tend to hold more value than the one person going on a rant that the desk staff wasn’t nice enough and there were water spots on the sink in her room.

    We were also able to find out if various hotels had things nearby that we wanted or the hidden fees and pitfalls the hotel won’t tell you about.

    For things like milk or pens, no.
    For big ticket items or travel, yes.

  4. clyde55 says:

    It depends. The first thing I check is how many review there are. Then I’ll try to separate the corporate shills and the reviews by people who hate everything. I know it’s a guessing game but after a while you kind of develop a sense of those things. This site helps you hone your skills in that regards.

    If there are a large percentage of negative reviews (more than 50 per cent) then I tend to be wary of a product, and do even more research.

    Likewise, if several product reviews from well known sites are good, and the customer reviews where the product is sold are generally positive, I will generally buy the product. It’s how I bought a digital camera a few weeks ago, and as it turned out it was everything I had hoped it would be.

  5. just_paranoid says:

    @bohemian: very good point on the hidden hotel fees. i use reviews for those as well and noticed. even then though someone who was complaining about a $5 fee, and i looked at what all you got for that $5, and to me it sounded like a deal and was. i’d like to think i’m good at pointing out the people who are impossible to please on these reviews.

    zenni optical was a good example. there were 5 bad reviews for every 1 good one. i’ve since ordered from them on 8 different orders and have, no joke, about 15 pairs of prescription eyewear. all were perfect. they did accidentally mail my order to someone else, and mailed me their order on accident (mixed up labels on packages), but that is typical error in a mail room and one call to customer service sorted it all out, i was very impressed.

  6. AlexDitto says:

    I suppose online reviews should work the same way Wikipedia so successfully does: if enough people hold the same opinion about a certain aspect of a product, chances are good that will show up in all their reviews, and you can generally assume that it’s either true or they’re all complete idiots, whereas one person shouting in all caps should probably be disregarded as either a vandal or a whiner.

    Of course, there’s always some risk involved when you over-analyze too much, and I think in the case of, say, ball point pens the very act of reading reviews probably wastes more time and effort then a few spotty ballpoints would.

  7. num1skeptic says:

    @just_paranoid:your first statement nailed it dead on.

  8. TechnoDestructo says:

    @just_paranoid:

    In online reviews, good news travels faster. It may not travel FARTHER, but it travels faster.

    People often post reviews on a product they just got and absolutely love, before the shortcomings have had time to show themselves. This is especially true with cars and electronics…and cars MUCH moreso than electronics.

    Check out [www.carsurvey.org] , and look at how many reviews are posted by people who’ve only had the car a week. This often skews the reviews for a model towards the good. You’ll usually not see bad reviews with less than a couple thousand miles behind them. So if you see a model on Carsurvey with a lot of frowny faces, you know it means something. If it’s all smiles, it may not mean anything.

    You also see this with movies…people praising a movie to the skies before it has time to sink in that…you know in retrospect it really wasn’t that great.

    Anyhow, for a lot of things, the venom takes time to accumulate before it can be discharged.

  9. just_paranoid says:

    @TechnoDestructo: i have noticed that as well. i bought a cd player, and you could tell the reviews were wrote by people who’ve only had theirs a few days.

    btw- on a separate note, the vr3 in dash car dvd players at wal-mart are an excellent deal, and i know how to overide the safety feature that only allows you to watch dvd’s when the car is in park an parking brake is engaged. in case anyone wanted to know.

  10. just_paranoid says:

    @TechnoDestructo: and btw, good news travels fast, bad news travels faster is an old old old saying, before t.v. and probably radio was around.

  11. meballard says:

    I tend to look at the negative reviews first, but judge them on what is being said and when it was said. If most of the negative reviews were older, or if they didn’t really have anything substantive to say, then I know it’s probably not too bad. I personally feel you get more from analyzing the bad reviews than the positive reviews for determining if a product or store is at least decent.

    After that I will look at the other mixed and/or positive reviews to look at the highlights of the product.

  12. VoxPopuli says:

    Wow, giving our reviews online of online reviews – too meta for me on a Saturday afternoon.

    I also read such reviews with a grain of salt and look for repeated complaints/praise.

  13. DrGirlfriend says:

    On an individual-review basis, I pass over reviews that are very general (it’s great! it sucks!), that have really poor spelling/grammar (it just gives me a bad vibe overall), and that are vague in their descriptions (“I thought this hair dryer would be quieter. it wasn’t” or “This is too expensive for what you get”). Also, ones where the person is happy/unhappy with a product for the wrong reasons (“I’m taking away a star because I don’t like that it doesn’t come in pink!”)

    That kind of leaves a lot of reviews out, but it narrows it down to the people who took the time to explain their reasoning. But also, on a broader level, if a bunch of people reviewing a hair dryer all say that it should not be so noisy, then I’m going to take that as an actual trend and assume that if it bothers that many people, then there must be some truth to that (even if it is a vague descriptor on its own).

    I guess you really have to approach it objectively and avoid the impulse to take reviews at face value. It can be hard, especially if you’re sinking a decent amount of money into the product. Did this guy not like the product because it’s truly defective, or is there a chance that there was operator error? Is this girl liking the product for the actual features, or something that has no bearing on how the product actually works? Is someonegiving the product a bad review when it looks like his beef is with the website, or the shipping company?

  14. StevieD says:

    Shills and the competitors are so common that no online review can be trusted. Even big #’s of reviewers can not be trusted as vendors are quilty of pumping up products.

  15. bohemian says:

    A good point on the car reviews or people reviewing right after they buy something. Had someone asked my opinion of my current vehicle the week after we bought it they would have gotten a glowing review. If they asked me now, a few years later I would tell them it was a poorly designed pile of premature failures. I have been left stranded more times than I can count by sudden failures to find out later that it was a “feature” of the vehicle.

  16. dantsea says:

    The thing about review sites is that they’re not good for instant decision-making, there’s a certain amount of reading and data crunching that the reader has to do in order to make his/her informed decision.

    I look at the following factors:

    1. Common themes over a period of time: If 50 out 70 reviews in the past year note that Brand X has a loud fan, the odds are greater that Brand X makes an excessive amount of noise and lesser that the majority of writers are light sleepers.

    2. Number and tone of reviews: If the widget only has ten reviews and they’re all positive, my shill alarm goes off. Likewise, if the ten reviews are so uniformly negative, I smell a competitor’s agenda at work.

    3. Date clumping: You’d think that after ten or so years of online review sites, astroturfers would figure out that ten bad/good reviews posted within five minutes on the same date is counterproductive. And yet.

    4. Reputation of reviewer: Admittedly, this is easier on community sites like Yelp (which is popular in SF), where I can discard or discount some reviews based on what I know about who wrote them. On sites like Amazon, I try to see someone’s past review history. Some people seem to be disappointed with (or madly in love with) everything they purchase.

    5. Consult more than one site: If possible, look at what’s being said across two or more sites. It helps for the sake of clarity and finding out additional information about a product or service, and you might even spot a few shills spouting their boilerplate BS.

    But most of the time, I just go with my gut and take risks. I’ve had great experiences with businesses and products that the chatterazzi absolutely detested, and vice-versa.

  17. snoop-blog says:

    online reviews can be hard to decipher truth. not too mention some people just expect too much from the products/services they pay for.
    i take them with a grain of salt. i use to sell kirby vaccuums and we’d tell folks good news travels fast, bad news travels faster. so if you like what you see, you’ll tell a few people. if you hate what you see, you’ll tell everybody.
    so my point is, you will usually see more bad reviews than good. and for every good review you see, you could probably multiply that by 10 or more, to make up for the other satisfied people who didn’t leave a review.

  18. besides the things you mentioned, there are a couple of indicators I use for a review’s worth.

    1) Call em an elitist @$$hole if you want, but I tend to think there is a high correlation between intelligence and one’s ability to express thoughts clearly in written form. Look for clear, concise but informative reviews that don’t meander or stray from the topic at hand. Things like “ZOMG DiS iS TeH b3sT pnKake sPaTulA EvArrrrrrr!!!!11!one!eleven” get ignored.

    2) Avoid fanboys. This cuts two ways. Someone who thinks Bill Gates is Jesus reincarnated should not be informing your opinion about buying a Zune, but on the other hand, you probably shouldn’t pay attention to the guy that owns 12 iPods either.

    Regardless of the quality of reviews, however, you should set some budgetary constraints. Personally, I don’t buy anything that costs more than a couple hundred dollars unless I’ve personally used it or the seller has a particularly liberal return policy (I love you REI!). The fact of the matter is my standards are higher than most people’s, so just because the unwashed masses think something is great doesn’t mean that I’ll agree.

  19. indicate says:

    I consult multiple sites for professional reviews. I also like looking at the consumer reviews on places like Amazon and Buzzillions to see how the general public likes an item.

  20. Amry says:

    You know what? I used to give online reviews a lot of credit – I’d use them when making decisions about what products to buy, and they always heavily influenced my decisions about what hotels to stay in.

    Recently, though, it’s finally sunk in that there are so many people in this world who will complain about EVERYTHING, thus making consumer reviews are totally and completely flawed. You’d think after working in customer service my entire career this would have sunk in earlier, but no.

    So now, I’ll skim them, especially for neutral comments like, “color looks different in person” or etc, but I – gasp – have begun to make my own choices based on my own opinions! It makes for much less stressful buying and leaves no one to blame but myself if I’m not happy with something.

  21. no.no.notorious says:

    i look for on-going themes and well written reviews. i also read mildly longer ones, since i assume they have more detail.

  22. Ashcan says:

    @TechnoDestructo: The one thing that drives me a bit nuts are the people who review and praise a DVD or CD months before its release date. The item may be listed for preorder, and people will write a review as though they have already seen/heard it. Reviewing a DVD is not “Great Movie!!! Must See!!!”, but more of telling people how well the transfer was done, the quality of the extras, etc.

    The same is true of an upcoming music release. Just because you love the band does not mean that the future CD release won’t be a stinker.

  23. iamme99 says:

    I look for specific details. If you spend the time to rationally describe the issues and problems you encountered, then your review has more credence with me.

    I usually look for professionally written reviews but even these can be a crapshoot. I’ve been going crazy searching for a new digital camera under $400 for 2+ months now. I want certain functionality, like RAW capability, which limits the universe of choice. I’ll read one review, then another and too often, it seems like totally different cameras were reviewed, even though they were the exact same model!

    I usually check out the reviews on Amazon, Newegg or epinions. People at these sites tend to give more detailed and real experience reviews than elsewhere, IMO.

    I’ve also come to realize that many people are just willing to accept mediocrity in products they buy. Some people expect perfection, others as long as the product doesn’t explode when they turn it on, are happy with any degree of performance.

    And on the net, no matter what you post, there will ALWAYS be others who will take the opposite view, no matter how devoid of fact and logic. Complain something is noisy and you will get people saying it isn’t so. Note that something is quiet as a mouse and you will often get the opposite [lol].

  24. madanthony says:

    I think there is a “wisdom of crowds” aspect to online reviews. One review complaining about a problem with a product doesn’t mean much – that person might just have unusual taste or have gotten a bad unit. But if most of the people who review a product complain about it (or praise it), it’s probably worth thinking about.

  25. marsneedsrabbits says:

    If the reviews seem to have a theme (this sucks or this is the best thing ever), I pay especially close attention.
    I didn’t like it because it wasn’t as green as I imagined it isn’t really a valid review.
    And I pay particular attention to what amounts to customer service reviews on Consumerist and other sites because I really don’t want to buy anything from companies where, if my thing breaks, they won’t stand behind their product.
    Or worse, if they are total jerks. The one that comes to mind most prominently is Princess Cruiselines who took money and refused to give a cruise or a refund to a 78 year old woman, demanded that another woman fax her father’s death certificate to get off their mailing list, and so on.
    I will never do business with a company like that.

  26. SaraAB87 says:

    I take the time to look for common trends in the reviews, usually if several people comment on the same thing, its probably true. Also look at multiple sources for reviews if possible.

    Think about how you are going to be using the product, often times someone will complain about something that has no effect on how you will use the item, for example if someone comments that a product is not compatible with windows Vista and you only run a XP computer at home.

    I prefer reviews from real people, not from people who are paid to write reviews for a magazine.

    Your expectation of the product should also be related to the price point, if you spend more, you should naturally have more expectations. If you are buying a digital video camera for 80$ however, don’t set your expectations too high.

  27. Eric1285 says:

    Generally speaking, I take well written reviews pretty seriously. It doesn’t matter if it’s on Amazon or Epinions or whatever. If the review is well written and it sounds like the person knows what he’s talking about with regards to the product, then I figure it’s probably trustworthy. On the other hand, if the review contains typos, grammar mistakes, and other such annoyances I tend to disregard it.

  28. KJones says:

    It’s ironic that the Salon item mentions Roger Ebert because I (used to) use him as a gauge of whether I would like a movie (not necessarily if it’s good). Nine times out of ten, he and I would agree on movies, so I continued to read his reviews.

    I’m not advocating people pick Roger Ebert specifically, but instead pick a reviewer who generally agrees with you on certain products. Tastes are like reputation, they’re built over time and not instantly created. Conversely, I never read Rolling Stone magazine reviews; if you know Jann Wenner’s “tastes”, then you can predict an RS review before you open the cover. The “reviewers” parrot his preferences: psychadelic, 1970s dance bands, and “flavour of the month” crap like the Scheisse Girls.

    There’s one good (?) thing about online reviews from supposed “professional” opinions: they’re rarely “5 out of 5″ or “hate it!” statements. Individuals (eg. me) writing reviews generally call products perfect or complete garbage. Collecting enough moderate views can (though not always) give you a good overall picture of the product.

    And by the by, Xinyi is probably (though I’m not certain) a woman’s name. I lived in Taiwan for a short while and it sounds like female names.

  29. SeattleGuy says:

    I used to work at Staples and as a minority of one, in my store anyway, would encourage office machines buyers to utilize Amazon.com and CNet.com as a source for customer reviews.

    I would explain that generally there are three types of customer reviews:

    1) The extremely poor review that uses all CAPS and was probably written by a frustrated 15 year old. If they knock something about the product that no one else notes, then the whole thing becomes less objective.

    2) The incredibly good review packed with technical details and opinions seemingly based on the reviewer having used the product for years. Well written and very polished, these reviews appear to me to be written by a PR flack or CSR hack somewhere. Again, I avoid these.

    3) The rest. These appear to be reviews written by some casual consumer or small business owner (just like me?) and are more likely to contain comments that are applicable to the buyer.

    It is from that third category that the average consumer would cull out the information required to make a good decision.

    The original point of the above article is a good one, what is the basis for the on-line reviewers comments? If you are smart enough to use more than just one review upon which you base your opinion, you stand a better chance of not being disappointed. And if you couple that by making your purchase from a retailer (either brick & mortar or online) that has a reasonable return policy you can almost eliminate a bad experience.

  30. lenagainster says:

    If it wasn’t for the online reviews, we would all have to be our own testers. I look at all the recent reviews I can find from multiple sources (seller’s reviews, competitors reviews, Epinions, CNET, PCMag, technically related sites, etc.) and try to judge from the majority leanings.

  31. frankblevins says:

    Overall online reviews are helpful. I do tend to look at the positive ones first to get a feel of why people might like the product. Though sometimes helpful negative reviews are generally useless. The fact is some folks don’t like anything or have an axe to grind. Maybe they were fired or simply disliked their former employer. Like LENAGAINSTER I look at various review sites to help me make a judgement.

  32. Blueoysterjoe says:

    I mostly just look for trends.

    I also look for very specific little gotchas. For example, I was reading a bunch of reviews the other night for a bluetooth headset and I discovered that it was easy to plug it in wrong and short out the headset. There were several people who mentioned it, so I was a little wary about purchasing the headset.

    Sometimes, the reviewers will have tips. They might say, “Oh, this camera comes with an obnoxious flash setting, but it’s really easy to set the default to something better, so I didn’t mind”.

  33. themediatrix says:

    One technique I haven’t seen mentioned yet – if there are a lot of reviews, I look for recent posts where the reviewer mentions the other reviews as inspiration:

    “I tried this place because you yelpers seemed to love it so much,” or “After all the positive reviews I just had to order one and see for myself…”

    It’s a way to see if the product or service lived up to the hype or not.

  34. nrwfos says:

    I haven’t read all the comments, but I’m going to just write my opinion – which seems to be what these customer reviews are anyway. I buy from Amazon frequently. I do read the reviews when there are some available. Of course I prefer reviews with particulars no matter if the reviewer liked or dislike the product. I don’t usually waste time reading reviews on products that I’m familiar with or are inexpensive locally that I can try out on my own. Everything is subjective except the facts that a product does or doesn’t do what it’s maker claims. That is the first thing I want to know. I usually prefer the non-five star reviews. I want to know details. I don’t waste time “voting” on reviews (unless the only review is totally unhelpful). The voting thing just has to do with how many readers liked or didn’t like the reviews and that count goes toward the ranking of the reviewers with Amazon. Some reviewers have loved every book they have read (and they red many) and the reviews really suck. That’s not helpful to me. I’ve also found that votes are counter-productive since it doesn’t matter is they are good or bad votes – the reviewer still get points for ranking for every vote. Subjective reviews just really need to have some facts within them to count with me. Books I’ve found are hard to gauge as to how what I might like is within the scope of others’ reviews. Video games are another gray area for me unless the reviewer manages to warn me about something within the game (like one reviewer who warned that one game did bad things to his Xbox 360.). I’m really looking for facts – not “I loved it” or “I hated it”. They might look for the same type of gameplay that I am. With books, movies, and video games give me the hard facts – is the format good? Is the movie dull, slow, or obvious? Does the game get difficult quickly? Does the book have a problem with too many story lines or undeveloped characters? Is the story disjointed? Are there so many typos that you can’t read them (I’ve had trouble with books like this – makes you wonder if they had editors.). I other product reviews I look for opinions on quality of production. How well does the item perform whatever duty it was made to do? Did the reviewer put it to the tests and what tests were they. Unfortunately, I rarely find those kids of reviews. Views on taste oriented items I find to be a waste of time unless the condition of the items were bad when received (particularly when consumables are involved.) All in all I do read reviews and usually get some idea of the product I’m considering, but I hate the ones that rate the shipping only.

  35. nrwfos says:

    @nrwfos: I forgot to say that one of the most important things I like to learn from an online review about anything one wears is whether the item runs large or small in size. That can be a crucial piece of information to have.

  36. UpsetPanda says:

    When I’m shopping for shoes, I always go to Zappos. The people on Zappos are so incredibly helpful, and many of the newer reviews reference past reviews. The only problem is that on Zappos, when someone says “the shoes were very tight” for all I know they could have really, really wide feet, unless they specify their type of normal fit. I recommended to Zappos on my last purchase that they give a blank to urge people to put down their shoe size.

  37. That70sHeidi says:

    I like checking out the other reviews of those who give really high marks or really low marks, as well as the ones written really well (to see if they’re shills).

    Also, sometimes doing this for travel can hurt you more than help you. I was so depressed reading hotel reviews of some places in Los Angeles it put me off my trip and I ended up not going. NONE of the places sounded good for the price I wanted, and while every experience is different, I’d rather save up and try a place that has good reviews than risk it on a place that has bad reviews sprinkled with good.

  38. The_Truth says:

    Online reviews are only usefull in some places. I like http://www.newegg.com for electronic and computer reviews, they have such a large number of them that it makes it possible to get a decent feel for the product, plus a number of the reviews are high on detail.

    As for restaurants, most reviews are essentially worthless as they are tainted by opinions in an ever changing service (Ie a waiter is there one day and gone the next), now I just use http://www.miniaturecube.com, let it pick a random restaurant for me and just roll with it :-)