By Michael Fisher | October 2, 2013 4:51 PM
Know what I love doing at Pocketnow, more than almost anything else? Reviewing mobile devices. You could feed me a steady stream of smartphones and tablets, one every ten days, month after month, and I’d still love it. Yes, it’s exhausting, and sometimes balancing timeliness with thoroughness is an ulcer-inducing tightrope walk, but it’s still my absolute favorite part of being a technology writer.
Know what my least-favorite part is? Assigning review scores.
We’ve graced this editorial with the Pocketnow Insider tag, so let me be candid for a second: I absolutely cannot stand assigning numeric scores to products I review. The process is so fraught with unnecessary compromise, and so much truth is lost in trying to distill two to three thousand words of copy into a handful of numbers that it almost completely undermines the core purpose of the review: to tell you what it feels like to use the product. A score doesn’t do that. It tries (and almost always fails) to encapsulate a subjective experience in an objective format, and it sucks. Furthermore, it lowers the quality of the reader discussion, by pandering to those readers who only open a review to look at the score, then immediately jump down to the comments to praise or lambast the reviewer for a “too-low” (or sometimes “too-high”) number. And there are about five other major issues with numeric scoring that vex me. Vex me, I tell you. If it were up to me, Pocketnow would eliminate review scores entirely.
Fortunately for those who don’t feel the same way, it’s not up to me; the numeric rating -along with the popular “Scored For Me” card at the bottom of our reviews- is staying put for now. So readers who enjoy bickering about numbers can breathe a sigh of relief; there will still be fuel for your endless flame wars.
Sometimes the comment section rises above petty bickering, though. Sometimes it generates even-toned, well-reasoned counterarguments, presented in a respectful manner. And often, the points raised in these elegant rebuttals are really, really tough to ignore. Such was the case with our recent LG G2 review. So many readers hated our score for LG’s latest smartphone that we took notice. And the ratio of thoughtful-response to death-threat in the comments was high enough that we were compelled to reconsider our position. So today, we’re doing something we’ve never done before in our 13-year history: we’re changing a review score. The LG G2′s new rating is 8.4/10, a bump up from its original score of 7.7/10.
I want to make it absolutely clear that, while this change was inspired by the community, it is not a move meant simply to placate our readership. If we were interested in coddling the commenters, we’d already have given the Galaxy Note 3 the “perfect ten” the Samsung crowd is crying out for, and Brandon’s iPhone 5S rating would get busted down to a 3.5 while being aggressively drop-tested on the train tracks at rush hour. Well, none of that is happening, so, you know … deal.
It should also go without saying that LG itself had no role in changing this score. The company has not asked us to alter a single word or number in our review.
The reason we’re changing it is quite simple: the G2 is getting a higher score because it deserves it. Our own Jaime Rivera will be exploring the device more deeply in a forthcoming piece, so I’ll leave the majority of that argument to him. Put simply: the combination of the G2′s specs, feature set, and user experience is compelling enough that, upon a week’s reflection, the 7.7/10 score no longer sat well with us.
Why didn’t we get it right in the first place, as we normally do? Frankly, (and let’s go ahead and ditch the collective “we” form of address here) I fumbled the ball. In the always-tricky tightrope walk between subjectivity and objectivity, I placed too much emphasis on the former and not enough on the latter. I scored the G2 based too much on my personal tastes. Now, expressing those tastes is essential to a good review, and I’ll never apologize for having an opinion – but in this case, I failed to place the G2 into the proper context.
Doing that is hard – especially in a numeric sense. We generally don’t rate devices that fall below a certain quality threshold, so a lot of the products we review end up falling in the space between 7 and 9, as you’ve probably noticed. This leads to some pretty intense hair-splitting as we try to differentiate devices using only the decimal points within a 20% slice of the total spectrum, and it leads to some pretty absurd outcomes.
Case in point: the G2′s new score is identical to the rating we gave the iPhone 5C. Does that mean the two devices will provide the same user experience? Hell no. The iPhone is a small-screened iOS device with last year’s hardware meant for the “average consumer,” while the G2 is a spec-packed pseudo-phablet aimed at the Android power user. The experiences they offer couldn’t be more different. But, for the people likely to be considering each product, we think each one stands about a 4-in-5 chance of providing them the experience they’re looking for. Taken in context, then, it makes sense that each phone deserves an 8.4.
Is it a perfect system? No. But it’s the one we’re working with at the moment, and in combination with the Scored For Me card, I think it paints a fairly clear picture for most potential buyers.
To wind this down, I’ll emphasize that this revision deals only with the numeric score. Not a single word of our LG G2 review has been altered. I continue to find a lot to like about the G2, in terms of its excellent camera and display, its spec sheet, and its innovative button arrangement. I also stand behind my criticisms; the phone’s software doesn’t deliver a great experience, and its build quality is mediocre compared with some of the competition (and, more importantly, LG’s previous efforts). But the score was too low, and now that error has been corrected. As the saying goes, “we regret the error.”
Some bullheaded folks out there will no doubt lose respect for us, for not “standing our ground.” I don’t care. I also don’t care that this is a somewhat marginal correction, or that some will find this fix too tardy, or otherwise insufficient. There will always be disagreements. That’s fine.
The important part is that the score didn’t feel right, and now it does. That’s the part I care about. And now that it’s been corrected, I feel better knowing we’re accurately presenting our true impressions of the product, rather than stubbornly defending a score we no longer believe in. And until review scores as a whole are a thing of the past, I’ll live with the satisfaction of getting at least that much right.
Want to share your own impressions of the LG G2, especially in terms of its standing against Samsung’s Galaxy Note 3? Join us for the next episode of the Pocketnow U-Review, broadcasting live on October 3!