Benchmark tests were never the most reliable indicator of a phone’s performance, but this year has really seen benchmarks losing more and more of what worth they still had. A number of Android manufacturers have been caught red-handed configuring their phones to artificially dial-up performance when they detected benchmark software was being run, rather than organically governing things like clock speed and core use based on demand. We’ve seen some attempts to mitigate such actions, like obscuring the package name of affected benchmark apps, but one company behind such an app has a much more impactful solution in mind: Futuremark, maker of 3DMark, has decided to delist scores from devices that appear to be breaking its rules.
What are those rules? While the full answer is slightly more nuanced, the gist of it is spelled-out perfectly clear: “benchmark specific optimizations are not allowed.”
That means that popular devices like the Samsung Galaxy Note III and HTC One now appear at the bottom of Futuremark’s ranking list, stripped of their scores. It may be harsh, but if OEMs are already going to such underhanded methods to boost their benchmark rankings, maybe the only way to correct their behavior really is to wholly remove any incentive to cheat, as Futuremark’s doing here.