3DMark has blacklisted testing of a number of Huawei smartphones after a report found that the Chinese phone manufacturer gamed another device’s processor for top performance with benchmark apps.
AnandTech had tested two phones from Huawei as well as a phone from subsidiary Honor this week and found that the P20, P20 Pro and Honor Play were monitoring app types to operate at full power with chipsets at top speeds on benchmark apps and normally for other apps. This meant that benchmark scores across GFXBench and 3DMark were artificially inflated. Further testing done with the benchmark detection circumvented showed each device achieved 40 percent lower scores on average.
Dr. Wang Chenglu, president of software at Huawei’s Consumer Business Group, told the outlet that the company does not trust the current benchmark programs to paint what “real-world use” might produce and wants a standardized program. Wang also acknowledged that it is also competing against other manufacturers in performance marketing.
“Others do the same testing, get high scores and Huawei cannot stay silent,” Wang said.
In reaction to the report, Underwriters Laboratories Benchmarks, which owns 3DMark, has decided to blacklist the Huawei P20, P20 Pro, Nova 3 and Honor Play 3, based on data from AnandTech and its own testing.
At the very least, Huawei’s Dr. Wang told AnandTech that the company would perform benchmarking with third-party verification in the future.
From a wider point of view, benchmark cheating has been done in the same way for years and we don’t see this habit abetting soon.