This site may earn chapter commissions from the links on this folio. Terms of use.

UL, which owns 3DMark, has issued its ain argument on Huawei's benchmarking practices after the company was discovered to be artificially inflating its benchmark operation by pushing its SoCs out of thermal bounds when benchmarks were added to a whitelist that allowed the SoC to run at clocks that actually lucifer those listed on a phone's packaging. In all other cases, devices run far below their listed clocks to stay within thermal limits.

Huawei may take come with this idea as a ways to avoid being accused of overclocking its hardware — you tin't argue something is overclocked if information technology's only hitting its specified frequencies — just the larger problem is that the visitor's hardware tweaks completely blow the power budget and simply apply to whitelisted benchmarks.

Huawei and UL accept had several conversations on this topic in which Huawei attempted to explain that this wasn't cheating, simply reflected new AI optimizations being used to ensure maximum performance. At the aforementioned time, however, these AI optimizations are simply beingness used when the handset is benchmarked in specific tests as opposed to in games — and they're pushing the devices and then far out of bounds, they're overheating in some cases. Since the entire point of a benchmark is to capture useful information about the performance of a device, a benchmark mode that simply runs when the device is being tested is the exact opposite of helpful.

UL writes:

UL understands the intent of Huawei'due south approach, just is opposed to forcing the use of a "Operation Manner" past default when a benchmarking application is detected by the device. UL rules crave a device to run the criterion every bit if it were any other application.

Huawei respects consumers' right to choose what to do with their devices. Therefore, Huawei will provide users with open access to "Operation Mode" in EMUI nine.0, so that the user can choose when to apply the maximum power of their device…

To preclude confusion around current benchmarking results, subsequently word, UL and Huawei take temporarily delisted the benchmark scores of a range of Huawei devices, and will reinstate them subsequently Huawei grants all users of Huawei handsets admission to the Performance Style.

This kind of problem is far from new in mobile benchmarking, and while there'due south bear witness that Huawei is indeed trying to develop a new approach to improving its GPU performance through the use of AI, what the company did here appears far more than like a standard whitelist play a trick on to boost functioning in the apps people use to test phones by blowing the SoC's power curve. The device can't practically operate for any length of time while drawing 8W of power, and that's what matters — far more than than whatever question of AI optimization or feature usefulness.

Now Read: Samsung Goes Legit, Stops Cheating on Android Benchmarks, Tests Appear to Bear witness the OnePlus V Cheating on Tests, and Lies, Damned Lies, and Benchmarks