Samsung, Broadcom, Huawei, and other companies last week formed MobileBench, a new consortium which intends to develop, standardise, and promote better hardware and system-level benchmarking resources for mobile devices like smartphones and tablets.
Joining the three firms as founding members of MobileBench were consumer electronics manufacturer OPPO and fabless semiconductor firm Spreadtrum, both of which are based in China.
MobileBench will "initially focus on addressing the need of developers to better understand every element within the mobile platform and deliver optimal system-level performance to enhance the user experience," the new consortium said following its inaugural meeting in Shenzhen, China.
"The MobileBench Consortium will make significant inroads in providing objective ways to evaluate a mobile device's performance in the real world. Together with other industry leaders, we will standardize many of the core benchmarking tools that developers and engineers use, as we accelerate innovation in the development of mobile applications," Rafael Sotomayor, vice president of Mobile Platform Solutions at Broadcom, said in a statement.
The consortium's first benchmark testing tool is the eponymous "MobileBench," which the founding firms demoed in China last week. This "comprehensive program designed for engineers and system designers" includes tools for evaluating hardware components used in mobile devices, including application processors, embedded storage, memory chips, and more. A separate suite of benchmark testing tools called MobileBench-UX can be used to assess system-level applications.
MobileBench members said the consortium intends to "expand mobile testing from simply assessing individual components and other conventional factors to include more sophisticated test focuses such as video/image file viewing, video filming, and other specific mobile modes and services to which users can clearly relate."
The founding companies also said they plan to develop testing tools which could be used by consumers "to evaluate personal mobile devices on their own."
Benchmarking PC components and software is old hat, but the need for better, standardised tools to assess mobile devices and device components has become apparent. A few months ago, controversy erupted over results from one lab using the AnTuTu Benchmark which suggested that Intel's latest Atom Z2580 System-on-a-Chip (SoC) wildly outperformed ARM-based SoCs from Qualcomm, Samsung, and Nvidia while using significantly less power to do so.
It turned out that AnTuTu had some issues revolving around a compiler that apparently lets x86 CPU cores skip some steps in certain testing processes while making ARM chips perform them. The AnTuTu Benchmark wound up getting fixed to clean up this discrepancy, but the controversy underscored the need for more mature benchmarking tools for mobile device components and software.