How to Pick Good Benchmarks & Why They Matter - CES 2015

The video opens with Linus Tech Tips highlighting the importance of benchmarks that are both trustworthy and verifiable. The host explains that there are three basic types of benchmarks: consortium benchmarks, open-source benchmarks, and closed benchmarks. He notes that consortium benchmarks involve a large group of companies that collaborate to keep results fair and balanced, while open benchmarks have community involvement and often affordable or accessible source code to prevent tampering. Closed benchmarks, by contrast, can be costly and restrict access to the underlying code, which reduces transparency and community moderation. The discussion moves to Tablet Mark from Bapco to illustrate why credible benchmarking matters, especially in the tablet and PC segments where slim designs and limited I/O demand strong, verifiable performance data. The presenter emphasizes that Intel is presenting benchmarks that purportedly show a clear lead in the tablet/PC space, positioning their chips as strong performers under real-world workloads such as photo filtering, web browsing, and video playback. He showcases examples like a Broadwell-based Core M chip turboing to 2.6 GHz and explains that one of the showcased benchmarks is a consortium effort while another is an open- source benchmark called CR Expert, which runs on Chrome OS to compare Chromebooks and other devices in a standardized way. The overall goal is to enable more balanced, authoritative reviews by combining different benchmarking approaches and by sourcing data from multiple vendors, not just a single company. The host invites audience feedback on what to benchmark next and promises continued CES coverage with sponsor shout-outs and recommendations for further resources, underscoring the value of transparent testing for informed buying decisions. In later sections, the discussion delves into how the team at Linus Tech Tips plans to expand their benchmarking coverage beyond flagship devices. They describe how the CR Expert open benchmark can compare across platforms like Android, iOS, and Windows, but in the CES context, focuses on Chrome OS devices such as Chromebooks. The segment also covers the practical uses of benchmarks in everyday decisions, such as evaluating battery life and performance for common tasks like web browsing and portfolio tracking. The speakers conclude with a call to the audience to share what devices and scenarios they want tested next, framing benchmarks as a collaborative, ongoing effort to help consumers buy smarter. Overall, the video blends technical explanation with a peek into how large brands influence benchmark design, while advocating for transparency and community input to keep testing fair and useful.

Topics · technology · consumer-electronics · benchmarking · hardware