Introducing MLPerf for Community-driven Benchmarking of Embedded Vision Systems

Wednesday, May 22, 11:20 AM - 11:50 AM
Summit Track: 
Technical Insights II
Mission City M1-M3

Rigorous and systematic benchmarking is key to developing efficient and reliable computer systems. While a solid methodology is necessary, it is not sufficient without representative workloads. In this talk, we describe the progress of developing MLPerf (, a new suite of machine learning workloads contributed to by a wide community across industry and academia. In particular, we focus on vision workloads being included in the edge inference division, targeting mobile and embedded systems. We describe key metrics for initially selected vision tasks (image classification, object detection), models and datasets; describe reference implementations and workflows, with sample data; and provide guidelines for submitters and evaluators. We also present insights on how decision makers may interpret MLPerf results to accelerate innovation in efficient systems for machine vision.


Anton Lokhmotov

CEO, Dividiti

Dr. Anton Lokhmotov is a researcher, engineer and entrepreneur. As an engineering lead and manager at Arm in 2010-2015, Dr. Lokhmotov helped create the world’s first full-profile GPU acceleration implementation for mobile and embedded devices. Dr. Lokhmotov co-founded dividiti in 2015 to pursue a vision of efficient, reliable and cheap computing everywhere, from tiny sensors to supercomputers. As CEO, he led dividiti from developing its open-source Collective Knowledge technology for collaborative and reproducible R&D ( to $1M+ in revenue with Fortune 50 customers. He obtained a PhD in Computer Science from the University of Cambridge in 2008.

Vijay Janapa Reddi

Associate Professor, Harvard University

Vijay Janapa Reddi is an Associate Professor in the John A. Paulson School of Engineering and Applied Sciences at Harvard University. His research interests are in the area of computer architecture, compilers and runtime systems, specifically in the context of mobile and edge computing platforms. Vijay completed his Ph.D. at Harvard University in 2010.

See you at the Summit! May 20-23 in Santa Clara, California!
Register today and reserve your hotel room!