MLPerf Mobile Inference Benchmark
Authors:
Vijay Janapa Reddi,
David Kanter,
Peter Mattson,
Jared Duke,
Thai Nguyen,
Ramesh Chukka,
Ken Shiring,
Koan-Sin Tan,
Mark Charlebois,
William Chou,
Mostafa El-Khamy,
Jungwook Hong,
Tom St. John,
Cindy Trinh,
Michael Buch,
Mark Mazumder,
Relia Markovic,
Thomas Atta,
Fatih Cakir,
Masoud Charkhabi,
Xiaodong Chen,
Cheng-Ming Chiang,
Dave Dexter,
Terry Heo,
Gunther Schmuelling
, et al. (2 additional authors not shown)
Abstract:
This paper presents the first industry-standard open-source machine learning (ML) benchmark to allow perfor mance and accuracy evaluation of mobile devices with different AI chips and software stacks. The benchmark draws from the expertise of leading mobile-SoC vendors, ML-framework providers, and model producers. It comprises a suite of models that operate with standard data sets, quality metrics…
▽ More
This paper presents the first industry-standard open-source machine learning (ML) benchmark to allow perfor mance and accuracy evaluation of mobile devices with different AI chips and software stacks. The benchmark draws from the expertise of leading mobile-SoC vendors, ML-framework providers, and model producers. It comprises a suite of models that operate with standard data sets, quality metrics and run rules. We describe the design and implementation of this domain-specific ML benchmark. The current benchmark version comes as a mobile app for different computer vision and natural language processing tasks. The benchmark also supports non-smartphone devices, such as laptops and mobile PCs. Benchmark results from the first two rounds reveal the overwhelming complexity of the underlying mobile ML system stack, emphasizing the need for transparency in mobile ML performance analysis. The results also show that the strides being made all through the ML stack improve performance. Within six months, offline throughput improved by 3x, while latency reduced by as much as 12x. ML is an evolving field with changing use cases, models, data sets and quality targets. MLPerf Mobile will evolve and serve as an open-source community framework to guide research and innovation for mobile AI.
△ Less
Submitted 6 April, 2022; v1 submitted 3 December, 2020;
originally announced December 2020.
A latent variable approach to heat load prediction in thermal grids
Authors:
Johan Simonsson,
Khalid Tourkey Atta,
Dave Zachariah,
Wolfgang Birk
Abstract:
In this paper a new method for heat load prediction in district energy systems is proposed. The method uses a nominal model for the prediction of the outdoor temperature dependent space heating load, and a data driven latent variable model to predict the time dependent residual heat load. The residual heat load arises mainly from time dependent operation of space heating and ventilation, and domes…
▽ More
In this paper a new method for heat load prediction in district energy systems is proposed. The method uses a nominal model for the prediction of the outdoor temperature dependent space heating load, and a data driven latent variable model to predict the time dependent residual heat load. The residual heat load arises mainly from time dependent operation of space heating and ventilation, and domestic hot water production. The resulting model is recursively updated on the basis of a hyper-parameter free implementation that results in a parsimonious model allowing for high computational performance. The approach is applied to a single multi-dwelling building in Lulea, Sweden, predicting the heat load using a relatively small number of model parameters and easily obtained measurements. The results are compared with predictions using an artificial neural network, showing that the proposed method achieves better prediction accuracy for the validation case. Additionally, the proposed methods exhibits explainable behavior through the use of an interpretable physical model.
△ Less
Submitted 13 February, 2020;
originally announced February 2020.