KIP publications

year 2021
author(s) Bernhard Klein, Lisa Kuhn, Johannes Weis, Arne Emmel, Yannik Stradmann, Johannes Schemmel, Holger Fröning
title Towards Addressing Noise and Static Variations of Analog Computations using Efficient Retraining
KIP-Nummer HD-KIP 21-103
KIP-Gruppe(n) F9
document type Paper
doi 10.1007/978-3-030-93736-2_32
Abstract (en)

One of the most promising technologies to solve the energy efficiency problem for artificial neural networks on embedded systems is analog computing, which, however, is fraught with noise due to summations of unwanted or disturbing energy, and static variations related to manufacturing. While these inaccuracies can have a negative effect on the accuracy, in particular for naively deployed networks, the robustness of the networks can be significantly enhanced by a retraining procedure that considers the particular hardware instance. However, this hardware-in-the-loop retraining is very slow and thus often the bottleneck hindering the development of larger networks. Furthermore, it is hardware-instance-specific and requires access to the instance in question.

Therefore, we propose a representation of a hardware instance in software, based on simple, parallelization-friendly software structures, which could replace the hardware for the major fraction of retraining. The representation is based on lookup tables, splines as interpolated functions and additive Gaussian noise to cover static variations together with electrical noise of the multiplier array and column-wise integrators. The combined approach using the proposed representation together with some final epochs of hardware-in-the-loop retraining reduces the overall training time from over 10 hours to less than 2 hours compared to a full hardware-in-the-loop retraining, while notably increasing accuracy. This work highlights that including device-specific static variations and noise in the training process is essential for a time-efficient hardware-aware network training for analog computations, and that major parts can be extracted from the hardware instance and represented with simple and efficient software structures. This work is the first step towards hardware-specific but hardware-inaccessible training, addressing speed and accuracy.

  author   = {Klein, Bernhard and Kuhn, Lisa and Weis, Johannes and Emmel, Arne and Stradmann, Yannik and Schemmel, Johannes and Fr{\"o}ning, Holger},
  title    = {Towards Addressing Noise and Static Variations of Analog Computations Using Efficient Retraining},
  journal  = {Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year     = {2021},
  volume   = {},
  pages    = {409--420},
  doi      = {10.1007/978-3-030-93736-2_32}
Datei pdf
KIP - Bibliothek
Im Neuenheimer Feld 227
Raum 3.402
69120 Heidelberg