KIP publications

 
year 2017
author(s) Dominik Dold, Ákos F. Kungl, Andreas Baumbach, Johann Klähn, Ilja Bytschok, Paul Müller, Oliver Breitwieser, Andreas Grübl, Maurice Güttler, Dan Husmann, Mitja Kleider, Christoph Koke, Alexander Kugele, Christian Mauch, Eric Müller, Sebastian Schmitt, Johannes Schemmel, Karlheinz Meier, Mihai A. Petrovici
title Stochastic computation on spiking neuromorphic hardware
KIP-Nummer HD-KIP 17-78
KIP-Gruppe(n) F9
document type Paper
source Bernstein Conference 2017
doi 10.12751/nncn.bc2017.0075
Abstract (en)

In order to cope with ambiguous, incomplete and noisy sensory input, the brain is believed to rely on some form of probabilistic computation. Within this context of "Bayesian brain", it has been suggested to interpret neural firing activity as sampling from a probability distribution. Recently, it was shown that networks of Leaky Integrate-and-Fire (LIF) neurons can approximately sample from Boltzmann distributions with binary random variables when elevated into a high-conductance state via high-frequency Poisson noise which endows LIF neurons with approximately logistic activation functions. Such spiking Boltzmann machines can then be trained from sensory inputs to perform inference in the corresponding data spaces.

Here, we present the first realization of such LIF networks on the BrainScaleS system. BrainScaleS is a mixed-signal neuromorphic device that enables the fast emulation of spiking neural networks with a speedup of 104 compared to biological time. We report that on BrainScaleS, sigmoidal response functions can be reliably set up (Fig. 1A), which is a necessary precondition for training such networks on hardware. During training, parameter updates are calculated on a standard computer between consecutive emulations of the network on hardware. After training, the firing activity of the emulated network approximates the desired target distribution (Fig. 1B); the remaining deviations are due to the limited configurability of hardware parameters, as for instance the 4-bit weight resolution.

In addition, since the external bandwidth of the hardware system is limited, the total amount of available Poisson noise is bounded as well, restricting the maximal number of neurons that can be elevated into the high-conductance state. Inspired by the mammalian cortex, where neurons are exposed to the activity of some 104 presynaptic partners, we demonstrate in simulations that high-frequency Poisson noise can be successfully replaced by the spiking activity of adjacent functional networks (Fig. 1C). This way, networks of networks can be constructed where each neuron only uses the activity from adjacent LIF networks as irregular background input, with zero external Poisson input (Fig. 1D). We believe that this approach will enable the implementation of large-scale networks of deterministic LIF neurons on large-scale neuromorphic systems that can perform stochastic computations without bandwidth-consuming external noise.

bibtex
@conference{dold2017bernstein,
  author   = {Dold, Dominik and Kungl, Ákos F.  and Baumbach, Andreas and Kl\"ahn, Johann and Bytschok, Ilja and M\"uller, Paul and Breitwieser, Oliver and Gr\"ubl, Andreas and G\"uttler, Maurice and Husmann, Dan and Kleider, Mitja and Koke, Christoph and Kugele, Alexander and Mauch, Christian and M\"uller, Eric and Schmitt, Sebastian and Schemmel, Johannes and Meier, Karlheinz and Petrovici, Mihai A.},
  title    = {Stochastic computation on spiking neuromorphic hardware},
  booktitle = {Bernstein Conference 2017},
  year     = {2017},
  volume   = {},
  pages    = {},
  address  = {10.12751/nncn.bc2017.0075}
}
Sample Image
Datei poster
URL abstract
Datei figure
up
KIP - Bibliothek
Im Neuenheimer Feld 227
Raum 3.402
69120 Heidelberg