KIP publications

 
year 2019
author(s) Jakob Jordan, Mihai A. Petrovici, Oliver Breitwieser, Johannes Schemmel, Karlheinz Meier, Markus Diesmann, Tom Tetzlaff
title Deterministic networks for probabilistic computing
KIP-Nummer HD-KIP 17-104
KIP-Gruppe(n) F9
document type Paper
source Scientific Reports 9, 18303 (2019)
doi 10.1038/s41598-019-54137-7
Abstract (en)

Neural-network models of high-level brain functions such as memory recall and reasoning often rely on the presence of stochasticity. The majority of these models assumes that each neuron in the functional network is equipped with its own private source of randomness, often in the form of uncorrelated external noise. However, both in vivo and in silico, the number of noise sources is limited due to space and bandwidth constraints. Hence, neurons in large networks usually need to share noise sources. Here, we show that the resulting shared-noise correlations can significantly impair the performance of stochastic network models. We further demonstrate that this problem can be overcome by using deterministic recurrent neural networks as sources of stochasticity, exploiting the decorrelating effect of inhibitory feedback. Consequently, even a single recurrent network of a few hundred neurons can serve as a natural noise source for large ensembles of functional networks, each comprising thousands of units. We test the proposed framework for a diverse set of networks with different dimensionalities and entropies. A network reproducing handwritten digits with distinct predefined frequencies demonstrates the practical relevance. Finally, we show that the same design transfers to functional networks of spiking neurons.

bibtex
@article{jordan2019deterministic,
  author   = {Jakob Jordan, Mihai A. Petrovici, Oliver Breitwieser, Johannes Schemmel, Karlheinz Meier, Markus Diesmann, Tom Tetzlaff},
  title    = {Deterministic networks for probabilistic computing},
  journal  = {Scientific Reports},
  year     = {2019},
  volume   = {9},
  number   = {},
  pages    = {},
  month    = {December},
  doi      = {10.1038/s41598-019-54137-7},
  url      = {https://www.nature.com/articles/s41598-019-54137-7}
}
Sample Image
URL arXiv link
URL Scientific Reports
up
KIP - Bibliothek
Im Neuenheimer Feld 227
Raum 3.402
69120 Heidelberg