KIP publications

 
year 2017
author(s) Agnes Korcsak-Gorzo, Luziwei Leng, Oliver Julien Breitwieser, Johannes Schemmel, Karlheinz Meier, Mihai Alexandru Petrovici
title Simulated Tempering in Biologically Inspired Neural Networks
KIP-Nummer HD-KIP 17-133
KIP-Gruppe(n) F9
document type Paper
source Deutsche Physikerinnentagung
Abstract (en)

Inspired by neural structures in the brain, traditional artificial neural networks are widely used in machine learning and have been applied to hard problems such as classification and generation of images and sound. These networks usually consist of abstract units with a stochastic activation function. In our work, the biologically more plausible leaky integrate-and-fire (LIF) neuron is used. In the model, a set of differential equations accounts for biologically realistic membrane dynamics and a neuron spikes deterministically when its membrane potential exceeds a threshold. For inference tasks, Poisson noise is employed as background stimulus of the network to approximate the required stochastic dynamics. We compare our network with a restricted Boltzmann machine in a high dimensional data generation task. Due to the multimodal energy landscape created during learning, conventional algorithms such as Gibbs sampling is prone to get trapped in a local minimum. We show that by changing the background noise, the spiking activity of the LIF neuron can be tuned, leading to a rescaling of the energy landscape in terms of network activity, which resembles the principle of simulated tempering. This facilitates the network to jump out of local minimums and mix fast between different modes.

bibtex
@article{Korcsak2017,
  author   = {Agnes Korcsak-Gorzo, , Luziwei Leng, Oliver  Julien Breitwieser, Johannes Schemmel, Karlheinz Meier, Mihai Alexandru Petrovici},
  title    = {Simulated Tempering in Biologically Inspired Neural Networks},
  journal  = {Deutsche Physikerinnentagung},
  year     = {2017},
  volume   = {},
  pages    = {}
}
URL Programmheft Deutsche Physikerinnentagung
up
KIP - Bibliothek
Im Neuenheimer Feld 227
Raum 3.402
69120 Heidelberg