KIP publications

year 2020
author(s) E. Kreutzer, M. A. Petrovici, W. Senn
title Natural gradient learning for spiking neurons
KIP-Nummer HD-KIP 20-17
KIP-Gruppe(n) F9
document type Paper
Keywords (shown) natural gradient descent, parametrization invariance, dendritic learning, heterosynaptic plasticity, efficient learning, homeostasis
source COSYNE 2020, NICE 2020
Abstract (en)

Due to their simplicity and success in machine learning, gradient- based learning rules represent a popular choice for synaptic plastic- ity models. While they have been linked to biological observations, it is often ignored that their predictions generally depend on a specific representation of the synaptic strength. In a neuron, the im- pact of a synapse can be described using the state of many different observables such as neutortransmitter release rates or membrane potential changes. Which one of these is chosen when deriving a learning rule can drastically change the predictions of the model. This is doubly unsatisfactory, both with respect to optimality and from a conceptual point of view. By following the gradient on the manifold of the neuron’s firing distributions instead of one that is relative to some arbitrary synaptic weight parametrization, natural gradient descent provides a solution to both these problems. While the computational advantages of natural gradient are well-studied in ANNs, its predictive power as a model for in-vivo synaptic plas- ticity has not yet been assessed. By formulating natural gradient learning in the context of spiking interactions, we demonstrate how it can improve the convergence speed of spiking networks. Fur- thermore, our approach provides a unified, normative framework for both homo- and heterosynaptic plasticity in structured neurons and predicts a number of related biological phenomena.

  author   = {Kreutzer, Elena and Petrovici, Mihai A. and Senn, Walter},
  title    = {Natural gradient learning for spiking neurons},
  booktitle = {Proceedings of the Neuro-Inspired Computational Elements Workshop},
  year     = {2020},
  volume   = {},
  number   = {15},
  series   = {NICE ’20},
  pages    = {3},
  address  = {New York, NY, USA},
  month    = {3},
  publisher = {Association for Computing Machinery},
  note     = {ISBN: 9781450377188,}
Datei pdf
KIP - Bibliothek
Im Neuenheimer Feld 227
Raum 3.402
69120 Heidelberg