KIP publications

 
year 2019
author(s) Dominik Dold, Akos F. Kungl, Joćo Sacramento, Mihai A. Petrovici, Kaspar Schindler, Jonathan Binas, Yoshua Bengio, Walter Senn
title Lagrangian dynamics of dendritic microcircuits enables real-time backpropagation of errors
KIP-Nummer HD-KIP 19-11
KIP-Gruppe(n) F9
document type Paper
Keywords Structured neurons, backpropagation, predictive coding, prospective coding, least action, error-correcting plasticity
source Cosyne abstracts 2019, Lisbon, Portugal
Abstract (en)

A major driving force behind the recent achievements of deep learning is the backpropagation-of-errors algorithm (backprop), which solves the credit assignment problem for deep neural networks. Its effectiveness in abstract neural networks notwithstanding, it remains unclear whether backprop represents
a viable implementation of cortical plasticity. Here, we present a new theoretical framework that uses a
least-action principle to derive a biologically plausible implementation of backprop.

In our model, neuronal dynamics are derived as Euler-Lagrange equations of a scalar function (the Lagrangian). The resulting dynamics can be interpreted as those of multi-compartment neurons with
apical and basal dendrites, coupled with a Hodgkin-Huxley-like activation mechanism that undoes temporal
delays introduced by finite membrane time constants. We suggest that a neuron’s apical potential
encodes a local prediction error arising from the difference between top-down feedback from higher cortical
areas and the bottom-up prediction represented by activity in its home layer. This computation is enabled
by a stereotypical cortical microcircuit, projecting from pyramidal neurons to interneurons back to
the pyramidal neurons’ apical compartments. When a subset of output neurons is slightly nudged towards
a target behavior that cannot be explained away by bottom-up predictions, an error signal is induced that
propagates back throughout the network through feedback connections. By defining synaptic dynamics as
gradient descent on the Lagrangian, we obtain a biologically plausible plasticity rule that acts on the
forward projections of pyramidal and interneurons in order to reduce this error.

The presented model incorporates several features of biological neurons that cooperate towards approximating a time-continuous version of backprop, where plasticity acts at all times to reduce an output error induced by mismatch between different information streams in the network. The model is not only restricted to supervised learning, but can also be applied to unsupervised and reinforcement learning schemes, as demonstrated in simulations.

bibtex
@inproceedings{dold2019lagrangian,
  author   = {Dominik Dold, Akos F. Kungl, Joćo Sacramento, Mihai A. Petrovici, Kaspar Schindler, Jonathan Binas, Yoshua Bengio, Walter Senn},
  title    = {Lagrangian dynamics of dendritic microcircuits enables real-time backpropagation of errors},
  booktitle = {},
  year     = {2019},
  volume   = {},
  pages    = {}
}
Sample Image
Datei pdf
URL Cosyne 2019
up
KIP - Bibliothek
Im Neuenheimer Feld 227
Raum 3.402
69120 Heidelberg