Embodied synaptic plasticity with online reinforcement learning

Jacques Kaiser*, Michael Hoff, Andreas Konle, J. Camilo Vasquez Tieck, David Kappel, Daniel Reichard, Anand Subramoney, Robert Legenstein, Arne Roennau, Wolfgang Maass, Rüdiger Dillmann

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


The endeavor to understand the brain involves multiple collaborating research fields. Classically, synaptic plasticity rules derived by theoretical neuroscientists are evaluated in isolation on pattern classification tasks. This contrasts with the biological brain which purpose is to control a body in closed-loop. This paper contributes to bringing the fields of computational neuroscience and robotics closer together by integrating open-source software components fromthese two fields. The resulting framework allows to evaluate the validity of biologically-plausibe plasticity models in closed-loop robotics environments. We demonstrate this framework to evaluate Synaptic Plasticity with Online REinforcement learning (SPORE), a reward-learning rule based on synaptic sampling, on two visuomotor tasks: reaching and lane following. We show that SPORE is capable of learning to perform policies within the course of simulated hours for both tasks. Provisional parameter explorations indicate that the learning rate and the temperature driving the stochastic processes that govern synaptic learning dynamics need to be regulated for performance improvements to be retained. We conclude by discussing the recent deep reinforcement learning techniques which would be beneficial to increase the functionality of SPORE on visuomotor tasks.

Original languageEnglish
Article number81
Pages (from-to)1-11
Number of pages11
JournalFrontiers in Neurorobotics
Publication statusPublished - 3 Oct 2019


  • Neuromorphic vision
  • Neurorobotics
  • Reinforcement learning
  • Spiking neural networks
  • Synaptic plasticity

ASJC Scopus subject areas

  • Biomedical Engineering
  • Artificial Intelligence


Dive into the research topics of 'Embodied synaptic plasticity with online reinforcement learning'. Together they form a unique fingerprint.

Cite this