System disruptions
We are currently experiencing disruptions on the search portals due to high traffic. We are working to resolve the issue, you may temporarily encounter an error message.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Synaptic Delays for Insect-Inspired Temporal Feature Detection in Dynamic Neuromorphic Processors
Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Embedded Internet Systems Lab.ORCID iD: 0000-0001-5662-825X
Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Embedded Internet Systems Lab.ORCID iD: 0000-0003-1024-5821
2020 (English)In: Frontiers in Neuroscience, ISSN 1662-4548, E-ISSN 1662-453X, Vol. 14, article id 150Article in journal (Refereed) Published
Abstract [en]

Spiking neural networks are well suited for spatiotemporal feature detection and learning, and naturally involve dynamic delay mechanisms in the synapses, dendrites and axons. Dedicated delay neurons and axonal delay circuits have been considered when implementing such pattern recognition networks in dynamic neuromorphic processors. Inspired by an auditory feature detection circuit in crickets, featuring a delayed excitation by postinhibitory rebound, we investigate disynaptic delay elements formed by inhibitory--excitatory pairs of dynamic synapses. We configured such disynaptic delay elements in the DYNAP-SE neuromorphic processor and characterized the distribution of delayed excitations resulting from device mismatch. Interestingly, we found that the disynaptic delay elements can be configured such that the timing and magnitude of the delayed excitation depend mainly on the efficacy of the inhibitory and excitatory synapses, respectively, and that a neuron with multiple delay elements can be tuned to respond selectively to a specific pattern. Furthermore, we present a network with one disynaptic delay element that mimics the auditory feature detection circuit of crickets, and we demonstrate how varying synaptic weights, input noise and processor temperature affect the circuit. Dynamic delay elements of this kind open up for synapse level temporal feature tuning with configurable delays of up to 100 ms.

Place, publisher, year, edition, pages
Frontiers Media S.A., 2020. Vol. 14, article id 150
Keywords [en]
pattern recognition, Spiking neural network (SNN), Neuromorphic, delay line, embedded intelligence, DYNAP, Insect-inspired computing
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electronic systems
Identifiers
URN: urn:nbn:se:ltu:diva-77666DOI: 10.3389/fnins.2020.00150ISI: 000525052600001PubMedID: 32180698Scopus ID: 2-s2.0-85082714457OAI: oai:DiVA.org:ltu-77666DiVA, id: diva2:1392843
Funder
The Kempe Foundations, JCK-1809The Kempe Foundations, SMK-1429The Swedish Foundation for International Cooperation in Research and Higher Education (STINT), IG2011-2025
Note

Validerad;2020;Nivå 2;2020-04-14 (alebob)

Available from: 2020-02-13 Created: 2020-02-13 Last updated: 2023-09-05Bibliographically approved
In thesis
1. Using Inhomogeneous Neuronal–Synaptic Dynamics for Spatiotemporal Pattern Recognition in Neuromorphic Processors
Open this publication in new window or tab >>Using Inhomogeneous Neuronal–Synaptic Dynamics for Spatiotemporal Pattern Recognition in Neuromorphic Processors
2021 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Mixed-signal neuromorphic processors emulate the electrochemical dynamics of neurons and synapses using conventional analog CMOS-transistor technology and have potential for ultra-low-power machine learning and inference. However, the energy-efficiency of such systems is dependent on sparse, time-based information encoding and processing, and they are, furthermore, subject to imprecision from “device mismatch” in the analog circuitry. Hence, there is a need for methods for neuromorphic computing based on principles of dynamic neural processing for efficient use and programming of these low-power but inhomogeneous systems.

In this thesis, inspiration is drawn from a temporal feature-detection circuit in crickets for the design of a disynaptic delay element—based on excitatory–inhibitory balance—which induces neuronal excitation-delays for resource-efficient coincidence-based pattern recognition. Due to the inhomogeneous dynamics, such disynaptic elements generate a distribution of temporal delays when implemented in mixed-signal hardware, both between and within single neurons. Here, this is utilized as a source of the variability needed for spatiotemporal information representation for processing and learning—as a resource-efficient alternative to dedicated axonal or neuronal delays or emulation of dendritic dynamics.

In experiments with a DYNAP-SE neuromorphic processor, connected in a closed loop with a PC and a digital oscilloscope, disynaptic delays of up to 100 ms were characterized, with an intraneuronal variability of order 10 ms. Using the disynaptic delays, spatiotemporal receptive fields with up to five dimensions per hardware neuron were investigated in a Spatiotemporal Correlator (STC) type neural network, as well as in some simple networks inspired by the auditory system of crickets. The energy dissipation of the balanced synaptic elements is one order of magnitude lower per lateral connection (0.65 nJ vs 9.6 nJ per spike) than the original hardware implementation of the STC.

Thus, it is shown how the inhomogeneous synaptic circuits could be utilized for resource-efficient implementation of STC-type network layers, in a way that enables synapse-address reprogramming as a discrete mechanism for feature tuning. The presented approach may serve as a complement to more accurate but resource-intensive delay-based coincidence detection or dendritic integration, offering a digital network-state representation and adaptation concept that can fully benefit from the inhomogeneous analog neurosynaptic dynamics in the forward pass.

Place, publisher, year, edition, pages
Luleå University of Technology, 2021
Series
Licentiate thesis / Luleå University of Technology, ISSN 1402-1757
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Cyber-Physical Systems
Identifiers
urn:nbn:se:ltu:diva-85267 (URN)978-91-7790-889-0 (ISBN)978-91-7790-890-6 (ISBN)
Presentation
2021-10-25, E632, Luleå tekniska universitet, Luleå, 10:00 (English)
Opponent
Supervisors
Available from: 2021-06-11 Created: 2021-06-11 Last updated: 2023-09-05Bibliographically approved
2. Event-Driven Architectures for Heterogeneous Neuromorphic Computing Systems
Open this publication in new window or tab >>Event-Driven Architectures for Heterogeneous Neuromorphic Computing Systems
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Händelsedrivna arkitekturer för heterogena neuromorfa datorsystem
Abstract [en]

Mixed-signal neuromorphic processors have brain-like organization and device physics optimized for emulation of spiking neural networks (SNNs), and offer an energy-efficient alternative for implementing artificial intelligence in applications where deep learning based on conventional digital computing is unfeasible or unsustainable. However, efficient use of such hardware requires appropriate configuration of its inhomogeneous, analog neurosynaptic circuits, with methods for sparse, spike-timing-based information encoding and processing. Furthermore, as neuromorphic processors are event-driven and asynchronous with massively parallel dynamic processing and colocated memory, they differ fundamentally from conventional von Neumann computers. Therefore, there is a need to investigate programming approaches and learning mechanisms for efficient use of neuromorphic processors, as well as abstractions required for large-scale integration of such devices into the present computational infrastructure of distributed digital systems. In this thesis, a disynaptic excitatory–inhibitory (E–I) element for resource-efficient generation of synaptic delay dynamics for spike-timing-based computation in neuromorphic hardware is proposed. Chip-in-the-loop experiments with a DYNAP-SE neuromorphic processor and SNN simulations are presented, demonstrating how such E–I elements leverage hardware inhomogeneity for representational variance and feature tuning for time-dependent pattern recognition. Using the E–I elements, spatiotemporal receptive fields with up to five dimensions per hardware neuron were characterized, for instance in a modified Spatiotemporal Correlator (STC) network and in an insect-inspired SNN. The energy dissipation of the proposed E–I element is one order of magnitude lower per lateral connection (0.65 vs. 9.6 nJ per spike) than the original delay-based hardware implementation of the STC. Thus, it is shown how the analog synaptic circuits could be used for efficient implementation of STC network layers, in a way that enables digital synapse-address reprogramming as an observable and reproducible mechanism for feature tuning in SNN layers. This approach may serve as a complement to more accurate but resource-intensive delay-based SNNs, as it offers a digital network-state representation and adaptation concept that can fully benefit from the inhomogeneous neurosynaptic dynamics in the inference stage. Furthermore, a microservice-based conceptual framework for neuromorphic systems integration is proposed. The framework consists of a neuromorphic-system proxy that provides virtualization and communication capabilities required in distributed settings, combined with a declarative programming approach offering engineering-process abstraction. By combining several well-established concepts from different domains of computer science, this work addresses the gap between the state of the art in digitization and neuromorphic computing software development.

Place, publisher, year, edition, pages
Luleå: Luleå tekniska universitet, 2023
Series
Doctoral thesis / Luleå University of Technology 1 jan 1997 → …, ISSN 1402-1544
Keywords
Neuromorphic computing, Mixed-signal, Low-power, Non-von Neumann, Spatiotemporal pattern recognition, System integration
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Cyber-Physical Systems
Identifiers
urn:nbn:se:ltu:diva-94116 (URN)978-91-8048-219-6 (ISBN)978-91-8048-220-2 (ISBN)
Public defence
2023-02-06, A109, Luleå tekniska universitet, Luleå, 13:00 (English)
Opponent
Supervisors
Funder
The Kempe Foundations, JCK-1809EU, Horizon 2020, 737459
Available from: 2022-11-21 Created: 2022-11-21 Last updated: 2023-09-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textPubMedScopus

Authority records

Sandin, FredrikNilsson, Mattias

Search in DiVA

By author/editor
Sandin, FredrikNilsson, Mattias
By organisation
Embedded Internet Systems Lab
In the same journal
Frontiers in Neuroscience
Other Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
pubmed
urn-nbn

Altmetric score

doi
pubmed
urn-nbn
Total: 547 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf