CN109376853B - Echo state neural network output axon circuit - Google Patents

Echo state neural network output axon circuit Download PDF

Info

Publication number
CN109376853B
CN109376853B CN201811255479.8A CN201811255479A CN109376853B CN 109376853 B CN109376853 B CN 109376853B CN 201811255479 A CN201811255479 A CN 201811255479A CN 109376853 B CN109376853 B CN 109376853B
Authority
CN
China
Prior art keywords
value
control module
output end
buffer
matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811255479.8A
Other languages
Chinese (zh)
Other versions
CN109376853A (en
Inventor
廖永波
李红梅
李文昌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201811255479.8A priority Critical patent/CN109376853B/en
Publication of CN109376853A publication Critical patent/CN109376853A/en
Application granted granted Critical
Publication of CN109376853B publication Critical patent/CN109376853B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Neurology (AREA)
  • Cable Transmission Systems, Equalization Of Radio And Reduction Of Echo (AREA)
  • Complex Calculations (AREA)

Abstract

An echo state neural network output axon circuit relates to a neural network technology. The invention comprises the following steps: the device comprises a clock signal input end, an X vector register, a first counter, a first control module, a second control module, a first multiplier, an enabling signal generator, an inverse matrix arithmetic unit, a B matrix buffer, a Y vector register, a second counter, a third control module, a fourth control module, a second multiplier and a matrix multiplier, wherein the first control module, the second control module, the third control module and the fourth control module are used for extracting elements with the same sequence numbers as input values of the control ends of the elements from the vector register connected with the input ends of the elements. By adopting the technology of the invention, the calculation of the bit weight is calculated based on the mode of the circuit proposed by the patent, and the input data, the calculation unit, the storage length and the calculation capability are all determined in the circuit, so the bit weight value obtained by the circuit calculation is directly matched with the hardware neural network, and the risk of unmatched software and hardware is solved.

Description

Echo state neural network output axon circuit
Technical Field
The present invention relates to neural network technology.
Background
The echo state network architecture is depicted in fig. 1, where the circles represent memory cells and the squares represent modules. And a teacher signal pair (U1-uL, Y1-yM) is externally input at each moment, the teacher signals are vectors and are respectively stored in the U unit and the Y unit, the random number generator module randomly generates an input weight matrix, a reservoir weight matrix and a feedback weight matrix, and the input weight matrix, the reservoir weight matrix and the feedback weight matrix are respectively stored in the Win unit, the W unit and the Wback unit for subsequent calling. The Reservoir module calculates the intermediate layer state values X1-xK of the network and stores the intermediate layer state values in an X unit. And the training module calculates the output weight and sends the output weight to the Wout unit. yy 1-yyM represent the net actual output vectors.
In the prior art, a general bit weight confirmation method is to calculate confirmed applications through a CPU or a GPU, and then write the calculated bit weight values into a hardware neural network, and this method is limited by the storage bit length of the configurable bit weight of the hardware neural network and the floating point operation bit length of hardware, and has a risk of mismatching of software and hardware interfaces.
Disclosure of Invention
The technical problem to be solved by the invention is to provide an echo state neural network output axon circuit realized in a hardware mode, which has the characteristics of high reliability and high efficiency.
The technical scheme adopted by the invention for solving the technical problems is that the echo state neural network output axon circuit is characterized by comprising the following units:
a clock signal input for receiving a clock signal;
an X vector register for storing an X vector of dimension K;
the input end of the first counter is connected with the clock signal input end, and the first counter is provided with an i value output end and a j value output end and is used for outputting an i value and a j value when receiving a clock trigger signal;
the initial value of the value i is 1, the value K is taken as a period, and the value i is circularly increased by 1 along with each clock trigger signal until the value i is equal to K;
the j value is initially 1, and is increased by 1 when the i value is equal to K each time until j equals K;
the input end of the first control module is connected with the X vector register, the control end of the first control module is connected with the i value output end, and the output end of the first control module is connected with the first buffer;
the input end of the second control module is connected with the X vector register, the control end of the second control module is connected with the j value output end, and the output end of the second control module is connected with the second buffer;
the two input ends of the first multiplier are respectively connected with the first buffer and the second buffer, and the output end of the first multiplier is connected with the D matrix buffer;
the enabling signal generator is connected with the i value output end and the j value output end, the output end of the enabling signal generator is connected with the enabling end of the inverse matrix arithmetic unit, and the enabling signal generator is used for outputting enabling signals to the inverse matrix arithmetic unit when i is equal to K and j is equal to K;
the inverse matrix arithmetic unit is used for carrying out inverse operation on the matrix stored in the D buffer when the enabling signal is received;
the B matrix buffer is connected with the output end of the inverse matrix arithmetic unit and is used for storing the output of the inverse matrix arithmetic unit;
a Y vector register for storing a Y vector of dimension M;
the input end of the second counter is connected with the clock signal input end, and the second counter is provided with an h value output end and a g value output end; the clock trigger signal is used for outputting an h value and a g value when receiving the clock trigger signal;
the initial value of the h value is 1, the M value is taken as a period, and the h value is circularly increased by 1 along with each clock trigger signal until h is equal to M;
the initial value of g is 1, and the value of g is increased by 1 when the value of h is equal to M each time until the value of g is equal to M;
the input end of the third control module is connected with the X vector register, the control end of the third control module is connected with the h value output end, and the output end of the third control module is connected with the third buffer;
the input end of the fourth control module is connected with the Y vector register, the control end of the fourth control module is connected with the g value output end, and the output end of the fourth control module is connected with the fourth buffer;
the two input ends of the second multiplier are respectively connected with the third buffer and the fourth buffer, and the output end of the second multiplier is connected with the a matrix buffer;
the two input ends of the matrix multiplier are respectively connected with the B matrix buffer and the A matrix buffer, and the output end of the matrix multiplier is connected with the C matrix buffer;
the first control module, the second control module, the third control module and the fourth control module are used for extracting elements with the same sequence numbers as input values of the control ends of the elements from a vector register connected with the input ends of the elements.
The clock trigger signal is a clock rising edge.
By adopting the technology of the invention, the calculation of the bit weight is calculated based on the mode of the circuit proposed by the patent, and the input data, the calculation unit, the storage length and the calculation capability are all determined in the circuit, so the bit weight value obtained by the circuit calculation is directly matched with the hardware neural network, and the risk of unmatched software and hardware is solved.
Drawings
Fig. 1 is a diagram of an echo state network architecture.
FIG. 2 is a circuit diagram of a training module according to the present invention.
Detailed Description
See fig. 2.
In fig. 2, the correspondence between the identifiers of the respective parts and the respective parts of the present invention is as follows:
counter 1-first counter
Counter 2-second counter
X-X vector register
Y-Y vector register
Control Module 1-first control Module
Control Module 2-second control Module
Control Module 3-third control Module
Control Module 4-fourth control Module
Temp 1-first buffer
Temp 2-second buffer
Temp 3-third buffer
Temp 4-fourth buffer
A-A matrix buffer
B-B matrix buffer
C-C matrix buffer
D-D matrix buffer
dij-D matrix element register
ahg-A matrix element register
The invention provides a hardware implementation mode of a training module, which specifically comprises the following steps:
a clock signal input for receiving a clock signal;
an X vector register for storing an X vector of dimension K;
the input end of the first counter is connected with the clock signal input end, and the first counter is provided with an i value output end and a j value output end and is used for outputting an i value and a j value when receiving a clock trigger signal;
the initial value of the value i is 1, the value K is taken as a period, and the value i is circularly increased by 1 along with each clock trigger signal until the value i is equal to K; for example, when an initial value i is 1, i is increased by 1 after a clock rising edge is received, until i is K, and then a clock rising edge is received, the value of i is changed from K to the initial value 1, which is equivalent to the cyclic operation of the hands of the clock, and is called "cyclic increase 1".
The j value is initially 1, and is increased by 1 when the i value is equal to K each time until j equals K;
the input end of the first control module is connected with the X vector register, the control end of the first control module is connected with the i value output end, and the output end of the first control module is connected with the first buffer;
the input end of the second control module is connected with the X vector register, the control end of the second control module is connected with the j value output end, and the output end of the second control module is connected with the second buffer;
the two input ends of the first multiplier are respectively connected with the first buffer and the second buffer, and the output end of the first multiplier is connected with the D matrix buffer through the D matrix element register;
the enabling signal generator is connected with the i value output end and the j value output end, the output end of the enabling signal generator is connected with the enabling end of the inverse matrix arithmetic unit, and the enabling signal generator is used for outputting enabling signals to the inverse matrix arithmetic unit when i is equal to K and j is equal to K;
the inverse matrix arithmetic unit is used for carrying out inverse operation on the matrix stored in the D buffer when the enabling signal is received;
the B matrix buffer is connected with the output end of the inverse matrix arithmetic unit and is used for storing the output of the inverse matrix arithmetic unit;
a Y vector register for storing a Y vector of dimension M;
the input end of the second counter is connected with the clock signal input end, and the second counter is provided with an h value output end and a g value output end; the clock trigger signal is used for outputting an h value and a g value when receiving the clock trigger signal;
the initial value of the h value is 1, the M value is taken as a period, and the h value is circularly increased by 1 along with each clock trigger signal until h is equal to M;
the initial value of g is 1, and the value of g is increased by 1 when the value of h is equal to M each time until the value of g is equal to M;
the input end of the third control module is connected with the X vector register, the control end of the third control module is connected with the h value output end, and the output end of the third control module is connected with the third buffer;
the input end of the fourth control module is connected with the Y vector register, the control end of the fourth control module is connected with the g value output end, and the output end of the fourth control module is connected with the fourth buffer;
the two input ends of the second multiplier are respectively connected with the third buffer and the fourth buffer, and the output end of the second multiplier is connected with the A matrix buffer through the A matrix element register;
the two input ends of the matrix multiplier are respectively connected with the B matrix buffer and the A matrix buffer, and the output end of the matrix multiplier is connected with the C matrix buffer;
the first control module, the second control module, the third control module and the fourth control module are used for extracting elements with the same sequence numbers as input values of the control ends of the elements from a vector register connected with the input ends of the elements.

Claims (2)

1. An echo state neural network output axon circuit, comprising the following units:
a clock signal input for receiving a clock signal;
an X vector register for storing an X vector of dimension K;
the input end of the first counter is connected with the clock signal input end, and the first counter is provided with an i value output end and a j value output end and is used for outputting an i value and a j value when receiving a clock trigger signal;
the initial value of the value i is 1, the value K is taken as a period, and the value i is circularly increased by 1 along with each clock trigger signal until the value i is equal to K;
the j value is initially 1, and is increased by 1 when the i value is equal to K each time until j equals K;
the input end of the first control module is connected with the X vector register, the control end of the first control module is connected with the i value output end, and the output end of the first control module is connected with the first buffer;
the input end of the second control module is connected with the X vector register, the control end of the second control module is connected with the j value output end, and the output end of the second control module is connected with the second buffer;
the two input ends of the first multiplier are respectively connected with the first buffer and the second buffer, and the output end of the first multiplier is connected with the D matrix buffer;
the enabling signal generator is connected with the i value output end and the j value output end, the output end of the enabling signal generator is connected with the enabling end of the inverse matrix arithmetic unit, and the enabling signal generator is used for outputting enabling signals to the inverse matrix arithmetic unit when i is equal to K and j is equal to K;
the inverse matrix arithmetic unit is used for carrying out inverse operation on the matrix stored in the D buffer when the enabling signal is received;
the B matrix buffer is connected with the output end of the inverse matrix arithmetic unit and is used for storing the output of the inverse matrix arithmetic unit;
a Y vector register for storing a Y vector of dimension M;
the input end of the second counter is connected with the clock signal input end, and the second counter is provided with an h value output end and a g value output end; the clock trigger signal is used for outputting an h value and a g value when receiving the clock trigger signal;
the initial value of the h value is 1, the M value is taken as a period, and the h value is circularly increased by 1 along with each clock trigger signal until h is equal to M;
the initial value of the g is 1, and the value of the g is increased by 1 when the value of h is equal to M each time until the value of g is equal to M;
the input end of the third control module is connected with the X vector register, the control end of the third control module is connected with the h value output end, and the output end of the third control module is connected with the third buffer;
the input end of the fourth control module is connected with the Y vector register, the control end of the fourth control module is connected with the g value output end, and the output end of the fourth control module is connected with the fourth buffer;
the two input ends of the second multiplier are respectively connected with the third buffer and the fourth buffer, and the output end of the second multiplier is connected with the A matrix buffer;
the two input ends of the matrix multiplier are respectively connected with the B matrix buffer and the A matrix buffer, and the output end of the matrix multiplier is connected with the C matrix buffer;
the first control module, the second control module, the third control module and the fourth control module are used for extracting elements with the same sequence numbers as input values of the control ends of the elements from a vector register connected with the input ends of the elements.
2. The echo state neural network output axon circuit of claim 1, wherein the clock trigger signal is a clock rising edge.
CN201811255479.8A 2018-10-26 2018-10-26 Echo state neural network output axon circuit Active CN109376853B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811255479.8A CN109376853B (en) 2018-10-26 2018-10-26 Echo state neural network output axon circuit

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811255479.8A CN109376853B (en) 2018-10-26 2018-10-26 Echo state neural network output axon circuit

Publications (2)

Publication Number Publication Date
CN109376853A CN109376853A (en) 2019-02-22
CN109376853B true CN109376853B (en) 2021-09-24

Family

ID=65389862

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811255479.8A Active CN109376853B (en) 2018-10-26 2018-10-26 Echo state neural network output axon circuit

Country Status (1)

Country Link
CN (1) CN109376853B (en)

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1981296A (en) * 2004-04-15 2007-06-13 神经科学研究基金会 Mobile brain-based device for use in a real world environment
CN103678257A (en) * 2013-12-20 2014-03-26 上海交通大学 Positive definite matrix floating point inversion device based on FPGA and inversion method thereof
CN105701540A (en) * 2016-01-11 2016-06-22 清华大学 Self-generated neural network construction method
CN106200655A (en) * 2016-06-27 2016-12-07 西安交通大学 The FPGA implementation method of BTT guided missile Neural Network Inversion automatic pilot
CN106528047A (en) * 2015-10-08 2017-03-22 上海兆芯集成电路有限公司 Neuro processing unit of selectively writing starting function output or accumulator value in neuro memory
CN106650923A (en) * 2015-10-08 2017-05-10 上海兆芯集成电路有限公司 Neural network elements with neural memory and neural processing unit array and sequencer
EP3182339A1 (en) * 2015-12-17 2017-06-21 Vrije Universiteit Brussel Reservoir computing device
CN107229967A (en) * 2016-08-22 2017-10-03 北京深鉴智能科技有限公司 A kind of hardware accelerator and method that rarefaction GRU neutral nets are realized based on FPGA
CN107862379A (en) * 2017-07-21 2018-03-30 电子科技大学 Neutral net FPGA
CN108090560A (en) * 2018-01-05 2018-05-29 中国科学技术大学苏州研究院 The design method of LSTM recurrent neural network hardware accelerators based on FPGA
CN108429573A (en) * 2018-03-02 2018-08-21 合肥工业大学 A kind of control method for the MMSE detection circuits hidden based on the time
CN108564169A (en) * 2017-04-11 2018-09-21 上海兆芯集成电路有限公司 Hardware processing element, neural network unit and computer usable medium
CN108629407A (en) * 2017-03-23 2018-10-09 意法半导体有限公司 Integrated form artificial neuron element apparatus
CN108629404A (en) * 2017-03-23 2018-10-09 意法半导体有限公司 Circuit is not answered for integrate artificial neuron component
EP3451239A1 (en) * 2016-04-29 2019-03-06 Cambricon Technologies Corporation Limited Apparatus and method for executing recurrent neural network and lstm computations

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11501131B2 (en) * 2016-09-09 2022-11-15 SK Hynix Inc. Neural network hardware accelerator architectures and operating method thereof
US11188818B2 (en) * 2017-04-03 2021-11-30 International Business Machines Corporation Reservoir computing system

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1981296A (en) * 2004-04-15 2007-06-13 神经科学研究基金会 Mobile brain-based device for use in a real world environment
CN103678257A (en) * 2013-12-20 2014-03-26 上海交通大学 Positive definite matrix floating point inversion device based on FPGA and inversion method thereof
CN106528047A (en) * 2015-10-08 2017-03-22 上海兆芯集成电路有限公司 Neuro processing unit of selectively writing starting function output or accumulator value in neuro memory
CN106650923A (en) * 2015-10-08 2017-05-10 上海兆芯集成电路有限公司 Neural network elements with neural memory and neural processing unit array and sequencer
EP3182339A1 (en) * 2015-12-17 2017-06-21 Vrije Universiteit Brussel Reservoir computing device
CN105701540A (en) * 2016-01-11 2016-06-22 清华大学 Self-generated neural network construction method
EP3451239A1 (en) * 2016-04-29 2019-03-06 Cambricon Technologies Corporation Limited Apparatus and method for executing recurrent neural network and lstm computations
CN106200655A (en) * 2016-06-27 2016-12-07 西安交通大学 The FPGA implementation method of BTT guided missile Neural Network Inversion automatic pilot
CN107229967A (en) * 2016-08-22 2017-10-03 北京深鉴智能科技有限公司 A kind of hardware accelerator and method that rarefaction GRU neutral nets are realized based on FPGA
CN108629407A (en) * 2017-03-23 2018-10-09 意法半导体有限公司 Integrated form artificial neuron element apparatus
CN108629404A (en) * 2017-03-23 2018-10-09 意法半导体有限公司 Circuit is not answered for integrate artificial neuron component
CN108564169A (en) * 2017-04-11 2018-09-21 上海兆芯集成电路有限公司 Hardware processing element, neural network unit and computer usable medium
CN107862379A (en) * 2017-07-21 2018-03-30 电子科技大学 Neutral net FPGA
CN108090560A (en) * 2018-01-05 2018-05-29 中国科学技术大学苏州研究院 The design method of LSTM recurrent neural network hardware accelerators based on FPGA
CN108429573A (en) * 2018-03-02 2018-08-21 合肥工业大学 A kind of control method for the MMSE detection circuits hidden based on the time

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
An FPGA Based Real Time Reservoir Computing System for Neuromorphic Processors;Y. Liao等;《2018 3rd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS)》;20180920;第1-13页 *
Analog hardware implementation of spike-based delayed feedback reservoir computing system;J. Li等;《2017 International Joint Conference on Neural Networks (IJCNN)》;20171231;第3439-3446页 *
Region-specific integration of embryonic stem cell-derived neuronal precursors into a pre-existing neuronal circuit;Neuser F等;《PloS one》;20130620;第8卷(第6期);第82-86页 *
Reservoir Computing optimisation for PEM fuel cell fault diagnostic;Morando S等;《2017 IEEE Vehicle Power and Propulsion Conference (VPPC)》;20171231;第1-7页 *
基于l_2正则化回声状态网络的模拟电路故障诊断[;王洪等;《电子器件》;20171231;第40卷(第5期);第1283-1286页 *

Also Published As

Publication number Publication date
CN109376853A (en) 2019-02-22

Similar Documents

Publication Publication Date Title
US12014272B2 (en) Vector computation unit in a neural network processor
US20220366255A1 (en) Neural Network Processor
EP3539059B1 (en) Performing kernel striding in hardware
CN107918794A (en) Neural network processor based on computing array
WO2018165514A1 (en) Transposing neural network matrices in hardware
CN107766935B (en) Multilayer artificial neural network
WO2020118608A1 (en) Deconvolutional neural network hardware acceleration method, apparatus, and electronic device
US20220164663A1 (en) Activation Compression Method for Deep Learning Acceleration
CN107423816A (en) A kind of more computational accuracy Processing with Neural Network method and systems
CN111767994B (en) Neuron computing device
CN115880132B (en) Graphics processor, matrix multiplication task processing method, device and storage medium
CN109376853B (en) Echo state neural network output axon circuit
CN109299487A (en) Neural network model, accelerator, modeling method and device, medium and system
CN110378469B (en) SCNN inference device based on asynchronous circuit, PE unit, processor and computer equipment thereof
CN115952847A (en) Processing method and processing device of neural network model
CN111291871B (en) Computing device and related product
JP5436327B2 (en) Electronic circuit and wiring system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant