1. The MegaSim Simulator
Downloading and building MegaSim
You may download the source code from here: https://bitbucket.org/bernabelinares/megasim (~4.3 MB), copy it from my USB flash drive or from the local server
megasim_local (file missing!)
Additionally, you can download the jAER [1] software for playing the generated events.
Building the simulator
There are two makefiles to compile MegaSim, one for macs (Makefile_osx) and one for linux (Makefile_linux):
make –f Makefile_osx
make –f Makefile_linux
then source source.sh to add megasim and the python scripts to your path.
The python scripts folder includes a collection of scripts for generating STDP functions, plot the evolution of kernels interactively and convert MegaSim’s AER events to jAER data and vice versa. The prerequisites for the python scripts are:
Python version = 2.7
Numpy version >= 1.10.4
Matplotlib version >= 1.5.0
Simulator overview
The Modular Event-driven Growing Asynchronous Simulator (MegaSim) is a tool designed for simulating Address-event-representation (AER) systems written in C. In MegaSim the user may define a topology by utilizing modules that process input events, nodes to connect modules with each other and input stimuli sources, for example data recorded from a DVS retina, cochlea, or generated artificially by some algorithm. Modules may have an arbitrary number of input/output ports and can either be population of neurons, algorithms described in C, or a combination of both. Modules communicate through nodes by utilizing the AER events. In MegaSim it is assumed that events are communicated from an output port to an input port through asynchronous communication using request (Rqst) and acknowledgment (Ack) signals, where there would be a delay between Rqst and Ack. A handshake-less link can be emulated by simply defining a zero delay for Ack.
Figure 1. An example topology in MegaSim
It is possible to model clocked networks by defining a clock module that periodically sends update events or model hybrid systems that include clocked updates with asynchronous handshaking.
Upon the completion of a simulation there will be an event file for each node describing all the events that have been communicated through this node. The following section describes the structure of AER events in MegaSim
AER representation in MegaSim
Table 1 shows how AER events are represented in MegaSim. For each event there are 3 timing values, and a number of event parameters typically in the form of X, Y and polarity. However these parameters can be any signed 32-bit integer and the user decides how a module interprets and processes them.
| pre-Rqst | Rqst | Ack | x | y | Polarity |
| 0 | 20 | 40 | 4 | 5 | -1 |
| 17 | 60 | 80 | 5 | 4 | 1 |
| ... | ... | ... | ... | ... | ... |
| 60 | -1 | -1 | 4 | 5 | 1 |
Where pre-Rqst represents the time an event is created inside a module, Rqst the time when this event is actually put on the node and Ack the time when acknowledgment signal is communicated. When an event has been queued in link, Rqst and Ack will be set to -1.
Handling and processing AER events
The simulator is responsible for looking at all nodes and selecting the earliest unprocessed events. Then calls the corresponding destination modules, and each module will process the incoming event, perform handshaking, perform computations at the destination module such as a) update all internal states since last time called, b) compute the effect of an incoming event, and c) if output events are generated they will be written on the output nodes with the present timestamp and then sorted according the their timestamps. This process is iterated until a maximum predefined simulation time, or until there are no more unprocessed events. Upon the completion of a simulation there will be an event file for each node describing all the events that have been communicated through this node.
MegaSim Modules
Currently MegaSim has the following modules:
- Module copy (module_copy): Which takes input events from a number of input ports and after some delay copies them on all output ports. Each event can be copied a number of times on each output port. This module can be used to simulate splitters, mergers or axonal delays.
- Module Convolution (module_conv): This module receives input events through a number of input ports, and for each input port applies a static convolution kernel. The generated output events can be sent to several output ports simultaneously.
- Module Convolution Plastic (module_conv_plastic): This module is identical to module_conv however it includes an extra port for each plastic kernel to receive special AER DW events from the STDP modules and update the weights appropriately.
- Module STDP (module_STDP): This module has two input ports one for the pre and one for the post events. It also has two buffers with user-defined size to store the pre and post events and uses them to compute the DWs based on a user defined STDP function.
- Module order-based STDP (module_STDP_Orderbased): This module also includes two input ports for the pre and post events but has only one buffer that stores the pre events and computes STDP based on the order of events and not their precise timing. This module is based on the work of Roclin et al. 2013 [2].
Module Convolution
The module_conv utilises an event-driven LIF neuron with linear leakage, described in detail in [3]. The state of this neuron is updated based on the current and previous input event. It uses a signed neuron capable of generating spikes with positive and negative polarity by utilising a positive and negative threshold. By disabling the negative threshold, however, it becomes a standard leaky integrate and fire (LIF) neuron model with linear leakage, similarly to [2].
|
.integers n_in_ports N n_out_ports O delay_to_process 1 delay_to_ack 2 fifo_depth 1 n_repeat 1 delay_to_repeat 15 Nx_array 31 Ny_array 31 Xmin 0 Ymin 0 THplus 190454428 THplusInfo 1 THminus -2147483647 THminusInfo 0 Reset_to_reminder 0 MembReset 0 TLplus 2000 TLminus 2000 Tmin 0 T_Refract 10 Nx_kernel Nx Ny_kernel Ny Dx Ndx Dy Ndy <Kernel weights for input port N> crop_xmin 0 crop_xmax 31 crop_ymin 0 crop_ymax 31 xshift_pre 0 yshift_pre 0 x_subsmp 1 y_subsmp 1 xshift_pos 0 yshift_pos 0 rectify 0 .floats |
where n_in_ports and n_out_ports defines the number of input and output ports, delay_to_process refers to the time required to process an incoming event, delay_to_ack delay to acknowledge an incoming event, fifo_depth the FIFO size for incoming events, Nx_array and Ny_array define the width and height of the feature map, Xmin and Ymin define the starting index of the array. THplus is the value of the positive threshold, THplusInfo is the enable/disable flag for the positive threshold, THminus is the negative threshold, THminusInfo is the enable/disable flag for the negative threshold, MembReset is the membrane reset voltage, TLplus and TLminus define the positive and negative linear leakage rate, Tmin defines the minimum time between 2 consecutive spikes, T_Refract is the refractory period.
In addition, for every input port the user must define the following parameters: Nx_kernel and Ny_kernel, which is the width and height of the kernel, Dx and Dy which is the displacement in X and Y of the kernel, and finally the weights of the kernel themselves. The output of the convolution module can be cropped, shifted and subsampled by using the crop_xmin, crop_xmax, crop_ymin, crop_ymax, xshift_pre, yshift_pre, x_subsmp, y_subsmp, xshift_pos, yshift_pos parameters. Finally, the rectify parameter will force the output events to have a positive polarity.
Module STDP
For this module two buffers are used, one for the pre and one for the post events. The user provides a precalculated STDP function that consists of ΔT and ΔW data and linear interpolation is used during the simulation to determine the intermediate values. Figure X shows two example STDP functions, one of which is a piecewise double exponential and a simplified rectangular one.
Figure 2. Two example STDP functions inclue a piecewise double-exponential and a rectangular one, for the module_stdp.
|
.integers n_in_ports 2 n_out_ports 1 delay_to_process 0 delay_to_ack 0 fifo_depth 0 n_repeat 1 delay_to_repeat 15 Nx_array 31 Ny_array 31 Xmin 0 Ymin 0 Nx_kernel 7 Ny_kernel 7 Dx -3 Dy -3 event_buffer_size 100 look_past 100 old_events_to_keep 50 stdp_function_data_points 20 -100 -88 -77 -66 -55 -44 -33 -22 -11 0 0 11 22 33 44 55 66 77 88 100 6 11 20 35 62 108 188 329 573 999 -800 -459 -263 -151 -86 -49 -28 -16 -9 -5. .floats |
where n_in_ports are the number of input ports, with the first port used for presynaptic events and the second port used for postsynaptic events, n_out_ports is set to 1 for the special DW events, Nx_array and Ny_array are the width and height of the plastic feature map, Nx_kernel and Ny_kernel is the width and height of the plastic kernel, Dx and Dy are the displacement in X and Y of the kernel, event_buffer_size defines the pre and post event buffer sizes, look_past parameter describes how many past events in the buffer to consider when applying STDP, old_events_to_keep defines how many old events to keep when the buffers are full, stdp_function_data_points define the number of data points of the STDP function, and the following row defines the DT (pre-post) data points, and the next row the DW data points.
Module Order-based STDP
The module_stdp_orderbased is based on the work or Roclin et al. [2], with a slight modification to account for the shared weights of convolutional networks. In order-based STDP a single buffer is used to store presynaptic events and whenever there is a postsynaptic event only the weights relevant to the events in the buffer are potentiated, while the remaining ones are depressed. In this type of STDP what is important is not the timing of events but the order they appear in the buffer, as seen in Figure . Results have shown that the size of the buffer affects to be roughly the size of the object to be recognised [2]. A special AER event is generated with the X and Y coordinates of the weights of the kernel to be updated and the DW+/- value.
Figure 3. The STDP function of the module_stdp_orderbased.
|
.integers n_in_ports 2 n_out_ports 1 delay_to_process 0 delay_to_ack 0 fifo_depth 0 n_repeat 1 delay_to_repeat 15 Nx_array 31 Ny_array 31 Xmin 0 Ymin 0 Nx_kernel 7 Ny_kernel 7 Dx -3 Dy -3 event_buffer_size 24 look_past 24 old_events_to_keep 23 num_potentiate 15 stdp_function_data_points 2 -200 -24 1000 -100 .floats |
Setting up a Simulation
This section describes how to describe a neural network topology in MegaSim. In this particular example a spiking convolutional layer will be used with 4 precomputed gabor-like kernels as shown in Figure 4.
Figure 4. Gabor kernels
In MegaSim there a netlist file describing the topology of a network, which includes the input sources, modules, nodes and parameters . The netlist file of this particular example can be seen in Table 2.
| 1. | .netlist |
| 2. | . source {input} i.txt |
| 3. | module_conv {input} {l1_1} gabor_01.prm gab_001.stt |
| 4. | module_conv {input} {l1_2} gabor_02.prm gab_002.stt |
| 5. | module_conv {input} {l1_3} gabor_03.prm gab_003.stt |
| 6. | module_conv {input} {l1_4} gabor_04.prm gab_004.stt |
| 7. | |
| 8. | .options |
| 9. | Tmax = 429290101 |
The netlist file begins with the “.netlist” keyword, and the following lines include the MegaSim modules, along with their input/output nodes (in brackets {}) and their configuration and initial states files. In this example, in line 2 we declare an input source, with a node name {input} and an input file with the AER events “i.txt”. Line 3 declares a convolutional module receiving input from the {input} node and has an output node {l1_1} followed by the parameter file “gabor_01.prm” (as described in the MegaSim modules section) and the initial state of the module “gab_001.stt”. In line 8, the options of the simulations are set “.options”, where we can set the maximum simulation time Tmax.
For this example the input data were recorded from DVS sensor and can be seen with jAER by playing the i.dat file. As a first step we are going to convert the DVS data to MegaSim AER data using the following command:
jaer_to_ascii.py i.dat
then execute the simulation:
megasim people_walking_gabor.sch
Convert the output of a convolutional module to jAER data:
convert_to_jaer.py node_l1_1.evs
the output can be played back with jAER, Figure 5, by selecting the es.cnm.imse.jaer.chip.retina.RetinaTeresa2 from the AEChip dropdown menu.
5. jAER output of the first convolutional module.
2. Results
2.1 Development of new MegaSim modules
We developed a number of new modules, for example a module that receives inputs from a number of spiking convolutional modules and flattens them into a vector in order to be fully connected to a population of LIF neurons. The second module is a population of fully connected (FC) LIF neurons with STDP, in order to have a spike-based classifier (module_fully_connected), and finally a merger module that receives the output activity of several populations, merges them, and sends UDP packets of their events to jAER [1].
Figure 6. The module_flatten, module_FC, and module_merger.
We tested the new modules on a simple task in which an input stimulus, of size 16x16, is a bar rotating randomly and is fully connected to a population of 4 LIF neurons with plastic synapses. In order to force each output neuron to respond to a particular orientation we used a population of teacher neurons, as seen in Figure 7.
7. Rotating bars benchmark with STDP and a population of teacher neurons.
Figure 8 shows the receptive fields of the neurons after learning, and the video below show the evolution of the weights during the learning process as well as the activity of the network.
8. The receptive fields of the output neurons after training.
2.2 Porting off-line trained deep nets on MegaSim
For this task we used the methodology described by O' Connor et al. [4] to train a deep belief network (DBN) off-line, with 7 hidden layers, and then map it to a spiking neural network and run it on MegaSim. Results are presented as classification accuracies between Brian, SpiNNaker, and Megasim (Table 6).
Architecture of (a) an RBM and (b) a Deep Belief Network (DBN) with 7 hidden layers.
| Classification accuracy (%) | |
| Brian | 96.21 |
| SpiNNaker | 96.22 |
| MegaSim |
The spiking activity of the DBN on MegaSim can be visualised through jAER:
The activity of the input layer is shown on the bottom left corner. Each pixel of an MNIST digit is converted into spike-trains using Poisson rates, with the rate proportional to the intesity of each pixel. Then the responses of the 7 hidden layers can be seen and on the top is the actual output activity (10 neurons, for digits 0 to 9) and the ground truth (label). The bottom right corner shows the activity of the reconstruction layer.
References
[1] [Online]. Available: http://jaer.wiki.sourceforge.net
[2] D. Roclin, O. Bichler, C. Gamrat, S. J. Thorpe, and J.-O. Klein, “Design study of efficient digital order-based STDP neuron implementations for extracting temporal features,” 2013 Int. Jt. Conf. Neural Networks, pp. 1–7, Aug. 2013.
[3] J. a Perez-Carrasco, B. Zhao, C. Serrano, B. Acha, T. Serrano-Gotarredona, S. Chen, and B. Linares-Barranco, “Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate-Coding. Application to Feedforward ConvNets,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 11, pp. 2706–2719, 2013.
[4] P. O’Connor, D. Neil, S.-C. Liu, T. Delbruck, and M. Pfeiffer, “Real-time classification and sensor fusion with a spiking deep belief network.,” Front. Neurosci., vol. 7, no. October, p. 178, Jan. 2013.