YARP-SpiNNaker-DVS-ATIS

Non-spiking computer vision with event-driven cameras.
YARP is the robot middle-ware used for the iCub robot. We use YARP with AER representation and can send packets of events over standard Ethernet network such that event-based sensors, computation, actuation and visualisation can be distributed among different computers and robots. We have a stereo pair of DVS sensors we brought to capocaccia (unfortunately no iCub) that we want to interface to spinnaker for real-time computation and also with the ATIS sensor.

Algorithms can then be developed for CPU computation and spinnaker computation, for example, one outcome we want is to implement Gabor filters on the spinnaker board for disparity (and depth) computation from the stereo pair. Another project involves synchronisation of audio and visual AER data using YARP time stamping. Another, real-time event processing under fast camera movement.

Anyone that is interested in using our hardware, has experience or interest in our proposed projects, or can even propose their own project within our frame-work is welcome. We also need help from the SpiNNaker guys and ATIS guys to help set-up these portions of our system.

Login to become a member send

Timetable

Day	Time	Location
Tue, 26.04.2016	19:00 - 20:00	Disco

Set-up

Unfortunately we couldn't bring the iCub robot but we have a stereo dvs camera set-up as found in the head of the iCub.

The iCub Robot with DVS cameras

YARP robot middle-ware, combined with our event-driven library is used to process event data over multiple computers distributed on a network. Importantly these modules are not restricted to CPU based computing and at CapoCaccia we will continue the interface between YARP and the SpiNNaker boards. The figure below illustrates our set-up between cameras, CPU-processing modules, SpiNNaker-processing modules and real-time visualisation.

Data Workflow

Results:

Gabor-filtering on the SpiNNaker board:

A single gabor filter is running on the SpiNNaker board that is sensitive to a vertical orientation. When the pen in instead turned sideways the filter no-longer spikes (as seen by the blue dot on the event visualisation). The simple example is used to demonstrate the successful flow of events through the system, and importantly both in and out of the SpiNNaker board.

Disparity:

In addition a disparity module was added to the circle detection, combining both left and right detections of the circle and the disparity is calculated as the horizontal distance between detected centres. The disparity can be used to find the 3D position of the ball given correct stereo camera calibration.

 

Leader

Members