IBM simulates 530 billion neurons, 100 trillion synapses on supercomputer

November 19, 2012

A network of neurosynaptic cores derived from long-distance wiring in the monkey brain: Neuro-synaptic cores are locally clustered into brain-inspired regions, and each core is represented as an individual point along the ring. Arcs are drawn from a source core to a destination core with an edge color defined by the color assigned to the source core. (Credit: IBM)

IBM Research – Almaden presented at Supercomputing 2012 last week the next milestone toward fulfilling the ultimate vision of the DARPA’s cognitive computing program, called Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE), according to Dr. Dharmendra S. Modha, Manager, Cognitive Computing, IBM Research – Almaden.

Announced in 2008, DARPA’s SyNAPSE program calls for developing electronic neuromorphic (brain-simulation) machine technology that scales to biological levels, using a cognitive computing architecture with 1010 neurons (10 billion) and 1014 synapses (100 trillion, based on estimates of the number of synapses in the human brain) to develop electronic neuromorphic machine technology that scales to biological levels.”

Simulating 10 billion neurons and 100 trillion synapses on most powerful supercomputer

IBM says it has now accomplished this milestone with its new “TrueNorth” system running on the world’s second-fastest operating supercomputer, the Lawrence Livermore National Lab (LBNL) Blue Gene/Q Sequoia, using 96 racks (1,572,864 processor cores, 1.5 PB memory, 98,304 MPI processes, and 6,291,456 threads).

IBM and LBNL achieved an unprecedented scale of 2.084 billion neurosynaptic cores* containing 53×1010  (530 billion) neurons and 1.37×1014 (100 trillion) synapses running only 1542 times slower than real time.

“We have not built a biologically realistic simulation of the complete human brain,” explains an abstract of the Supercomputing 2012 (SC12) paper (open-access PDF), selected from the 100 SC12 papers as one of the six finalists for the Best Paper Award. “Computation (‘neurons’), memory (‘synapses’), and communication (‘axons,’ ‘dendrites’) are mathematically abstracted away from biological detail toward engineering goals of maximizing function (utility, applications) and minimizing cost (power, area, delay) and design complexity of hardware implementation.”

Neurosynaptic core (credit: IBM)

Two billion neurosynaptic cores

“Previously, we have demonstrated a neurosynaptic core* and some of its applications,” continues the abstract. “We have also compiled the largest long-distance wiring diagram of the monkey brain. Now, imagine a network with over 2 billion of these neurosynaptic cores that are divided into 77 brain-inspired regions with probabilistic intra-region (“gray matter”) connectivity and monkey-brain-inspired inter-region (“white matter”) connectivity.

“This fulfills a core vision of the DARPA SyNAPSE project to bring together nanotechnology, neuroscience, and supercomputing to lay the foundation of a novel cognitive computing architecture that complements today’s von Neumann machines.”

To support TrueNorth, IBM has developed Compass, a multi-threaded, massively parallel functional simulator and a parallel compiler that maps a network of long-distance pathways in the macaque monkey brain to TrueNorth.

* The IBM-Cornell neurosynaptic core is a key building block of a modular neuromorphic architecture, according to Modha. The core incorporates central elements from neuroscience, including 256 leaky integrate-and-fire neurons, 1024 axons, and 256×1024 synapses using an SRAM crossbar memory. It fits in a 4.2mm square area, using a 45nm SOI process.

PAST IBM PRESS RELEASES:

DARPA SyNAPSE Phase 0
DARPA SyNAPSE Phase 1
DARPA SyNAPSE Phase 2