To accurately simulate global climate, researchers will need supercomputers more powerful than any yet designed. These so-called exascale computers would be capable of carrying out 10^18 floating point operations per second, or an exaflops. That's nearly 100 times more powerful than today's biggest supercomputer, Japan's "K Computer," which achieves 11.3 petaflops (1015 flops), and 1000 times faster than the Hopper supercomputer. The United States now appears poised to reach for the exascale, as do China, Japan, Russia, India, and the European Union. Advances in supercomputers have come at a steady pace over the past 20 years, enabled by the continual improvement in computer chip manufacturing. But this evolutionary approach won't cut it in getting to the exascale. Instead, computer scientists must first figure out ways to make future machines far more energy efficient and tolerant of errors, and find novel ways to program them.

"IBM researchers on Aug. 18, 2011 unveiled a new generation of experimental computer chips designed to emulate the brain’s abilities for perception, action and cognition. The cognitive computing chips, informally referred to as the “brain chip,” could yield many orders of magnitude less power consumption and space than used in today’s computers."

Here is a link.

ABSTRACT: The grand challenge of neuromorphic computation is to develop a flexible brain-like architecture capable of a wide array of real-time applications, while striving towards the ultra-low power consumption and compact size of the human brain—within the constraints of existing silicon and post-silicon technologies. To this end, we fabricated a key building block of a modular neuromorphic architecture, a neurosynaptic core, with 256 digital integrate-and-fire neurons and a 1024x256 bit SRAM crossbar memory for synapses using IBM’s 45nm SOI process. Our fully digital implementation is able to leverage favorable CMOS scaling trends, while ensuring one-to-one correspondence between hardware and software. In contrast to a conventional von Neumann architecture, our core tightly integrates computation (neurons) alongside memory (synapses), which allows us to implement efficient fan-out (communication) in a naturally parallel and event-driven manner, leading to ultra-low active power consumption of 45pJ/spike. The core is fully configurable in terms of neuron parameters, axon types, and synapse states and is thus  amenable to a wide range of applications. As an example, we trained a restricted Boltzmann machine offline to perform a visual digit recognition task, and mapped the learned weights to our chip.

Jae-sun Seo, Bernard Brezzo, Yong Liu, Benjamin D. Parker, Steven K. Esser, Robert K. Montoye, Bipin Rajendran, Jose A. Tierno, Leland Chang, Dharmendra S. Modha, and Daniel J. Friedman,
"A 45nm CMOS Neuromorphic Chip with a Scalable Architecture for Learning in Networks of Spiking Neurons,"
IEEE Custom Integrated Circuits Conference, September 2011.

ABSTRACT: Efforts to achieve the long-standing dream of realizing scalable learning algorithms for networks of spiking neurons in silicon have been hampered by (a) the limited scalability of analog neuron circuits; (b) the enormous area overhead of learning circuits, which grows with the number of synapses; and (c) the need to implement all inter-neuron communication via off-chip address-events. In this work, a new architecture is proposed to overcome these challenges by combining innovations in computation, memory, and communication, respectively, to leverage (a) robust digital neuron circuits; (b) novel transposable SRAM arrays that share learning circuits, which grow only with the number of neurons; and (c) crossbar fan-out for efficient on-chip inter-neuron communication. Through tight integration of memory (synapses) and computation (neurons), a highly configurable chip comprising 256 neurons and 64K binary synapses with on-chip learning based on spike-timing dependent plasticity is demonstrated in 45nm SOI-CMOS. Near-threshold, event-driven operation at 0.53V is demonstrated to maximize power efficiency for real-time pattern classification, recognition, and associative memory tasks. Future scalable systems built from the foundation provided by this work will open up possibilities for ubiquitous ultra-dense, ultra-low power brain-like cognitive computers.

Our long-distance wiring diagram of the macaque monkey brain was merged with IBM’s signature word “THINK” to create a logo for this distinguished event.

Steve Hamm, co-author of Making the World Work Better: The Ideas That Shaped a Century and a Company, interviewed me on the challenge of bringing together a multi-year, multi-disciplinary, multi-institutional collaboration.  The video is here.

"MOORE’S LAW—the prediction made in 1965 by Gordon Moore, that the number of transistors on a chip of given size would double every two years—has had a good innings." However, the transistors "have already shrunk to a size where every atom counts. Too few atoms can cause their insulation to break down, or allow current to leak to places it is not supposed to be because of a phenomenon called quantum tunnelling, in which electrons vanish spontaneously and reappear elsewhere. Too many atoms of the wrong sort, though, can be equally bad, interfering with a transistor’s conductivity. Engineers are therefore endeavouring to redesign transistors yet again, so that Dr Moore’s prediction can remain true a little longer."  

2006:              
Almaden Institute

2007:              
“Mouse”-scale simulations
Talk Video: Cognitive Computing Talk at UC Berkeley 
Talk Video: Cognitive Computing Talk at Decade of the Mind Symposium
“Rat”-scale simulations

2008:              
DARPA SyNAPSE Phase 0

2009:              
Talk Video: IEEE 125^th Anniversary 
“Cat”-scale simulations and ACM Gordon Bell Prize
DARPA SyNAPSE Phase 1

2010:              
“Network architecture of the long-distance pathways in the macaque brain”

2011:              
Cognitive Computing in Communications of the ACM
Talk Video: Cognitive Computing Keynote at DAC 
Chips and DARPA SyNAPSE Phase 2

"The problem is not that they cannot squeeze more transistors onto the chips — they surely can — but instead, like a city that cannot provide electricity for its entire streetlight system, that all those transistors could require too much power to run economically. They could overheat, too."

Authors:
Dharmendra S. Modha
Rajagopal Ananthanarayanan
Steven K. Esser
Anthony Ndirango
Anthony J. Sherbondy
Raghavendra Singh

Abstract:
Unite neuroscience, supercomputing, and nanotechnology to discover, demonstrate, and deliver the brain's core algorithms.

Abstract: The impressive capabilities of the mammalian brain—ranging from perception, pattern recognition and memory formation to decision making and motor activity control—have inspired their re-creation in a wide range of artificial intelligence systems for applications such as face recognition, anomaly detection, medical diagnosis and robotic vehicle control. Yet before neuron-based brains evolved, complex biomolecular circuits provided individual cells with the ‘intelligent’ behaviour required for survival. However, the study of how molecules can ‘think’ has not produced an equal variety of computational models and applications of artificial chemical systems. Although biomolecular systems have been hypothesized to carry out neural-network-like computations in vivo and the synthesis of artificial chemical analogues has been proposed theoretically experimental work has so far fallen short of fully implementing even a single neuron. Here, building on the richness of DNA computing and strand displacement circuitry, we show how molecular systems can exhibit autonomous brain-like behaviours. Using a simple DNA gate architecture that allows experimental scale-up of multilayer digital circuits, we systematically transform arbitrary linear threshold circuits (an artificial neural network model) into DNA strand displacement cascades that function as small neural networks. Our approach even allows us to implement a Hopfield associative memory with four fully connected artificial neurons that, after training in silico, remembers four single-stranded DNA patterns and recalls the most similar one when presented with an incomplete pattern. Our results suggest that DNA strand displacement cascades could be used to endow autonomous chemical systems with the capability of recognizing patterns of molecular events, making decisions and responding to the environment.

Hardware

Hardware