H.-S. Philip Wong (left), Weier Wan, and Gert Cauwenberghs (not pictured) helped build a new kind of computer chip that can run powerful algorithms. (Source: Philip Wong/Paul Jin)

Birth of NeuRRAM Chip Enables Compute-in-Memory to Run AI Programs

The next level of computer chips is here and it is amazing. It was also first thought of years ago. Somehow this approach to chips was right in front of researchers and yet was missed time and again. And understanding the new chip isn’t easy for the layman.

Computer scientists decades ago developed the materials to create new chips that perform computations where memory is stored — a technology known as compute-in-memory. But with traditional digital computers performing so well, these ideas were overlooked for decades.

“That work, just like most scientific work, was kind of forgotten,” said Wong, a professor at Stanford.

The first such device dates back to at least 1964 when electrical engineers at Stanford discovered they could manipulate certain metal oxides, to turn their ability to conduct electricity on and off. That’s significant because a material’s ability to switch between two states provides the backbone for traditional memory storage. Typically, in digital memory, a high voltage state corresponds to a 1, and a low voltage to a 0.

Most of us sitting at our keyboards don’t deal with chips very much daily. And the knowledge we may have about computer chips might be pretty limited.

A story on quantamagazine.org hails the arrival of a new approach to making computer chips do twice the amount of processing while using half the power to keep them operating. According to the article, in August, Weier Wan, H.-S. Philip Wong, Gert Cauwenberghs, and their colleagues unveiled the new neuromorphic chip called NeuRRAM. It includes 3 million memory cells and thousands of neurons built into its hardware to run algorithms, and uses a relatively new type of memory called resistive RAM, or RRAM. Unlike previous RRAM chips, NeuRRAM is programmed to operate in an analog fashion to save more energy and space. While digital memory is binary — storing either a 1 or a 0 — analog memory cells in the NeuRRAM chip can each store multiple values along a fully continuous range. That allows the chip to store more information from massive AI algorithms in the same amount of chip space.

How is this significant? The story’s writer, Allison Whitten, explains:

“As a result, the new chip can perform as well as digital computers on complex AI tasks like image and speech recognition, and the authors claim it is up to 1,000 times more energy efficient, opening up the possibility for tiny chips to run increasingly complicated algorithms within small devices previously unsuitable for AI like smart watches and phones.”

How It Works

To get an RRAM device to switch states, you apply a voltage across metal electrodes hooked up to two ends of the metal oxide. Normally, metal oxides are insulators, which means they don’t conduct electricity. But with enough voltage, the current builds up, eventually pushing through the material’s weak spots and forging a path to the electrode on the other side. Once the current has broken through, it can flow freely along that path.

Wong likens this process to lightning: When enough charge builds up inside a cloud, it quickly finds a low-resistance path, and lightning strikes. But unlike lightning, whose path disappears, the path through the metal oxide remains, meaning it stays conductive indefinitely. It’s possible to erase the conductive path by applying another voltage to the material. So researchers can switch an RRAM between two states and use them to store digital memory.

The NeuRRam chip can run algorithms because of its efficient use of energy.

The team’s design keeps the NeuRRAM chip tiny — the size of a fingernail — while squeezing 3 million RRAM memory devices that can serve as analog processors. While it can run neural networks at least as well as digital computers, the chip also (for the first time) can run algorithms that perform computations in different directions. Their chip can input a voltage to the rows of the RRAM array and read outputs from the columns as is standard for RRAM chips, but it can also do it backward from the columns to the rows, so it can be used in neural networks that operate with data flowing in different directions.

As with RRAM technology itself, this has long been possible, but no one thought to do it. It has similarities to how a brain works.

For now, the energy efficiency the team accomplished while running large AI algorithms on the NeuRRAM chip has created new hope that memory technologies may represent the future of computing with AI. Maybe one day we’ll even be able to match the human brain’s 86 billion neurons and the trillions of synapses that connect them without running out of power.

The article includes a great deal of technical information explaining the possibilities of the chip’s usage.

read more at quantamagazine.org