Computers – which became an inseparable part of our lives some eight decades ago – have become faster and smaller – hasn’t their basic architecture hasn’t changed. There’s still one part that stores information – memory (RAM or random-access memory, and the hard drive). Another part – the CPU or central processing unit – processes information.
But now, Associate Prof. Shahar Kvatinsky of the Technion-Israel Institute of Technology in Haifa, presents a structural alternative. Bringing the “thinking” and the “remembering” functions together into one unit, he has built a neural network right into the hardware of a chip, and as a proof of concept – taught it to recognize handwritten letters. The results of his study were recently been published in the prestigious journal Nature Electronics under the title “A memristive deep belief neural network based on silicon synapses.”
“We like to describe a computer as a ‘brain,’ but entirely separate hardware for storing information and for using it is not how an organic brain works,” explained Kvatinsky, who is a member of the Technion’s Viterbi Faculty of Electrical and Computer Engineering. He develops neuromorphic hardware – electronic circuits inspired by neuro-biological architectures present in the nervous system. The idea of such computers was first developed in the 1980s at the California Institute of Technology, but modern technological developments enabled considerable advances in that field since then.
Are modern computers surpassing the human brain?
One might think modern computers are already surpassing the human brain – since a computer has already defeated the best human chess and Go players. Although the answer is “yes,” AlphaGo, the program that defeated multiple Go masters, relied on 1,500 processors, and cost $3,000 for electricity per game. The human players’ energy consumption for the same game amounted to a sandwich, more or less, and that same player is also capable of talking, driving and performing countless other functionalities. Computers still have a long way to go.
In collaboration with Tower Semiconductor – the world’s leading producer of high-value analog semiconductor solutions that provides in analog integrated circuits to more than 300 customers worldwide in the automotive, medical, industrial, consumer, aerospace and defense industries, among others – Kvatinsky and his team designed and built a computer chip that, like an organic brain, does everything. It stores the information and processes it. This chip is also hardware-only, meaning its programming isn’t separate; it is integrated into the chip.
What this chip does is learn – specifically, learning handwriting recognition, a feat achieved through deep-belief algorithms. Unlike most of the neuromorphic chips investigated these days that use emerging unconventional technologies, this one is based on commercial technology manufactured by Tower Semiconductor.
Presented with multiple handwritten examples of each letter, the chip learned which one is which and achieved 97% accuracy in recognition with extremely low energy consumption.
Artificial neural networks learn in a way similar to living brains. They are presented with examples (examples of handwritten letters, in this particular study) and “figure out” on their own the elements that make one letter different from others but similar to the same letter in different handwriting. When the neural network is implemented as hardware, the learning process strengthens the conductivity of some nodes. This is very similar to how, when we learn, the connections between neurons in our brains are strengthened.
There are countless potential uses for chips that do everything. For example, suggested Kvatinsky, such a chip could be incorporated into the camera sensor of smartphones and similar devices, eliminating the conversion of analog data into digital – a step that all such devices perform before any form of enhancement is applied to the image. Instead, all processing could be performed directly on the raw image before it is stored in a compressed digital form.
“Commercial companies are in a constant race to improve their product,” Kvatinsky explained. “They can’t afford to go back to the drawing board and reimagine the product from scratch. That’s an advantage the academia has – we can develop a new concept that we believe could be better and, release it when it can compete with what’s already on the market.”
This study was led by two researchers in Kvatinsky’s lab. Postdoctoral fellow Dr. Wei Wang, who now heads his own research group in Shenzhen, China, conceived the theoretical concepts of the hardware-based deep-belief network and performed the experimental measurements, while doctoral student Loai Danial, who has since completed his doctoral studies and is working in Mobileye, designed the physical chip and led the steps involved in its fabrication. The work was supported by the European Research Council (ERC) and the EU Horizon 2020 Future and Emerging Technologies (FET)-OPEN program.