Researchers at Eindhoven University of Technology (TU/e) have developed a neuromorphic device (technology that mimics the workings of the human brain) that can perform on-chip training. This would eliminate the need to train AI models in a computer environment before transferring them to the chip. The successful trial could lead to more efficient AI chips, although a practical application will take some time.
Researchers Yoeri van de Burgt and Marco Fattori are studying the possibilities of mimicking the functioning of human brains in (computer) environments. Brains are very good at storing, processing, and using new information, so it is unsurprising that this kind of research happens, especially in light of AI training.
Van de Burgt, associate professor in the Mechanical Engineering faculty, believes neuromorphic chips, i.e. hardware, are a ‘promising alternative’ to software-based neural networks currently in widespread use for AI training. Such neural network models consist of nodes whose mutual strength is determined by a number –called a weight. This is similar to how, in the brain, neurons (nerve cells) are connected by synapses that transmit electrical and chemical signals.
“Neural networks can help solve complex problems with large amounts of data, but as the networks get larger, they bring increasing energy costs and hardware limitations,” Van de Burgt said in an article on the TU/e site.
Electrical charge
The need to first train AI models on a computer often makes this process cumbersome, time-consuming and energy-inefficient. Training AI directly on hardware also has its limitations. That’s because of the memory resistors that ‘remember’ how much electrical charge has flowed through them in the past. These memristors must be programmed one at a time and then checked for errors because of their unpredictable (the term is stochastic) nature.
However, Van de Burg, who collaborated with Marco Fattori of Electrical Engineering for this research, was able to create a circuit design that circumvents the current limitations. They managed to create a two-layer neural network based on electrochemical random-access memory (EC-RAM) components made of organic materials. The next step is to add more layers.
More layers needed for practical applicability
By comparison, an LLM like GPT-4 has 120 layers, while Llama 3 has 80. Those layers are essential to a model’s complexity. So, before this technology can be fully used for training AIs, more layers are needed.
Van de Burg and Fattori’s work is part of research initiated by Tim Stevens and Eveline van Doremaele. The results appeared in Science Advances. Van Doremaele received her doctorate last year for a dissertation on neuromorphic chips.
Also read: Cerebras, maker of wafer-scale AI chips, prepares for IPO