Conversations About AI: Self-Learning, Continuous Learning, and Crowdsourcing

Neural networks are capable of learning from structured and unstructured data.  The computing power needed to operate this form of artificial intelligence (AI), however, is enormous. The current example is ChatGPT-3, which according to Statista, a German statistics company consumed energy equal to the requirements of 200 households with three or more people for a year. Still, the AI could only word match using a best-guess methodology to know that the word “thank” would likely be followed by “you,” even though other combinations were statistically still valid.

Neuromorphic Computers Will Change AI

A neural network trains using hundreds of billions of parameters requiring lots of power. The human brain works very differently. Our nerve cells and synapses combine processing and memory. Can this be duplicated by AI running on transistors? That’s what researchers at the Max Planck Institute are trying to produce. This is leading to a new kind of computer called “neuromorphic” because its chips are designed to mimic the way our neurons and synapses work which means less energy is needed to accomplish what more traditional AI algorithms would need to learn. Neuromorphic computing is likely the future of AI.

Florian Marquardt, Director at the Max Planck Institute, University of Erlangen in Nuremberg, Germany, calls neuromorphic computers, self-learning physical machines. He states, “The core idea is to carry out the training in the form of a physical process, in which the parameters of the machine are optimized by the process itself.” Conventional neural networks require external feedback to adjust synaptic connections. But neuromorphic computers don’t.

A paper describing the work being done at Max Planck was recently published in Physical Review X. Based on current progress in designing their neuromorphic computer, the researchers are employing optical connectivity with the machine able to do reversible, non-linear processing. This means it can run forward and backward as it learns on its own. When will we see the first optical neuromorphic computer? Marquardt sees a working self-learning machine ready to go in three years and predicts it will have a dramatic impact on the evolution of AI.

AI Knowledge Expander Allows for Continuous Learning

The Max Planck Institute isn’t the only place where progress in self-learning AI is happening. In Canada, two days ago, Verses AI Inc., filed a provisional patent for what it calls the Verses Knowledge Expander. Verses describes its technology as replacing current AI.

Gabriel René, founder and CEO of Verses, describes the Knowledge Expander as being the essential component for artificial general intelligence (AGI). It functions similarly to our brains to “find the right place” within a data set to reflect the correct causal relationship between new and existing information. This allows inference in real-world scenarios by encoding the probability of relationships.

Verses has developed Genius (TM) which is patterned after our brains. Genius learns, adapts and interacts with the world. It is a software solution that is doing what neuromorphic computing platforms are intended to do. It would be interesting to see if the two could be combined.

Crowdsourcing is Enabling AI to Learn Faster

The Massachusetts Institute of Technology (MIT) along with research assistance from Harvard and the University of Washington, has developed HuGE, a reinforcement learning method that uses crowdsourced feedback to have AI learn to do complex tasks.

HuGE stands for Human Guided Exploration. Normally, AI training involves reinforcement learning guided by a human programmer who provides instructions for it to learn to execute the right action. Repetition leads to positive results. It is an iterative process that, in many cases, takes a long time.

But what if the AI could learn from the crowd? This is called crowdsourcing and involves interacting with the public with the Internet the medium. It turns out that nonexpert feedback given over the Internet lets an AI learn faster even if crowdsourced information contains errors which the researchers call “noise.” The rationale for this learning approach is described by Pulkit Agrawal, Assistant Professor in MIT’s Department of Electrical Engineering and Computer Science. He states in an MIT news release:

“One of the most time-consuming and challenging parts in designing a robotic agent today is engineering the reward function. Today reward functions are designed by expert researchers — a paradigm that is not scalable if we want to teach our robots many different tasks. Our work proposes a way to scale robot learning by crowdsourcing the design of reward function and by making it possible for nonexperts to provide useful feedback.”

Here is an example of reinforcement learning using the crowd. An AI is asked to look at two photographs with a specific task requirement. One of the photographs points to the right choice. The other is wrong. Going to the crowd provides unfiltered human feedback. The majority of human responses form the guidance to allow the AI to make a decision.

Where an expert would have written computer code providing steps and instructions, reinforcement learning from the crowd is simpler. It is like laying a trail of “breadcrumbs” for the AI to reach its goal. Learning through exploration like this gives AI the autonomy to succeed on its own. It turns out it learns faster.

The MIT team has been using HuGE to train robotic arms to do tasks. Crowdsourced data has come from more than 100 users in 13 countries. Learning this way ensures that the AI aligns with humans in its learning approach. The goal ahead is to have HuGE learn from natural language communication and physical interactions.