AI teaches itself and outperforms human-designed algorithms - Machine learning & AI

Credit: Pixabay/CC0 Public Domain

Like humans, artificial intelligence learns by trial and error, but traditionally, it requires humans to set the ball rolling by designing the algorithms and rules that govern the learning process. However, as AI technology advances, machines are increasingly doing things themselves. An example is a new AI system developed by researchers that invented its own way to learn, resulting in an algorithm that outperformed human-designed algorithms on a series of complex tasks.

For decades, human engineers have designed the algorithms that agents use to learn, especially reinforcement learning (RL), where an AI learns by receiving rewards for successful actions. While learning comes naturally to humans and animals, thanks to millions of years of evolution, it has to be explicitly taught to AI. This process is often slow and laborious and is ultimately limited by human intuition.

Taking their cue from evolution, which is a random trial and error process, the researchers created a large digital population of AI agents. These agents tried to solve numerous tasks in many different, complex environments using a particular learning rule.

Overseeing them was a "meta-network," a parent AI that analyzed how well the agents performed and then changed the learning rule so the next generation of agents could learn faster and perform better. This allowed the system to discover a new learning rule, DiscoRL, which the researchers called Disco57 (evaluated on 57 Atari games), that was superior to any previously designed by humans.

The team then used Disco57 to train a new AI agent and compared its performance against some of the best human-designed algorithms, such as PPO and MuZero. First, it was trained on well-known Atari games, and then on unseen challenges, including games like ProcGen, Crafter and NetHack.

The results were outstanding. On the Atari Benchmark (a set of classic Atari video games used to evaluate AI performance), the DiscoRL-trained achieved better results than all human-designed algorithms. When confronted with unseen challenges, it performed at a state-of-the-art level, proving the system had discovered its own learning rule.

"Our findings suggest that the RL algorithms required for advanced artificial intelligence may soon be automatically discovered from the experiences of agents, rather than manually designed," wrote the researchers in their paper published in the journal Nature. "This work has taken a step towards machine-designed reinforcement learning algorithms that can compete with and even outperform some of the best manually-designed algorithms in challenging environments." 

by Paul Arnold, Phys.org

edited by Gaby Clark, reviewed by Robert Egan 

Source: AI teaches itself and outperforms human-designed algorithms