Developing machines in a manner akin to how animal trainers teach dogs or horses has proven to be a pivotal approach in advancing artificial intelligence. This method was duly recognized on Wednesday with the presentation of the highest honor in computer science. Two trailblazers in the field, Andrew Barto and Richard Sutton, were awarded this year’s A.M. Turing Award, often regarded as the tech world’s version of the Nobel Prize.
The research initiated by Barto, now 76, and Sutton, at 67, in the late 1970s has been instrumental in catalyzing recent AI innovations. Central to their pioneering work was the development of “hedonistic” machines capable of modifying their behavior based on positive reinforcement. This technique, known as reinforcement learning, was crucial when a Google program outmaneuvered the top human Go players in 2016 and 2017. It has also significantly contributed to improving cutting-edge AI technologies, such as ChatGPT, optimizing financial markets, and enhancing dexterous robotic tasks like solving a Rubik’s Cube.
Barto reminisces about the early skepticism surrounding the field. When he and his Ph.D. student Sutton were formulating their theories and algorithms at the University of Massachusetts, Amherst, it was not a mainstream pursuit. “We were on the fringes,” Barto shared in an interview. “Receiving this award is immensely satisfying as it acknowledges the importance and fascination of our work—something not widely accepted initially.”
Sponsored by Google, the annual $1 million recognition was announced by the Association for Computing Machinery. While Barto, now retired, and Sutton, a long-standing professor at the University of Alberta in Canada, are not the first AI trailblazers to earn the award named after Alan Turing, their research directly addressed Turing’s 1947 vision of a device that can evolve from experience—a fundamental aspect of reinforcement learning, according to Sutton.
Drawing inspiration from psychology and neuroscience, they explored how neurons responding to rewards or punishments contribute to learning. In an influential paper of the early 1980s, Barto and Sutton applied their strategy to a simulated task of balancing a pole on a moving cart. They later co-authored an authoritative textbook on the subject.
Google’s chief scientist, Jeff Dean, highlighted in a statement that “Their methodologies continue to underpin the AI surge, leading to significant advancements, attracting enthusiastic researchers, and generating substantial investments.”
In a discussion, Barto and Sutton sometimes clashed over assessing the risks linked to self-improving AI agents. They differentiated their work from today’s popular generative AI models—such as those behind chatbots by OpenAI and Google—which mimic human communication. Sutton mentioned the distinct paths of learning from people’s data versus an AI agent’s own experiences.
While Sutton downplays existential fears related to AI, Barto advises vigilance concerning unforeseen implications. Describing himself as a Luddite in retirement, Barto contrasts with Sutton’s embracement of a future featuring entities surpassing current human intelligence, an ideology linked to posthumanism.
Sutton expressed, “Humans are extraordinary, impressive machines, but they’re not the ultimate form. The essence of AI is to understand and improve ourselves, perhaps evolving beyond our current capabilities.”
