2026 Recipients of C&C Prize
Group B
Dr. Andrew G. Barto Professor Emeritus, University of Massachusetts Amherst |
Dr. Richard S. Sutton Professor, University of Alberta |
Dr. Christopher J. Watkins Professor, Royal Holloway, University of London |
Citation
Pioneering Contributions to the Establishment of Reinforcement Learning Concepts and Algorithms
Achievements
Artificial intelligence (AI) is a technology that enables computers to perform intellectual activities — such as perception, judgment, and decision-making — traditionally carried out by humans. It holds the promise of bringing about significant transformations in society, industry, and people’s lives through increased productivity and the creation of new services. While supervised learning trained on a set of examples labeled with correct responses has been the mainstream approach to learning in traditional AI, a key challenge has been to enable systems to learn through interaction with the external environment and make autonomous decisions even in situations where correct responses are not provided in advance. Dr. Andrew G. Barto, Dr. Richard S. Sutton, and Dr. Christopher J. Watkins addressed this challenge by establishing the conceptual and mathematical foundations of reinforcement learning and developing practical, highly versatile reinforcement learning algorithms that do not rely on labeled training examples, thereby making significant contributions to the advancement of modern AI.
Although the idea of learning from rewards and penalties instead of from labeled examples had long been known, traditional application of machine learning assumed that the agent — the entity doing the learning — had access to labeled training examples, and methods for learning from rewards and penalties, often called learning by trial-and-error, had not yet been fully exploited in AI. In the early 1980s, Dr. Barto and Dr. Sutton combined psychology-based trial-and-error learning with the concept of sequential decision-making to establish the basic framework of reinforcement learning, with Markov decision processes as its mathematical foundation. Furthermore, in their 1983 study of the inverted pendulum problem, they proposed a method that would later serve as the prototype for the Actor-Critic method. This established a general framework for reinforcement learning that allows agents to learn to improve their behavior through continuing interaction with their environments.
Subsequently, it became clear that, in reinforcement learning, optimizing long-term policies requires considering not only the reward immediately following an action but also rewards to be received in the future. In 1988, Dr. Sutton proposed Temporal Difference Learning. Temporal Difference Learning is a method that combines the advantages of the Monte Carlo method and dynamic programming; it uses the difference between the current prediction and the prediction for the next time step (the temporal difference) to sequentially update the expected future cumulative reward. This method laid the foundation for long-term value estimation in unknown environments. Furthermore, Dr. Barto and Dr. Sutton developed the Actor-Critic method, in which an “Actor” that selects actions and a “Critic” that evaluates the results learn in coordination. In the Actor-Critic method, the Critic evaluates value using Temporal Difference Learning, and the Actor uses those results to learn a policy that maximizes cumulative reward. Thanks to these achievements, reinforcement learning evolved into a systematic academic field capable of long-term value estimation and policy improvement. In addition, the two authors published the seminal textbook "Reinforcement Learning: An Introduction" in 1998, contributing significantly to the popularization and advancement of reinforcement learning research.
Meanwhile, Dr. Watkins combined the reinforcement learning framework developed by Dr. Barto and Dr. Sutton with dynamic programming and conducted research on methods for learning optimal policies without prior knowledge of the environment model. He then proposed Q-learning in his 1989 doctoral dissertation. In Q-learning, a Q-value — representing the expected cumulative reward to be obtained in the future for each state-action pair — is maintained, and the Q-value is updated sequentially using the reward received after an action and information about the next state. By repeating this update process, the algorithm learns a policy that selects the action with the highest Q-value in each state. The most distinctive feature of Q-learning is that it learns an optimal policy solely from experience, without requiring a predefined environmental model, thereby achieving model-free optimal control. Because Q-learning has a relatively simple structure, it has become widely adopted as a representative algorithm for model-free control.
Reinforcement learning, pioneered by Dr. Barto, Dr. Sutton, and Dr. Watkins, has become a general-purpose decision-making technology not limited to specific tasks and has evolved into deep reinforcement learning through its integration with deep neural networks. Its applications have expanded to include game AI, communication network control, semiconductor design, and the optimization of logistics and energy management. Furthermore, reinforcement learning that utilizes human feedback has become a technology for improving the quality of generative AI responses, helping to ensure that they are safe, reliable, and aligned with human intent. It is also expected to play a crucial role in the future development of an AI-driven society as a core technology supporting behavioral learning in physical AI and next-generation robot control. The achievements of Dr. Barto and Dr. Sutton, who established the key concepts and mathematical foundations of reinforcement learning, and those of Dr. Watkins, who devised Q-learning and significantly advanced the practical application of reinforcement learning, have had an immense impact on the development of information processing and intelligent information technology. These achievements are highly regarded as outstanding contributions worthy of the C&C Prize.