What is reinforcement learning?
1 Answer
Reinforcement learning, or RL, is a type of machine learning where an AI system learns by trial and error, receiving rewards for good actions and penalties for bad ones. Over time, it learns which actions lead to the best results.
The core components:
1. Agent: the AI that makes decisions.
2. Environment: the world or system the agent interacts with, such as a game, a robot's surroundings or a simulated market.
3. Actions: the choices the agent can make.
4. State: the current situation the agent observes.
5. Reward: a signal indicating how good or bad an outcome was.
6. Policy: the strategy the agent learns for choosing actions.
The agent tries actions, observes results and rewards, and gradually adjusts its strategy to maximise total rewards over time.
An analogy: think of training a dog. You do not explain the rules; you reward good behaviour with treats. Through repetition, the dog learns which actions earn rewards. Reinforcement learning works similarly, but with millions of rapid trials.
Famous examples:
1. Games. DeepMind's AlphaGo defeated world champion Go player Lee Sedol in 2016, a milestone because Go was considered extremely difficult for computers. Its successor, AlphaZero, learned chess, Go and shogi from scratch by playing against itself. RL systems have also mastered complex video games.
2. Robotics. Teaching robots to walk, grasp objects and balance, often in simulation first.
3. Optimisation. Managing data centre cooling to reduce energy use, optimising traffic signals and controlling industrial processes.
4. Recommendations and advertising, adapting to user responses.
Reinforcement learning in modern language models:
RL is a key part of training AI assistants. Reinforcement learning from human feedback, RLHF, uses human preferences as rewards to make models more helpful and safe. More recently, reinforcement learning has been central to training reasoning models: models attempt maths, coding and logic problems where answers can be checked automatically, and are rewarded for correct solutions. Through this process, models learn to reason step by step, check their work and try alternative approaches.
Challenges:
Designing rewards is difficult. Agents sometimes find unexpected shortcuts that maximise rewards without achieving the real goal, known as reward hacking.
RL often requires huge numbers of trials, which is why simulations are used.
Real-world mistakes during learning can be costly or dangerous.
Reinforcement learning is how AI learns not just to know things, but to do things well.