Self-play is how the agent learns. Instead of a fixed heuristic opponent, it plays against copies of itself and updates from the outcomes, so the opponent gets harder exactly as fast as the agent does. The training uses Neural Fictitious Self-Play: two networks learn together, a best-response network that chases the highest-value move and an average-policy network that captures how the agent tends to play over time, which is what keeps self-play from chasing its own tail.
The risk with any loop that trains on its own results is that it grades its own homework. A rising reward curve can mean the agent is better, or only that it has learned to farm its own reward signal. So progress is not measured by the training reward. It is measured by a separate evaluator that runs the frozen agent against fixed baseline opponents and reports win rates with confidence intervals. The reward drives learning; an independent, unchanging yardstick decides whether learning actually happened.