The Prisoner's Dilemma game is widely used to investigate how cooperation between unrelated individuals can evolve by natural selection. In this game, each player can either 'cooperate' (invest in a common good) or 'defect' (exploit the other's investment). If the opponent cooperates, you get R if you cooperate and T if you defect. If the opponent defects, you get S if you cooperate and P if you defect. Here T > R > 0 and P > S, so that 'defect' is the best response to any action by the opponent. Thus in a single play of the game, each player should defect. In our game, a fixed maximum number of rounds of the Prisoner's Dilemma game is played against the same opponent. A standard argument based on working backwards from the last round shows that defection on all rounds is the only stable outcome. In contrast, we show that if extrinsic factors maintain variation in behaviour, high levels of co-operation are stable. Our results highlight the importance of extrinsic variability in determining the outcome of evolutionary games.