T cell dysfunction is a hallmark of many cancers, but the basis for T cell dysfunction and the mechanisms by which antibody blockade of the inhibitory receptor PD-1 (anti-PD-1) reinvigorates T cells are not fully understood. Here we show that such therapy acts on a specific subpopulation of exhausted CD8+ tumor-infiltrating lymphocytes (TILs). Dysfunctional CD8+ TILs possess canonical epigenetic and transcriptional features of exhaustion that mirror those seen in chronic viral infection. Exhausted CD8+ TILs include a subpopulation of 'progenitor exhausted' cells that retain polyfunctionality, persist long term and differentiate into 'terminally exhausted' TILs. Consequently, progenitor exhausted CD8+ TILs are better able to control tumor growth than are terminally exhausted T cells. Progenitor exhausted TILs can respond to anti-PD-1 therapy, but terminally exhausted TILs cannot. Patients with melanoma who have a higher percentage of progenitor exhausted cells experience a longer duration of response to checkpoint-blockade therapy. Thus, approaches to expand the population of progenitor exhausted CD8+ T cells might be an important component of improving the response to checkpoint blockade.