In response to different environmental stresses, eIF2α phosphorylation represses global translation coincident with preferential translation of ATF4, a master regulator controlling the transcription of key genes essential for adaptative functions. Here, we establish that the eIF2α/ATF4 pathway directs an autophagy gene transcriptional program in response to amino acid starvation or endoplasmic reticulum stress. The eIF2α-kinases GCN2 and PERK and the transcription factors ATF4 and CHOP are also required to increase the transcription of a set of genes implicated in the formation, elongation and function of the autophagosome. We also identify three classes of autophagy genes according to their dependence on ATF4 and CHOP and the binding of these factors to specific promoter cis elements. Furthermore, different combinations of CHOP and ATF4 bindings to target promoters allow the trigger of a differential transcriptional response according to the stress intensity. Overall, this study reveals a novel regulatory role of the eIF2α-ATF4 pathway in the fine-tuning of the autophagy gene transcription program in response to stresses.