Translational repression and deadenylation of eukaryotic mRNAs result either in the sequestration of the transcripts in a nontranslatable pool or in their degradation. Removal of the 5' cap structure is a crucial step that commits deadenylated mRNAs to 5'-to-3' degradation. Pat1, Edc3 and the DEAD-box protein Dhh1 are evolutionary conserved factors known to participate in both translational repression and decapping, but their interplay is currently unclear. We report the 2.8 Å resolution structure of yeast Dhh1 bound to the N-terminal domain of Pat1. The structure shows how Pat1 wraps around the C-terminal RecA domain of Dhh1, docking onto the Phe-Asp-Phe (FDF) binding site. The FDF-binding site of Dhh1 also recognizes Edc3, revealing why the binding of Pat1 and Edc3 on Dhh1 are mutually exclusive events. Using co-immunoprecipitation assays and structure-based mutants, we demonstrate that the mode of Dhh1-Pat1 recognition is conserved in humans. Pat1 and Edc3 also interfere and compete with the RNA-binding properties of Dhh1. Mapping the RNA-binding sites on Dhh1 with a crosslinking-mass spectrometry approach shows a large RNA-binding surface around the C-terminal RecA domain, including the FDF-binding pocket. The results suggest a model for how Dhh1-containing messenger ribonucleoprotein particles might be remodeled upon Pat1 and Edc3 binding.