DEAD-box proteins are the largest family of nucleic acid helicases, and are crucial to RNA metabolism throughout all domains of life. They contain a conserved 'helicase core' of two RecA-like domains (domains (D)1 and D2), which uses ATP to catalyse the unwinding of short RNA duplexes by non-processive, local strand separation. This mode of action differs from that of translocating helicases and allows DEAD-box proteins to remodel large RNAs and RNA-protein complexes without globally disrupting RNA structure. However, the structural basis for this distinctive mode of RNA unwinding remains unclear. Here, structural, biochemical and genetic analyses of the yeast DEAD-box protein Mss116p indicate that the helicase core domains have modular functions that enable a novel mechanism for RNA-duplex recognition and unwinding. By investigating D1 and D2 individually and together, we find that D1 acts as an ATP-binding domain and D2 functions as an RNA-duplex recognition domain. D2 contains a nucleic-acid-binding pocket that is formed by conserved DEAD-box protein sequence motifs and accommodates A-form but not B-form duplexes, providing a basis for RNA substrate specificity. Upon a conformational change in which the two core domains join to form a 'closed state' with an ATPase active site, conserved motifs in D1 promote the unwinding of duplex substrates bound to D2 by excluding one RNA strand and bending the other. Our results provide a comprehensive structural model for how DEAD-box proteins recognize and unwind RNA duplexes. This model explains key features of DEAD-box protein function and affords a new perspective on how the evolutionarily related cores of other RNA and DNA helicases diverged to use different mechanisms.