Plant reproduction relies on interactions between parental and zygotic components. Elaborate reciprocal signaling pathways enable coordination of the genetic programs between these components. A first and important step in this communication is the tight control of cell cycle events in the gametes prior to fertilization. This prepares for coordinated fertilization and the initiation of seed development. The dialog between the various actors of reproduction extends after fertilization, with the endosperm taking a central role. Importantly, the endosperm mediates a maternal input that is based on memory of the transcriptional states of imprinted genes, which is crucial for harmonious seed growth. Our current knowledge suggests that the endosperm is an integrator of the different components and genetic programs that are involved in seed development.