In C. elegans, mRNA production is initially repressed in the embryonic germline by a protein unique to C. elegans germ cells, PIE-1. PIE-1 is degraded upon the birth of the germ cell precursors, Z2 and Z3. We have identified a chromatin-based mechanism that succeeds PIE-1 repression in these cells. A subset of nucleosomal histone modifications, methylated lysine 4 on histone H3 (H3meK4) and acetylated lysine 8 on histone H4 (H4acetylK8), are globally lost and the DNA appears more condensed. This coincides with PIE-1 degradation and requires that germline identity is not disrupted. Drosophila pole cell chromatin also lacks H3meK4, indicating that a unique chromatin architecture is a conserved feature of embryonic germ cells. Regulation of the germline-specific chromatin architecture requires functional nanos activity in both organisms. These results indicate that genome-wide repression via a nanos-regulated, germ cell-specific chromatin organization is a conserved feature of germline maintenance during embryogenesis.