To ensure proper gene regulation within constrained nuclear space, chromosomes facilitate access to transcribed regions, while compactly packaging all other information. Recent studies revealed that chromosomes are organized into megabase-scale domains that demarcate active and inactive genetic elements, suggesting that compartmentalization is important for genome function. Here, we show that very specific long-range interactions are anchored by cohesin/CTCF sites, but not cohesin-only or CTCF-only sites, to form a hierarchy of chromosomal loops. These loops demarcate topological domains and form intricate internal structures within them. Post-mitotic nuclei deficient for functional cohesin exhibit global architectural changes associated with loss of cohesin/CTCF contacts and relaxation of topological domains. Transcriptional analysis shows that this cohesin-dependent perturbation of domain organization leads to widespread gene deregulation of both cohesin-bound and non-bound genes. Our data thereby support a role for cohesin in the global organization of domain structure and suggest that domains function to stabilize the transcriptional programmes within them.