We study the effect of delays on the dynamics of large networks of neurons. We show that delays give rise to a wealth of bifurcations and to a rich phase diagram, which includes oscillatory bumps, traveling waves, lurching waves, standing waves arising via a period-doubling bifurcation, aperiodic regimes, and regimes of multistability. We study the existence and the stability of the various dynamical patterns analytically and numerically in a simplified rate model as a function of the interaction parameters. The results derived in that framework allow us to understand the origin of the diversity of dynamical states observed in large networks of spiking neurons.