Why are some ecosystems so rich, yet contain so many rare species? High species diversity, together with rarity, is a general trend in neotropical forests and coral reefs. However, the origin of such diversity and the consequences of food web complexity in both species abundances and temporal fluctuations are not well understood. Several regularities are observed in complex, multispecies ecosystems that suggest that these ecologies might be organized close to points of instability. We explore, in greater depth, a recent stochastic model of population dynamics that is shown to reproduce: (i) the scaling law linking species number and connectivity; (ii) the observed distributions of species abundance reported from field studies (showing long tails and thus a predominance of rare species); (iii) the complex fluctuations displayed by natural communities (including chaotic dynamics); and (iv) the species-area relations displayed by rainforest plots. It is conjectured that the conflict between the natural tendency towards higher diversity due to immigration, and the ecosystem level constraints derived from an increasing number of links, leaves the system poised at a critical boundary separating stable from unstable communities, where large fluctuations are expected to occur. We suggest that the patterns displayed by species-rich communities, including rarity, would result from such a spontaneous tendency towards instability.