Elsevier

Physics Reports

Volume 536, Issue 2, 10 March 2014, Pages 41-74
Physics Reports

Power laws and self-organized criticality in theory and nature

https://doi.org/10.1016/j.physrep.2013.11.002Get rights and content

Abstract

Power laws and distributions with heavy tails are common features of many complex systems. Examples are the distribution of earthquake magnitudes, solar flare intensities and the sizes of neuronal avalanches. Previously, researchers surmised that a single general concept may act as an underlying generative mechanism, with the theory of self organized criticality being a weighty contender.

The power-law scaling observed in the primary statistical analysis is an important, but by far not the only feature characterizing experimental data. The scaling function, the distribution of energy fluctuations, the distribution of inter-event waiting times, and other higher order spatial and temporal correlations, have seen increased consideration over the last years. Leading to realization that basic models, like the original sandpile model, are often insufficient to adequately describe the complexity of real-world systems with power-law distribution.

Consequently, a substantial amount of effort has gone into developing new and extended models and, hitherto, three classes of models have emerged. The first line of models is based on a separation between the time scales of an external drive and an internal dissipation, and includes the original sandpile model and its extensions, like the dissipative earthquake model. Within this approach the steady state is close to criticality in terms of an absorbing phase transition. The second line of models is based on external drives and internal dynamics competing on similar time scales and includes the coherent noise model, which has a non-critical steady state characterized by heavy-tailed distributions. The third line of models proposes a non-critical self-organizing state, being guided by an optimization principle, such as the concept of highly optimized tolerance.

We present a comparative overview regarding distinct modeling approaches together with a discussion of their potential relevance as underlying generative models for real-world phenomena. The complexity of physical and biological scaling phenomena has been found to transcend the explanatory power of individual paradigmal concepts. The interaction between theoretical development and experimental observations has been very fruitful, leading to a series of novel concepts and insights.

Introduction

Experimental and technological advancements, like the steady increase in computing power, makes the study of natural and man-made complex systems progressively popular and conceptually rewarding. Typically, a complex system contains a large number of various, potentially non-identical components, which often have an internal complex structure of their own. Complex systems may exhibit novel and emergent dynamics arising from local and nonlinear interactions of the constituting elements. A prominent example for an emergent property, and possibly the phenomenon observed most frequently in real-world complex systems, is the heavy-tailed scaling behavior of variables describing a structural feature or a dynamical characteristic of the system. An observable is considered to be heavy-tailed if the probability of observing extremely large values is more likely than it would be for an exponentially distributed variable (Feldman and Taqqu, 1998).

Heavy-tailed scaling has been observed in a large variety of real-world phenomena, such as the distribution of earthquake magnitudes (Pisarenko and Sornette, 2003), solar flare intensities (Dennis, 1985), the sizes of wildfires (Newman, 2005), the sizes of neuronal avalanches (Klaus et al., 2011), wealth distribution (Levy and Solomon, 1997), city population distribution (Newman, 2005), the distribution of computer file sizes (Douceur and Bolosky, 1999, Gros et al., 2012), and various other examples (Bak, 1997, Jensen, 1998, Newman, 2005, Newman, 1996, Clauset et al., 2009, Broder et al., 2000, Adamic and Huberman, 2000).

Notably there are many types of distributions considered to be heavy-tailed, such as the Lévy distribution, the Cauchy distribution, and the Weibull distribution. Still, investigations often focus on heavy-tailed scaling in its simplest form, the form of a pure power law (viz the Pareto distribution). In fact, it is difficult to differentiate between various functional types of heavy tails on a finite interval, especially if the data have a large variance and if the sample size is relatively small. In Fig. 1 we illustrate the behavior of three distribution functions characterized by heavy tails, the Pareto, the log-normal and the log-Cauchy probability distributions p(x) (left panel), and their corresponding complementary cumulative probability distributions (CCDF) C(x)=xp(x)dx (right panel). The respective functional forms are given in Table 1. In spite of having more complex scaling properties, log-normal and log-Cauchy distributions can be approximated on a finite interval by a power law, that is by a straight line on a log–log plot. Note that the difference between log-Cauchy and Pareto distribution is more evident when C(x) is compared.

Clauset et al. (2009) have argued, that statistical methods traditionally used for data analysis (e.g.  least-square fits) often misestimate the parameters describing heavy-tailed data sets, and consequently the actual scaling behavior. For a more reliable investigation of the scaling behavior one should employ methods going beyond visually fitting data sets with power laws, such as maximum likelihood estimates and cross-model validation techniques. Additionally, one should take into account the fact that most empirical data need to be binned (Virkar and Clauset, 2012), a procedure that reduces the available data resolution.

Large data sets, spanning several orders of magnitudes, are needed to single out the model which best fits the data and reproduces the heavy tail; even when advanced statistical techniques are applied. The collection of significantly larger data sets is however often difficult to achieve through experimental studies of large-scale complex systems, which often deal with slowly changing phenomena in noisy environments. Using rigorous statistical methods, Clauset et al. (2009) re-analyzed data sets for which a least-square fit did indicate power-law scaling. They found that in some cases the empirical data actually exhibit exponential or log-normal scaling, whereas in other cases a power law, or a power law with an exponential cutoff, remains a viable description—as none of the alternative distributions could be singled out with statistical significance. Thus, in the absence of additional evidence, it is best to assume the simplest scaling of the observed phenomena, adequately described with the Pareto distribution.

Over the past decades various models have been developed in order to explain the abundance of power-law scaling found in complex systems. Some of these power-law generating models were developed for describing specific systems, and have hence only a restricted applicability. Other models, however, aim to explain universal properties of a range of complex systems. They have enjoyed significant success and contributed to the development of the paradigm that power laws emerge naturally in real-world and man-made complex systems.

The seminal work of  Bak et al. (1987) developed into an influential theory which unifies the origins of the power-law behavior observed in different complex systems—the so called theory of self-organized criticality (SOC). An important role for the success of SOC is the connection to the well-established theory of second order phase transitions in equilibrium statistical mechanics, for which the origin of scale-free behavior is well understood. The basic idea of SOC is that a complex system will spontaneously organize, under quite general conditions, into a state which is at the transition between two different regimes, that is at a critical point, without the need for external intervention or tuning. At such spontaneously maintained phase transition a model SOC system exhibits power-law scaling of event sizes, event durations and, in some cases, the 1/f scaling of the power spectra. These properties were also observed, to a certain extent, in natural phenomena such as earthquakes, solar flares, forest fires, and, more recently, neuronal avalanches.

In the following chapters we will discuss in more detail the pros and cons of the SOC theory and its application to real-world phenomena. In Fig. 2 we show the CCDF of some of the empirical data sets analyzed in  Clauset et al. (2009). Note, that none of the shown quantities exhibit power-law-like scaling across the entire range of observations.

SOC is observed in a range of theoretical models. However, several additional features characterize real-world complex systems and these features are mostly not captured by the standard modeling approach within the SOC framework. For example, power-law scaling in heterogeneous or noisy environments, or complex dynamics with dissipative components (Janosi and Kertesz, 1993), are common features of real-world systems. As an alternative to SOC,  Carlson and Doyle (1999) proposed a mechanism called highly optimized tolerance (HOT) and argued that power-law distributions can manifest themselves in systems with heterogeneous structures, as a consequence of being designed to operate optimally in uncertain environments; either by human design in the case of man-made systems, or by natural selection in the case of living organisms. The HOT mechanism does not require critical dynamics for the emergence of heavy-tailed scaling.

In the following chapters we will describe in more details the main concepts of SOC and HOT, together with several other proposals for power-law generating mechanisms, and we will discuss their successes and limitations in predicting and explaining the dynamical behavior and the structure of real-world complex systems. In this context we will provide an assessment, in comparison with theory predictions, of reported statistical properties of the empirical time series of earthquake magnitudes, solar flares intensities and sizes of neuronal avalanches. In addition we will discuss the theory of branching processes and the application of critical branching to the characterization of the dynamical regime of physical systems. Another important question—that we will address and discuss within the framework of vertex routing models—is to which extent critical dynamical systems actually show power-law scaling and how the process of experimentally observing a critical system influences the scaling of the collected data.

Section snippets

Theory of self-organized criticality

In their seminal work  Bak et al. (1987) provided one of the first principles unifying the origins of the power law behavior observed in many natural systems. The core hypotheses was that systems consisting of many interacting components will, under certain conditions, spontaneously organize into a state with properties akin to the ones observed in a equilibrium thermodynamic system near a second-order phase transition. As this complex behavior arises spontaneously without the need for external

Alternative models for generating heavy-tailed distributions

The quest for explaining and understanding the abundance of power-law scaling in complex systems has produced, in the past several decades, a range of models and mechanisms for the generation of power laws and related heavy-tailed distributions.

Some among these models provide relatively simple generating mechanisms (Newman, 2005), e.g. many properties of random walks are characterized by power laws, while others are based on more intricate principles, such as the previously described SOC

Branching processes

One speaks of an avalanche when a single event causes multiple subsequent events. Similar to a snowball rolling down a snowfield and creating other toppling snowballs. Avalanches will stop eventually, just as snowballs will not trundle down the hill forever. At the level of the individual snowballs this corresponds to a branching process—a given snowball may stop rolling or nudge one or more downhill snowballs to start rolling. The theory of random branching processes captures such dynamics of

Modeling experimental data

A mathematical model of real-world phenomena should both replicate the phenomena and capture the structure and the function of the described physical system. One may divide theory models as “descriptive” or “explanatory” (Willinger et al., 2002). A descriptive model tries to reproduce the statistical properties of the phenomena in question, while containing often unrealistic and simplistic assumptions about the structure of the modeled system. Thus, not attempting to explain the underlying

Conclusions

The concept of self-organized criticality (SOC) is an intensely studied and discussed mechanism for generating power-law distributed quantities. This theory has been proposed as an explanation for power-law scaling observed in various real-world phenomena. We have focused here on several well-studied phenomena, notably earthquakes, solar flares, and neuronal avalanches; just a three out of a plethora of phenomena exhibiting fat tails. Given the amount of existing empirical data, it is important

Acknowledgments

The authors thank, in no particular order: Alain Destexhe, Nima Dehghani, Didier Sornette, Viola Priesemann, Juan Antonio Bonachela Fajardo and Dietmar Plenz, for helpful discussions, comments, and suggestions.

References (177)

  • LeeD.S. et al.

    Sandpile avalanche dynamics on scale-free networks

    Physica A

    (2004)
  • LevyM. et al.

    New evidence for the power-law distribution of wealth

    Physica A

    (1997)
  • AdamicL.A. et al.

    Power-law distribution of the world wide web

    Science

    (2000)
  • AlstrømP.

    Mean-field exponents for self-organized critical phenomena

    Phys. Rev. A

    (1988)
  • BaiesiM. et al.

    Scale-free networks of earthquakes and aftershocks

    Phys. Rev. E

    (2004)
  • BakP.

    How Nature Works

    (1997)
  • BakP. et al.

    Punctuated equilibrium and criticality in a simple model of evolution

    Phys. Rev. Lett.

    (1993)
  • BakP. et al.

    Self-organized criticality: an explanation of the 1/f noise

    Phys. Rev. Lett.

    (1987)
  • BarabásiA.L. et al.

    Emergence of scaling in random networks

    Science

    (1999)
  • BedardC. et al.

    Does the 1/f frequency scaling of brain signals reflect self-organized critical states?

    Phys. Rev. Lett.

    (2006)
  • BeggsJ.M. et al.

    Neuronal avalanches in neocortical circuits

    J. Neurosci.

    (2003)
  • Ben-HurA. et al.

    Universality in sandpile models

    Phys. Rev. E

    (1996)
  • BoffettaG. et al.

    Power laws in solar flares: self-organized criticality or turbulence?

    Phys. Rev. Lett.

    (1999)
  • BonabeauE.

    Sandpile dynamics on random graphs

    J. Phys. Soc. Japan

    (1995)
  • BonachelaJ.A. et al.

    Self-organization without conservation: are neuronal avalanches generically critical?

    J. Stat. Mech. Theory Exp.

    (2010)
  • BonachelaJ.A. et al.

    Self-organization without conservation: true or just apparent scale-invariance?

    J. Stat. Mech. Theory Exp.

    (2009)
  • BoulterC.J. et al.

    Nonuniversality and scaling breakdown in a nonconservative earthquake model

    Phys. Rev. E

    (2003)
  • BurridgeR. et al.

    Model and theoretical seismicity

    Bull. Seismol. Soc. Am.

    (1967)
  • CarboneV. et al.

    To what extent can dynamical models describe statistical features of turbulent flows?

    Europhys. Lett.

    (2002)
  • CardyJ.L.

    Scaling and Renormalization in Statistical Physics, Vol. 5

    (1996)
  • CarlsonJ.M. et al.

    Highly optimized tolerance: a mechanism for power laws in designed systems

    Phys. Rev. E

    (1999)
  • CarlsonJ.M. et al.

    Highly optimized tolerance: robustness and design in complex systems

    Phys. Rev. Lett.

    (2000)
  • CarlsonJ.M. et al.

    Complexity and robustness

    Proc. Natl. Acad. Sci. USA

    (2002)
  • CarusoF. et al.

    Olami–Feder–Christensen model on different networks

    Eur. Phys. J. B

    (2006)
  • CarusoF. et al.

    Analysis of self-organized criticality in the Olami–Feder–Christensen model and in real earthquakes

    Phys. Rev. E

    (2007)
  • CelikogluA. et al.

    Earthquakes, model systems and connections to q-statistics

    Acta Geophys.

    (2012)
  • ChabanolM.L. et al.

    Analysis of a dissipative model of self-organized criticality with random neighbors

    Phys. Rev. E

    (1997)
  • CharbonneauP. et al.

    Avalanche models of solar flares

  • ChristensenK. et al.

    Sandpile models with and without an underlying spatial structure

    Phys. Rev. E

    (1993)
  • ClausetA. et al.

    Power-law distributions in empirical data

    SIAM Rev.

    (2009)
  • CrosbyN.B. et al.

    Frequency distributions and correlations of solar X-ray flare parameters

    Sol. Phys.

    (1993)
  • DahmenK.A. et al.

    Micromechanical model for deformation in solids with universal predictions for stress–strain curves and slip avalanches

    Phys. Rev. Lett.

    (2009)
  • DavidsenJ. et al.

    Are seismic waiting time distributions universal?

    Geophys. Res. Lett.

    (2004)
  • De ArcangelisL.

    Are dragon-king neuronal avalanches dungeons for self-organized brain activity?

    Eur. Phys. J. Spec. Top.

    (2012)
  • De ArcangelisL. et al.

    Universality in solar flare and earthquake occurrence

    Phys. Rev. Lett.

    (2006)
  • De MenechM. et al.

    Rare events and breakdown of simple scaling in the abelian sandpile model

    Phys. Rev. E

    (1998)
  • DehghaniN. et al.

    Avalanche analysis from multielectrode ensemble recordings in cat, monkey, and human cerebral cortex during wakefulness and sleep

    Front. Physiol.

    (2012)
  • DennisB.R.

    Solar hard X-ray bursts

    Sol. Phys.

    (1985)
  • DickmanR. et al.

    Paths to self-organized criticality

    Braz. J. Phys.

    (2000)
  • DorogovtsevS.N. et al.

    Structure of growing networks with preferential linking

    Phys. Rev. Lett.

    (2000)
  • Cited by (241)

    View all citing articles on Scopus
    View full text