Power laws and self-organized criticality in theory and nature
Introduction
Experimental and technological advancements, like the steady increase in computing power, makes the study of natural and man-made complex systems progressively popular and conceptually rewarding. Typically, a complex system contains a large number of various, potentially non-identical components, which often have an internal complex structure of their own. Complex systems may exhibit novel and emergent dynamics arising from local and nonlinear interactions of the constituting elements. A prominent example for an emergent property, and possibly the phenomenon observed most frequently in real-world complex systems, is the heavy-tailed scaling behavior of variables describing a structural feature or a dynamical characteristic of the system. An observable is considered to be heavy-tailed if the probability of observing extremely large values is more likely than it would be for an exponentially distributed variable (Feldman and Taqqu, 1998).
Heavy-tailed scaling has been observed in a large variety of real-world phenomena, such as the distribution of earthquake magnitudes (Pisarenko and Sornette, 2003), solar flare intensities (Dennis, 1985), the sizes of wildfires (Newman, 2005), the sizes of neuronal avalanches (Klaus et al., 2011), wealth distribution (Levy and Solomon, 1997), city population distribution (Newman, 2005), the distribution of computer file sizes (Douceur and Bolosky, 1999, Gros et al., 2012), and various other examples (Bak, 1997, Jensen, 1998, Newman, 2005, Newman, 1996, Clauset et al., 2009, Broder et al., 2000, Adamic and Huberman, 2000).
Notably there are many types of distributions considered to be heavy-tailed, such as the Lévy distribution, the Cauchy distribution, and the Weibull distribution. Still, investigations often focus on heavy-tailed scaling in its simplest form, the form of a pure power law (viz the Pareto distribution). In fact, it is difficult to differentiate between various functional types of heavy tails on a finite interval, especially if the data have a large variance and if the sample size is relatively small. In Fig. 1 we illustrate the behavior of three distribution functions characterized by heavy tails, the Pareto, the log-normal and the log-Cauchy probability distributions (left panel), and their corresponding complementary cumulative probability distributions (CCDF) (right panel). The respective functional forms are given in Table 1. In spite of having more complex scaling properties, log-normal and log-Cauchy distributions can be approximated on a finite interval by a power law, that is by a straight line on a log–log plot. Note that the difference between log-Cauchy and Pareto distribution is more evident when is compared.
Clauset et al. (2009) have argued, that statistical methods traditionally used for data analysis (e.g. least-square fits) often misestimate the parameters describing heavy-tailed data sets, and consequently the actual scaling behavior. For a more reliable investigation of the scaling behavior one should employ methods going beyond visually fitting data sets with power laws, such as maximum likelihood estimates and cross-model validation techniques. Additionally, one should take into account the fact that most empirical data need to be binned (Virkar and Clauset, 2012), a procedure that reduces the available data resolution.
Large data sets, spanning several orders of magnitudes, are needed to single out the model which best fits the data and reproduces the heavy tail; even when advanced statistical techniques are applied. The collection of significantly larger data sets is however often difficult to achieve through experimental studies of large-scale complex systems, which often deal with slowly changing phenomena in noisy environments. Using rigorous statistical methods, Clauset et al. (2009) re-analyzed data sets for which a least-square fit did indicate power-law scaling. They found that in some cases the empirical data actually exhibit exponential or log-normal scaling, whereas in other cases a power law, or a power law with an exponential cutoff, remains a viable description—as none of the alternative distributions could be singled out with statistical significance. Thus, in the absence of additional evidence, it is best to assume the simplest scaling of the observed phenomena, adequately described with the Pareto distribution.
Over the past decades various models have been developed in order to explain the abundance of power-law scaling found in complex systems. Some of these power-law generating models were developed for describing specific systems, and have hence only a restricted applicability. Other models, however, aim to explain universal properties of a range of complex systems. They have enjoyed significant success and contributed to the development of the paradigm that power laws emerge naturally in real-world and man-made complex systems.
The seminal work of Bak et al. (1987) developed into an influential theory which unifies the origins of the power-law behavior observed in different complex systems—the so called theory of self-organized criticality (SOC). An important role for the success of SOC is the connection to the well-established theory of second order phase transitions in equilibrium statistical mechanics, for which the origin of scale-free behavior is well understood. The basic idea of SOC is that a complex system will spontaneously organize, under quite general conditions, into a state which is at the transition between two different regimes, that is at a critical point, without the need for external intervention or tuning. At such spontaneously maintained phase transition a model SOC system exhibits power-law scaling of event sizes, event durations and, in some cases, the scaling of the power spectra. These properties were also observed, to a certain extent, in natural phenomena such as earthquakes, solar flares, forest fires, and, more recently, neuronal avalanches.
In the following chapters we will discuss in more detail the pros and cons of the SOC theory and its application to real-world phenomena. In Fig. 2 we show the CCDF of some of the empirical data sets analyzed in Clauset et al. (2009). Note, that none of the shown quantities exhibit power-law-like scaling across the entire range of observations.
SOC is observed in a range of theoretical models. However, several additional features characterize real-world complex systems and these features are mostly not captured by the standard modeling approach within the SOC framework. For example, power-law scaling in heterogeneous or noisy environments, or complex dynamics with dissipative components (Janosi and Kertesz, 1993), are common features of real-world systems. As an alternative to SOC, Carlson and Doyle (1999) proposed a mechanism called highly optimized tolerance (HOT) and argued that power-law distributions can manifest themselves in systems with heterogeneous structures, as a consequence of being designed to operate optimally in uncertain environments; either by human design in the case of man-made systems, or by natural selection in the case of living organisms. The HOT mechanism does not require critical dynamics for the emergence of heavy-tailed scaling.
In the following chapters we will describe in more details the main concepts of SOC and HOT, together with several other proposals for power-law generating mechanisms, and we will discuss their successes and limitations in predicting and explaining the dynamical behavior and the structure of real-world complex systems. In this context we will provide an assessment, in comparison with theory predictions, of reported statistical properties of the empirical time series of earthquake magnitudes, solar flares intensities and sizes of neuronal avalanches. In addition we will discuss the theory of branching processes and the application of critical branching to the characterization of the dynamical regime of physical systems. Another important question—that we will address and discuss within the framework of vertex routing models—is to which extent critical dynamical systems actually show power-law scaling and how the process of experimentally observing a critical system influences the scaling of the collected data.
Section snippets
Theory of self-organized criticality
In their seminal work Bak et al. (1987) provided one of the first principles unifying the origins of the power law behavior observed in many natural systems. The core hypotheses was that systems consisting of many interacting components will, under certain conditions, spontaneously organize into a state with properties akin to the ones observed in a equilibrium thermodynamic system near a second-order phase transition. As this complex behavior arises spontaneously without the need for external
Alternative models for generating heavy-tailed distributions
The quest for explaining and understanding the abundance of power-law scaling in complex systems has produced, in the past several decades, a range of models and mechanisms for the generation of power laws and related heavy-tailed distributions.
Some among these models provide relatively simple generating mechanisms (Newman, 2005), e.g. many properties of random walks are characterized by power laws, while others are based on more intricate principles, such as the previously described SOC
Branching processes
One speaks of an avalanche when a single event causes multiple subsequent events. Similar to a snowball rolling down a snowfield and creating other toppling snowballs. Avalanches will stop eventually, just as snowballs will not trundle down the hill forever. At the level of the individual snowballs this corresponds to a branching process—a given snowball may stop rolling or nudge one or more downhill snowballs to start rolling. The theory of random branching processes captures such dynamics of
Modeling experimental data
A mathematical model of real-world phenomena should both replicate the phenomena and capture the structure and the function of the described physical system. One may divide theory models as “descriptive” or “explanatory” (Willinger et al., 2002). A descriptive model tries to reproduce the statistical properties of the phenomena in question, while containing often unrealistic and simplistic assumptions about the structure of the modeled system. Thus, not attempting to explain the underlying
Conclusions
The concept of self-organized criticality (SOC) is an intensely studied and discussed mechanism for generating power-law distributed quantities. This theory has been proposed as an explanation for power-law scaling observed in various real-world phenomena. We have focused here on several well-studied phenomena, notably earthquakes, solar flares, and neuronal avalanches; just a three out of a plethora of phenomena exhibiting fat tails. Given the amount of existing empirical data, it is important
Acknowledgments
The authors thank, in no particular order: Alain Destexhe, Nima Dehghani, Didier Sornette, Viola Priesemann, Juan Antonio Bonachela Fajardo and Dietmar Plenz, for helpful discussions, comments, and suggestions.
References (177)
- et al.
Small-world structure of earthquake network
Physica A
(2004) Scaling behavior of random maps
Phys. Lett. A
(1989)- et al.
Macroscopic models of local field potentials and the apparent noise in brain activity
Biophys. J.
(2009) - et al.
Graph structure in the web
Comput. Netw.
(2000) - et al.
Self-organized criticality on small world networks
Physica A
(2002) - et al.
Cascading toppling dynamics on scale-free networks
Physica A
(2005) - et al.
Self-organized criticality with and without conservation
Physica A
(1993) Scaling and universality in statistical physics
Physica A
(1990)Earthquake size distribution: Power-law with exponent ?
Tectonophysics
(2010)- et al.
Sandpiles on Watts–Strogatz type small-worlds
Physica A
(2005)
Sandpile avalanche dynamics on scale-free networks
Physica A
New evidence for the power-law distribution of wealth
Physica A
Power-law distribution of the world wide web
Science
Mean-field exponents for self-organized critical phenomena
Phys. Rev. A
Scale-free networks of earthquakes and aftershocks
Phys. Rev. E
How Nature Works
Punctuated equilibrium and criticality in a simple model of evolution
Phys. Rev. Lett.
Self-organized criticality: an explanation of the 1/f noise
Phys. Rev. Lett.
Emergence of scaling in random networks
Science
Does the frequency scaling of brain signals reflect self-organized critical states?
Phys. Rev. Lett.
Neuronal avalanches in neocortical circuits
J. Neurosci.
Universality in sandpile models
Phys. Rev. E
Power laws in solar flares: self-organized criticality or turbulence?
Phys. Rev. Lett.
Sandpile dynamics on random graphs
J. Phys. Soc. Japan
Self-organization without conservation: are neuronal avalanches generically critical?
J. Stat. Mech. Theory Exp.
Self-organization without conservation: true or just apparent scale-invariance?
J. Stat. Mech. Theory Exp.
Nonuniversality and scaling breakdown in a nonconservative earthquake model
Phys. Rev. E
Model and theoretical seismicity
Bull. Seismol. Soc. Am.
To what extent can dynamical models describe statistical features of turbulent flows?
Europhys. Lett.
Scaling and Renormalization in Statistical Physics, Vol. 5
Highly optimized tolerance: a mechanism for power laws in designed systems
Phys. Rev. E
Highly optimized tolerance: robustness and design in complex systems
Phys. Rev. Lett.
Complexity and robustness
Proc. Natl. Acad. Sci. USA
Olami–Feder–Christensen model on different networks
Eur. Phys. J. B
Analysis of self-organized criticality in the Olami–Feder–Christensen model and in real earthquakes
Phys. Rev. E
Earthquakes, model systems and connections to -statistics
Acta Geophys.
Analysis of a dissipative model of self-organized criticality with random neighbors
Phys. Rev. E
Avalanche models of solar flares
Sandpile models with and without an underlying spatial structure
Phys. Rev. E
Power-law distributions in empirical data
SIAM Rev.
Frequency distributions and correlations of solar X-ray flare parameters
Sol. Phys.
Micromechanical model for deformation in solids with universal predictions for stress–strain curves and slip avalanches
Phys. Rev. Lett.
Are seismic waiting time distributions universal?
Geophys. Res. Lett.
Are dragon-king neuronal avalanches dungeons for self-organized brain activity?
Eur. Phys. J. Spec. Top.
Universality in solar flare and earthquake occurrence
Phys. Rev. Lett.
Rare events and breakdown of simple scaling in the abelian sandpile model
Phys. Rev. E
Avalanche analysis from multielectrode ensemble recordings in cat, monkey, and human cerebral cortex during wakefulness and sleep
Front. Physiol.
Solar hard X-ray bursts
Sol. Phys.
Paths to self-organized criticality
Braz. J. Phys.
Structure of growing networks with preferential linking
Phys. Rev. Lett.
Cited by (241)
The atypical transport system: Natural gas pipeline system
2024, Energy ReportsComplex systems approach to natural language
2024, Physics ReportsPower-law patterns in the Phanerozoic sedimentary records of carbon, oxygen, sulfur, and strontium isotopes
2024, Journal of Palaeogeography