Physica A 342 (2004) 126 – 131
www.elsevier.com/locate/physa
The need for alternative information measures A. Plastino∗ Instituto de F sica (IFLP), Universidad Nacional de La Plata and Argentina’s National Research Council (CONICET), C.C. 727, 1900 La Plata, Argentina Received 2 November 2003 Available online 12 May 2004
Abstract In 1948 Shannon advanced information theory (IT) as a new branch of mathematics and a powerful tool for understanding the intricacies of the communication process. Nine years later Jaynes’ conceived the maximum entropy principle and was able to shed with it much light onto statistical mechanics and thermodynamics. Shannon’s logarithmic information measure, the MaxEnt protagonist, was successfully connected with the thermodynamic entropy. Since 1988 a new project, called nonextensive thermostatistics (NET), came into being after the pioneering work by Tsallis and collaborators. It has achieved today a remarkable degree of success, amid a not small degree of controversy. It is also a MaxEnt construction but uses a di2erent information measure in the leading role. We advance here some epistemological considerations on the proper place of any information measure within a theoretical physics’ construct in order put NET into a more solid conceptual framework. c 2004 Elsevier B.V. All rights reserved. PACS: 03.67.−a; 89.70.+c; 03.65.Bz Keywords: Information physics; Wheeler program; Information measure
1. Introduction Non-extensive thermostatistics is by now considered as a new paradigm for statistical mechanics [1–12], being based on Tsallis’ non-extensive information measure [3] q 1− pn Sq = k B ; (1) q−1 ∗
Fax: +54-21-258-138. E-mail address:
[email protected] (A. Plastino).
c 2004 Elsevier B.V. All rights reserved. 0378-4371/$ - see front matter doi:10.1016/j.physa.2004.04.068
A. Plastino / Physica A 342 (2004) 126 – 131
127
where kB stands for Boltzmann constant, and {pn } is a set of normalized probabilities. The real parameter q is called the index of non-extensivity, the conventional Boltzmann–Gibbs statistics being recovered in the limit q = 1. Its undeniable merits notwithstanding, NET has originated a rather unusual degree of heated controversy (see, for instance, Refs. [13,14]). It is our contention here that this controversy is based on an epistemological misunderstanding. The goal of the present e2ort is that of hopefully clearing things up.
2. Information theory and statistical mechanics Information theory (IT) entered physics via Jaynes’ maximum entropy principle (MaxEnt) in 1957 with two papers in which statistical mechanics was re-derived aD la IT [15–17]. Since IT’s central concept is that of information measure (IM) [17–20], a proper understanding of its role must at the outset be put into its proper perspective. In the study of Nature, scientiEc truth is established through the agreement between two independent instances that can neither bribe nor suborn each other: analysis (pure thought) and experiment [21]. The analytic part employs mathematical tools and concepts. The following scheme thus ensues: WORLD OF MATHEMATICAL ENTITIES WV LABORATORY The mathematical realm was called by Plato Topos Uranus (TP). Science in general, and physics in particular, is thus primarily (although not exclusively, of course) to be regarded as a TP ⇔ “Experiment” two-way bridge, in which TP concepts are related to each other in the form of “laws” that are able to adequately describe the relationships obtaining among suitable chosen variables that describe the phenomenon one is interested in. In many instances, although not in all of them, these laws are integrated into a comprehensive theory (e.g., classical electromagnetism, based upon Maxwell’s equations) [22–26]. If recourse is made to MaxEnt ideas in order to describe thermodynamics, the above scheme becomes now IT as a part of TP WV Thermal Experiment; or in a more general scenario IT WV Phenomenon to be described: It should then be clear that the relation between an information measure and entropy is IM WV Entropy S:
128
A. Plastino / Physica A 342 (2004) 126 – 131
One can then categorically state that an IM is not an entropy! How could it be? The Erst belongs to the Topos Uranus, because it is a mathematical concept. The second to the laboratory, because it is a measurable physical quantity. Unnecessary controversies arise from the wrong assimilation (IM ≡ thermodynamic S, no matter what the context may be). All one can say is that, at most, in some special cases, an association IM ⇔ entropy S can be made. As shown by Jaynes [20], this association is both useful and proper in many situations. But, can one with certainty dare to venture the “extreme” notion that, in every conceivable case, Shannon s logarithmic IM WV Entropy is the only possible linkage? The controversy around Tsallis’ thermostatistics seems to revolve around such an absurd assertion.
3. MaxEnt rationale The central IM idea is that of giving quantitative form to the everyday concept of ignorance [17]. If, in a given scenario, N distinct outcomes (i = 1; : : : ; N ) are possible, then three situations may ensue [17]: (1) Zero ignorance: predict with certainty the actual outcome. (2) Maximum ignorance: Nothing can be said in advance. The N outcomes are equally likely. (3) Partial ignorance: we are given the probability distribution {Pi }; i = 1; : : : ; N . The underlying philosophy of the application of IT ideas to physics via the celebrated MaxEnt of Jaynes’ [15] is that originated by Bernoulli and Laplace (the fathers of Probability Theory) [20], namely: the concept of probability refers to a state of knowledge. An information measure quantiEes the information (or ignorance) content of a probability distribution [20]. If our state of knowledge is appropriately represented by a set of, say, M expectation values, then the “best”, least unbiased probability distribution is the one that • reSects just what we know, without “inventing” unavailable pieces of knowledge [17,20], and, additionally, • maximizes ignorance: the truth, all the truth, nothing but the truth. Such is the MaxEnt rationale [17]. It should be then patently clear that, in using MaxEnt, one is NOT maximizing a physical entropy. One is maximizing ignorance in order to obtain the least biased distribution compatible with the a priori knowledge, and Shannon’s IM is deEnitely not the only measure of ignorance. Many others exist. Some people have diTculty in making the proper distinction because their minds are Exed upon the wrong IM ≡ S linkage.
A. Plastino / Physica A 342 (2004) 126 – 131
129
4. The need for alternative IM’s The requirement of additivity for certain thermodynamic quantities places strict constraints with regards to the symmetries of the concomitant phase (or conEguration) space and is indivisibly linked with the homogeneity of the system under consideration, an assumption that remains frequently unmentioned (possibly because it is often fulElled). Today, exotic and complex thermodynamic systems or processes are the subject of considerable attraction: colossal magneto-resistance manganites, amorphous and glassy nano-clusters, high-energy collision processes, etc., characterized by the common feature of non-equilibrium states stationary for signiEcantly long periods of time (compared to typical time-scales of their microscopic dynamics). Scale invariance and hierarchical structures are here preserved, but the pertinent conEguration spaces are generally inhomogeneous. As a consequence, the additivity requirement ceases to be satisEed and traditional thermostatistics displays some shortcomings. The best theoretical description that has been thus far obtained uses the strictures of non-extensive thermostatistics (NET) [1,2,4]. Indeed, it is well known that the standard, extensive thermostatistics’ (ST) proper application Eeld includes systems for which (1) the spatial range of the pertinent microscopic interactions is small (short-range) (2) ditto for the microscopic memory time-range (3) dynamical evolution takes place in an Euclidean-like space-time. Outside such a realm ST does not, in general, adequately work [4,6] while NET does an excellent job [1–12]. We can, additionally, cite Boghosian’s recent work for the incompressible Navier–Stokes equation in Boltzmann lattice models [27], Baldovin and Robledo’s “at the edge of chaos” studies for the logistic map’s universality class [28,29], Oliveira et al.’s work on manganites (non-extensive magnetic materials) [30], and many others [31]. In applying MaxEnt, two distinct situations are often encountered. One either works with • Shannon’s logarithmic information measure, whose extremization leads to a probability distribution (PD) of exponential form. • Tsallis’ information measure [32], whose extremization leads to a power law-PD [12]. It is evident that, in addition to exponential distributions, we often encounter powerlaw distributions (PLD) as well. PLD are certainly ubiquitous in physics (critical phenomena are just a conspicuous example [33]). We just cannot dispense with IM’s alternative to Shannon’s one without giving up on much of physics. What a signiEcant sacriEce becomes necessary to assuage some unconscious inner urge to confuse an IM with a thermal entropy! Additionally some Hamiltonian systems (of hamiltonian H ) exist for which, if one connects, as usual [22], the average kinetic energy with the temperature T , one Ends
130
A. Plastino / Physica A 342 (2004) 126 – 131
that [1,2,4] dSq 1 ˙ ; dH T
(2)
which certainly authorizes one to establish, for these systems, the quite interesting link [Sq ⇔ thermal entropy], that it will be quite hard to disavow. 5. Conclusions As Hamlet said to Horatio, there are more things on heaven and Earth : : : . In a similar vein, there is • much more to MaxEnt than Shannon’s celebrated information measure • interesting probability distributions that are not of exponential form. In particular, there is a growing interest nowadays in exotic and complex thermodynamic systems or processes, as stated above, for which one has to deal with non-equilibrium states stationary for signiEcantly long periods of time. In these instances, a new thermostatitics, the so-called non-extensive one, has proved to be an extremely useful theoretical construct. Information measures other than Shannon’s are indispensable indeed. References [1] V. Latora, A. Rapisarda, C. Tsallis, Physica A 305 (2002) 129, and references therein. [2] G. Kaniadakis, M. Lissia, A. Rapisarda (Eds.), Nonextensive statistical mechanics and physical applications, Physica A (Special Vol.) 305 (1/2) (2002). [3] C. Tsallis, J. Stat. Phys. 52 (1988) 479. [4] M. Gell-Mann, C. Tsallis, Nonextensive Entropy—Interdisciplinary Applications, Oxford University Press, Oxford, 2003. [5] C. Tsallis, Braz. J. Phys. 29 (1999) 1, and references therein. [6] S. Abe, Y. Okamoto, Nonextensive Statistical Mechanics and its Applications, Lecture Notes in Physics, 560, Springer, Heidelberg, 2001. [7] A. Plastino, A.R. Plastino, Braz. J. Phys. 29 (1999) 50; A. Plastino, A.R. Plastino, Braz. J. Phys. 29 (1999) 79. [8] F. Pennini, A.R. Plastino, A. Plastino, Physica A 258 (1998) 446. [9] M.P. Almeida, Physica A 300 (2001) 424. [10] S. Abe, A.K. Rajagopal, Phys. Rev. Lett. 91 (2003) 120601. [11] A.R. Plastino, A. Plastino, Phys. Lett. A 177 (1993) 177. [12] A.R. Plastino, A. Plastino, Phys. Lett. A 193 (1994) 251. [13] M. Nauenberg, Phys. Rev. E 67 (2003) 036114. [14] A. Cho, Science 23 (2003) 1268. [15] E.T. Jaynes, Phys. Rev. 106 (1957) 620; E.T. Jaynes, Phys. Rev. 108 (1957) 171. [16] D.J. Scalapino, in: W.T. Grandy Jr., P.W. Milonni (Eds.), Physics and Probability. Essays in Honor of Edwin T. Jaynes, Cambridge University Press, NY, 1993, and references therein. [17] A. Katz, Principles of Statistical Mechanics, The Information Theory Approach, Freeman and Co., San Francisco, 1967.
A. Plastino / Physica A 342 (2004) 126 – 131
131
[18] C.E. Shannon, Bell System Technol. J. 27 (1948) 379. [19] T.M. Cover, J.A. Thomas, Elements of Information Theory, Wiley, NY, 1991. [20] E.T. Jaynes, in: R.D. Rosenkrantz (Ed.), Papers on Probability, Statistics and Statistical Physics, Reidel, Dordrecht, Holland, 1983, pp. 210–314. [21] B. Russell, A History of Western Philosophy, Simon & Schuster, NY, 1945. [22] R.B. Lindsay, H. Margenau, Foundations of Physics, Dover, NY, 1957. [23] P.W. Bridgman, The Nature of Physical Theory, Dover, NY, 1936. [24] P. Duhem, The Aim and Structure of Physical Theory, Princeton University Press, Princeton, New Jersey, 1954. [25] R.B. Lindsay, Concepts and Methods of Theoretical Physics, Van Nostrand, NY, 1951. [26] H. Weyl, Philosophy of Mathematics and Natural Science, Princeton University Press, Princeton, New Jersey, 1949. [27] B.M. Boghosian, P.J. Love, P.J. Coveney, I.V. Karlin, S. Succi, J. Yepez, cond-mat/0211093. [28] F. Baldovin, A. Robledo, Phys. Rev: E 66 (2002) 045104. [29] F. Baldovin, A. Robledo, cond-mat/0304410. [30] M.S. Reis, V.S. Amaral, J.P. Araujo, I.S. Oliveira, Phys. Rev. B 68 (2003) 014404. [31] A periodically updated bibliography on nonextensive thermostatistics can be found in the URL http://tsallis.cat.cbpf.br/biblio.htm. [32] C. Tsallis, R.S. Mendes, A.R. Plastino, Physica A 261 (1998) 534. [33] N. Goldenfeld, Lectures on Phase Transitions and the Renormalization Group, Addison-Wesley, NY, 1992.