- Email: [email protected]
Maximum entropy in action
Journal of Atmospheric and Terrestrial Physics, Vol.57, No. 1, pp. 99-104, 1995
ElsevierScienceLtd Printedin GreatBritain 0021-9169/95 $9.50+0.00
Pergamon
Book reviews Maximum Entropy in Action, BUCK, BRIANand MACAULAY, VINCENT A., 1991, 220 pp., Clarendon Press, Oxford, £30 pb, ISBN 0-19-8539630. More than 35 yr have now passed since Edwin Jaynes published his landmark papers on the Principle of Maximum Entropy (PME), within the context of statistical physics. These papers led to a stunning clarification of the foundations of equilibrium statistical mechanics and pointed the way toward eventual formulation of the nonequilibrium problem in a similar manner. While various applications were carried out in subsequent years, and fresh insights into numerous many-body systems uncovered, this theoretical development had little immediate impact on what conventional wisdom deemed to be the burning issues in the field. Many of the mathematical difficulties traditionally associated with the general many-body problem remained, albeit often in different forms. As with Shannon's generalization of the entropy concept in 1948, however, the insights and tools introduced by Jaynes transcended the immediate application for which they were developed, as he well recognized. Indeed, the PME was seen to be a powerful tool of scientific inference in itself and, as such, applicable to many other fields of endeavor. That is, Shannon discovered 1hat the entropy of a probability distribution encompassed the historical application to thermodynamics, while Jaynes realized that the maximization of a similar entity by Gibbs was in reality a special case of a more general principle of probability theory. The breakthroughs in applying these ideas in new ways can be attributed to Burg's application to exploration geophysics in 1967, and to the imaginative use of the PME in radio astronomy by Gull and Daniell in 1978. In almost every field concerned with data analysis today, one readily sees the influence of the PME, as well as the broader reach of Bayesian inference itself. Unfortunately, there is only a scattered literature discussing these method:~ of analysis, most often hidden within journals relating to various specialities. The only real foci have been the proceedings of the annual Maxent Workshops over the past 12 yr, which tended to survey the numerous research applications in a technical manner. However, in the summer of 1989, almost the entire body of what might be called the 'Cambridge School of Maximum Entropy and Bayesian Analysis' gzthered in Oxford to convene a summer school in the subject, which resulted in this generally fine collection of introductory articles. With one exception, the authors are recognized experts in their various fields, representing a reasonabte spectrum of interests. The volume is rather welcome not only because it is almost unique in this field, but also because it spans commendably the gulf between elementary ideas and their technical applications. The historic development of maximum-entropy tools in data analysis over the past two decades is sketched nicely in the editors' introduction (though, in my view, they are remiss in failing to note the impetus provided earlier by John Burg). A first chapter by Geoff Daniell provides an introduction and background material for maximum-entropy methods in data processing, and John Skilling discusses some of the
fundamental technical points justifying this approach in the second chapter. There then follow four chapters on applications in nuclear magnetic resonance, spectroscopy, plasma physics, and X-ray crystallography, and two more related to basic issues in statistical mechanics and thermodynamics. Steve Gull, in particular, provides some new insights into the old problem of Brownian motion that should be appreciated by those who have wrestled with that phenomenon. I noted a number of minor quibbles upon reading through the various articles, but only one seems at all disturbing. The article by G6rard Bricogne on X-ray crystallography and the phase problem really has little of value to say about the application of maximum entropy in this area. The notion is employed as a minor adjunct to old ways of analyzing this problem, rather than as a completely new method that might provide new and deeper insights for the experimenter. The reader interested in this particular application might be rewarded far more by perusing Richard Bryan's article on the same subject in the proceedings of the Maxent Workshop held in Cambridge in 1988 (Maximum Entropy and Bayesian Methods, J. Skilling (Ed.), Kluwer, Dordrecht, 1989). Nevertheless, this is, in general, a collection of nice expositions that can provide a solid introduction to the use of maximum-entropy techniques in data analysis, and in that sense fills a definite need. One looks for similar volumes from here and elsewhere in the future---and hopefully as well done--on applications in other fields such as geophysics, medical diagnostics and economics. W. T. GRANDY University of Wyoming Statistics in the Environmental & Earth Sciences, WALDENA. T. and GUTTORP P. (Eds), 1992, 306 pp., Hodder & Stoughton Limited, £49.50 hb, ISBN 0-340-54530-5. There is an undoubted need for scientists involved in atmospheric and space science to appreciate the importance of statistics in the treatment of new experimental results. This book tackles several important practical problems and, in so doing, links intriguing strands of current statistical thinking and methodology. This advanced book contains 13 chapters by different experts. The first half is concerned with environmental issues. Chapter 1 discusses the evaluation of different models for forecasting (e.g. of plumes from high chimneys, urban ozone, acid rain or climate change) ; issues such as the significance of departures of certain forecast parameters and the relevance of statistical inference are addressed. The optimum design of a monitoring network, in terms of optimising the information derived from it, is considered in chapter 2, with chapters 3 and 4 dealing specifically with spatially dependent data and the estimation of spatial covariances. Rainfall modelling, so important in the context of global change and within individual convective cells, is considered in chapters 5 and 6, respectively. The second half of the book covers issues in the Earth sciences, especially those involving time series analysis and
99
ElsevierScienceLtd Printedin GreatBritain 0021-9169/95 $9.50+0.00
Pergamon
Book reviews Maximum Entropy in Action, BUCK, BRIANand MACAULAY, VINCENT A., 1991, 220 pp., Clarendon Press, Oxford, £30 pb, ISBN 0-19-8539630. More than 35 yr have now passed since Edwin Jaynes published his landmark papers on the Principle of Maximum Entropy (PME), within the context of statistical physics. These papers led to a stunning clarification of the foundations of equilibrium statistical mechanics and pointed the way toward eventual formulation of the nonequilibrium problem in a similar manner. While various applications were carried out in subsequent years, and fresh insights into numerous many-body systems uncovered, this theoretical development had little immediate impact on what conventional wisdom deemed to be the burning issues in the field. Many of the mathematical difficulties traditionally associated with the general many-body problem remained, albeit often in different forms. As with Shannon's generalization of the entropy concept in 1948, however, the insights and tools introduced by Jaynes transcended the immediate application for which they were developed, as he well recognized. Indeed, the PME was seen to be a powerful tool of scientific inference in itself and, as such, applicable to many other fields of endeavor. That is, Shannon discovered 1hat the entropy of a probability distribution encompassed the historical application to thermodynamics, while Jaynes realized that the maximization of a similar entity by Gibbs was in reality a special case of a more general principle of probability theory. The breakthroughs in applying these ideas in new ways can be attributed to Burg's application to exploration geophysics in 1967, and to the imaginative use of the PME in radio astronomy by Gull and Daniell in 1978. In almost every field concerned with data analysis today, one readily sees the influence of the PME, as well as the broader reach of Bayesian inference itself. Unfortunately, there is only a scattered literature discussing these method:~ of analysis, most often hidden within journals relating to various specialities. The only real foci have been the proceedings of the annual Maxent Workshops over the past 12 yr, which tended to survey the numerous research applications in a technical manner. However, in the summer of 1989, almost the entire body of what might be called the 'Cambridge School of Maximum Entropy and Bayesian Analysis' gzthered in Oxford to convene a summer school in the subject, which resulted in this generally fine collection of introductory articles. With one exception, the authors are recognized experts in their various fields, representing a reasonabte spectrum of interests. The volume is rather welcome not only because it is almost unique in this field, but also because it spans commendably the gulf between elementary ideas and their technical applications. The historic development of maximum-entropy tools in data analysis over the past two decades is sketched nicely in the editors' introduction (though, in my view, they are remiss in failing to note the impetus provided earlier by John Burg). A first chapter by Geoff Daniell provides an introduction and background material for maximum-entropy methods in data processing, and John Skilling discusses some of the
fundamental technical points justifying this approach in the second chapter. There then follow four chapters on applications in nuclear magnetic resonance, spectroscopy, plasma physics, and X-ray crystallography, and two more related to basic issues in statistical mechanics and thermodynamics. Steve Gull, in particular, provides some new insights into the old problem of Brownian motion that should be appreciated by those who have wrestled with that phenomenon. I noted a number of minor quibbles upon reading through the various articles, but only one seems at all disturbing. The article by G6rard Bricogne on X-ray crystallography and the phase problem really has little of value to say about the application of maximum entropy in this area. The notion is employed as a minor adjunct to old ways of analyzing this problem, rather than as a completely new method that might provide new and deeper insights for the experimenter. The reader interested in this particular application might be rewarded far more by perusing Richard Bryan's article on the same subject in the proceedings of the Maxent Workshop held in Cambridge in 1988 (Maximum Entropy and Bayesian Methods, J. Skilling (Ed.), Kluwer, Dordrecht, 1989). Nevertheless, this is, in general, a collection of nice expositions that can provide a solid introduction to the use of maximum-entropy techniques in data analysis, and in that sense fills a definite need. One looks for similar volumes from here and elsewhere in the future---and hopefully as well done--on applications in other fields such as geophysics, medical diagnostics and economics. W. T. GRANDY University of Wyoming Statistics in the Environmental & Earth Sciences, WALDENA. T. and GUTTORP P. (Eds), 1992, 306 pp., Hodder & Stoughton Limited, £49.50 hb, ISBN 0-340-54530-5. There is an undoubted need for scientists involved in atmospheric and space science to appreciate the importance of statistics in the treatment of new experimental results. This book tackles several important practical problems and, in so doing, links intriguing strands of current statistical thinking and methodology. This advanced book contains 13 chapters by different experts. The first half is concerned with environmental issues. Chapter 1 discusses the evaluation of different models for forecasting (e.g. of plumes from high chimneys, urban ozone, acid rain or climate change) ; issues such as the significance of departures of certain forecast parameters and the relevance of statistical inference are addressed. The optimum design of a monitoring network, in terms of optimising the information derived from it, is considered in chapter 2, with chapters 3 and 4 dealing specifically with spatially dependent data and the estimation of spatial covariances. Rainfall modelling, so important in the context of global change and within individual convective cells, is considered in chapters 5 and 6, respectively. The second half of the book covers issues in the Earth sciences, especially those involving time series analysis and
99