READING LIST


Those who wish to learn more about the MaxEnt method are strongly encouraged to study the following references. This list was assembled for teaching of the Masters/PhD course in Maximum Entropy Analysis at the Neils Bohr Institute, Københavns Universitet, Denmark, in 2007, and has been updated regularly.

CORE READING MATERIALS
The application of MaxEnt is examined primarily in the following references:
 
OTHER RECOMMENDED READING 
The following other references are useful for a detailed understanding of the field:

Probability Theory: 
  • Feller, W. (1957; 1968), An Introduction to Probability Theory and Its Applications, vol 1., 2nd end, John Wiley, NY. (note: Feller takes the traditional "frequentist" view of probability)
  • Tijms. H. (2004), Understanding Probability : Chance Rules in Everyday Life Cambridge University Press.

Bayes’ Theorem and Bayesian Inference
  • Jeffreys, H. (1961) Theory of Probability, 3rd ed., Clarendon Press, Oxford.
  • Cox, R.T. (2001) Algebra of Probable Inference, The Johns Hopkins University Press.
  • Jaynes, E.T. (1988), How does the brain do plausible reasoning?, in Erickson, G. J. & Smith, C. R. (eds.) Maximum-Entropy and Bayesian Methods in Science and Engineering, vol. 1, Kluwer, Dordrecht, 1: http://bayes.wustl.edu/etj/articles/brain.pdf (low res).
  • Sivia, D.S. (1996), Data Analysis: A Bayesian Tutorial, Oxford University Press.

Maximum Entropy Analysis:
  • Kapur, J.N. & Kesevan, H.K. (1987), The Generalized Maximum Entropy Principle (with Applications), Sandford Educational Press, Waterloo, Canada.
  • Kapur, J.N. (1989), Maximum-Entropy Models in Science and Engineering, John Wiley & Sons (Asia) Pte Ltd, ISBN: 812240216X.
  • Tribus, M. (1969), Rational Descriptions, Decisions and Designs, Permagon Press, NY.
  • Levine, R.D. & Tribus, M. (eds) (1978), The Maximum Entropy Formalism,  MIT Press, Cambridge, MA.

You can also track down a series of AIP publications entitled “Bayesian Inference and Maximum Entropy Methods in Science and Engineering”, the proceedings of annual workshops on maximum entropy analysis and Bayesian inference.


Jaynes’ other papers:
Jaynes' entropy concentration theorem is outlined in:

Most of Jaynes' earlier critical papers are reproduced in:
  • Jaynes, E.T., Rosenkratz, R.D. (ed.) (1983) Papers on Probability, Statistics and Statistical Physics, D. Reidel Publ. Co., Dordrecht, Netherlands.

A biography of Jaynes, and copies of all of his papers (many scanned at low resolution), are available at
http://bayes.wustl.edu/etj/etj.html .

Traditional Statistical Mechanics and Thermodynamics
There are a vast number of books in this field. Some of the better ones include:
  • Atkins, P.W. (1982), Physical Chemistry, 2nd ed., Oxford University Press, Oxford, chap. 20, especially appendix A1.
  • Davidson, N. (1962) Statistical Mechanics, McGraw-Hill, NY (applies the combinatorial basis)
  • Desloge, E.A. (1966), Statistical Physics, Holt, Rinehart & Winston, Inc., NY.
  • Eyring, H., Henderson, D., Stover, B.J. & Eyring, E.M. (1964) Statistical Mechanics and Dynamics, John Wiley & Sons, NY.
  • Hill, T.L. (1956) Statistical Mechanics: Principles and Selected Applications, McGraw-Hill, NY.
  • Schrödinger, E. (1952) Statistical Thermodynamics, Cambridge U.P., Cambridge.
  • Tolman, R.C. (1938) The Principles of Statistical Mechanics, Oxford Univ. Press, London (applies the combinatorial basis).

HISTORICAL REFERENCES
Combinatorial Basis of Entropy (Boltzmann Principle)
The landmark papers in the establishment of the combinatorial basis of entropy are:

Probability Theory and Bayesian Inference

For accounts of the foundations of probability theory and Bayesian inference, see also:

Information Theory 
Several important contributions are:
  • Shannon, C.E. (1948) Bell System Technical Journal, 27: 379-423; 623-659.
  • Kullback, S. & Leibler, R.A. (1951), Annals Math. Stat., 22: 79-86.
  • Shore, J.E. & Johnson, R.W. (1980) IEEE Trans. Information Theory IT-26(1): 26-37.
  • Levine, R.D. (1980) An information theoretical approach to inversion problems, J. Phys. A: Math. Gen. 13: 91-108.

Thermodynamic Entropy and Free Energy: 
We cannot neglect the following:
  • Clausius, R. (1865) Poggendorfs Annalen 125: 335; English transl.: R.B. Lindsay, in J. Kestin (ed.) (1976) The Second Law of Thermodynamics, Dowden, Hutchinson & Ross, PA, (1976) 162.
  • Clausius,  R. (1876) Die Mechanische Wärmetheorie (The Mechanical Theory of Heat), F. Vieweg, Braunschwieg; English transl.: W.R. Browne (1879), Macmillan & Co., London.
  • Massieu, M. (1869) Comptes Rendus 69: 858-862; 1057-1061.
  • Gibbs, J.W. (1875-1878) Trans. Connecticut Acad., Oct. 1875-May 1876: 108; May 1877-July 1878: 343; Am. J. Sci. 16 (1878) 441; also in J.W. Gibbs (1961), The Scientific Papers of J. Willard Gibbs, Dover Publ., NY, 55.
History of Science 
Accounts of the fascinating individuals who contributed to probability theory are given in many texts and websites, and also in:
  • King, A.C. & Read, C.B. (1963), Pathways to Probability, Holt, Rinehart and Winston, Inc., NY.