In a note to What is Life? Schrödinger explained his use of this phrase.
... if I had been catering for them [physicists] alone I should have let the discussion turn on free energy instead. It is the more familiar notion in this context. But this highly technical term seemed linguistically too near to energy for making the average reader alive to the contrast between the two things.
Information theory
In
information theory and
statistics, negentropy is used as a measure of distance to normality.[4][5][6] Out of all
distributions with a given mean and variance, the normal or
Gaussian distribution is the one with the highest
entropy. Negentropy measures the difference in entropy between a given distribution and the Gaussian distribution with the same mean and variance. Thus, negentropy is always nonnegative, is invariant by any linear invertible change of coordinates, and vanishes
if and only if the signal is Gaussian.
Negentropy is defined as
where is the
differential entropy of the Gaussian density with the same
mean and
variance as and is the differential entropy of :
Correlation between statistical negentropy and Gibbs' free energy
There is a physical quantity closely linked to
free energy (
free enthalpy), with a unit of entropy and isomorphic to negentropy known in statistics and information theory. In 1873,
Willard Gibbs created a diagram illustrating the concept of free energy corresponding to
free enthalpy. On the diagram one can see the quantity called
capacity for entropy. This quantity is the amount of entropy that may be increased without changing an internal energy or increasing its volume.[9] In other words, it is a difference between maximum possible, under assumed conditions, entropy and its actual entropy. It corresponds exactly to the definition of negentropy adopted in statistics and information theory. A similar physical quantity was introduced in 1869 by
Massieu for the
isothermal process[10][11][12] (both quantities differs just with a figure sign) and then
Planck for the
isothermal-
isobaric process.[13] More recently, the Massieu–Planck
thermodynamic potential, known also as free entropy, has been shown to play a great role in the so-called entropic formulation of
statistical mechanics,[14] applied among the others in molecular biology[15] and thermodynamic non-equilibrium processes.[16]
In particular, mathematically the negentropy (the negative entropy function, in physics interpreted as free entropy) is the
convex conjugate of
LogSumExp (in physics interpreted as the free energy).
Brillouin's negentropy principle of information
In 1953,
Léon Brillouin derived a general equation[17] stating that the changing of an information bit value requires at least energy. This is the same energy as the work
Leó Szilárd's engine produces in the idealistic case. In his book,[18] Brillouin further explored this problem concluding that any cause of this bit value change (measurement, decision about a yes/no question, erasure, display, etc.) will require the same amount of energy.
^P. Comon, Independent Component Analysis – a new concept?, Signal Processing, 36 287–314, 1994.
^Didier G. Leibovici and Christian Beckmann,
An introduction to Multiway Methods for Multi-Subject fMRI experiment, FMRIB Technical Report 2001, Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB), Department of Clinical Neurology, University of Oxford, John Radcliffe Hospital, Headley Way, Headington, Oxford, UK.