🚀 go-pugleaf

meserve (meserve@hal.COM) wrote:

: Is it possible to draw an equally direct
: equivalence relation between energy and information?

This is an ancient topic going all the way back to Maxwell's demon.  There is
a relationship but it is not direct.  It is the Second Law of
Thermodynamics.  Basically, to gain some information one must reduce the
entropy of a system.  To do that one must extract heat from the system (or
rather, allow it to flow out).  The Second Law is:

  dS >= dq/T

where dq is the heat into the system at temperature T and dS is the increase
in entropy for that heat.  (Notice how much clearer this is than "negentropy
principles".)  Multiplying both sides by -1 we get

  -dS <= -dq/T

A decrease in entropy (-ds) corresponds to an information gain.  You can see
that the heat must flow out (-dq).  Further discussion and a more complete
derivation is in:

http://www-lmmb.ncifcrf.gov/~toms/paper/edmm
or equivalently
ftp://ftp.ncifcrf.gov/pub/delila/edmm.ps

see also

http://www-lmmb.ncifcrf.gov/~toms/paper/secondlaw

I have assumed for a while now that the first person to see this and to
convert the formula above into the form Emin = kTln(2) joules/bit was von
Neumann in:

@inproceedings{vonNeumann1966,
author = "von Neumann, John",
editor = "A. W. Burks",
title = "{Fourth University of Illinois lecture}",
booktitle = "Theory of Self-Reproducing Automata",
publisher = "University of Illinois Press",
address = "Urbana",
pages = "66-67",
callnumber = "QA267.V55",
comment = "This is probably the earliest statement of kTln(2) (1949 lecture)",
comment = "since Szilard didn't say it first because bits weren't defined in",
comment = "1929.",
year = "1966"}

However my father just sent me Bell Telephone Technical Memorandum 52-180-30
by I. G. Wilson dated August 1, 1952, in which on page 11 Wilson states

"INFORMATION AND ENERGY
   In information theory, H is expressed in bits per symbol;
in thermodynamics, in joules per degree.  In suggesting a link
between information and energy, Felker has worked out the minimum
energy cost of a bit as 0.114 x 10^-19 joules."

How amazing!  It may not have been von Neumann!  Who was Felker?  Did he
publish this?  If so, where and when?  Is I. G. Wilson still around?

Was Felker right?  This one is easy to find out.  If we just equate:

   0.114 x 10^-19 joules/bit = k T ln(2)

we can compute T.  I got 273 kelvin, so he apparently was computing for 0
degrees C.  Fair enough, he clearly had to understand what he was doing.

  Tom Schneider
  National Cancer Institute
  Laboratory of Mathematical Biology
  Frederick, Maryland  21702-1201
  toms@ncifcrf.gov
  http://www-lmmb.ncifcrf.gov/~toms/

ps:  Please post responses, don't email me directly unless it is for a
private project you would like to discuss.  This way more people can think
about the (in this case historical) problem and contribute to it.  Thanks!