🚀 go-pugleaf

+ "Simon" <s.salm@cable.a2000.nl>:

| X is a discrete random variable that takes on one of the values x1, x2,
| ..... , xn, with respective probabilities p1, p2, .... ,pn.
| 
| The entropy of X is defined as: H(X) = -Sum{pi*log(pi)} in which log
| means log with base 2 and Sum means summation over all i.
| 
| Please, could someone explain to me why H(X) is maximized when all
| of the pi are equal??  (It is quite easy to prove when n=2)(But what
| to do when n>2???)

If your library has a copy of the book "Information theory and coding"
by Norman Abramson (it's from 1963, so it may be hidden away in the
basement), you can find a clear explanation there.  Otherwise, you can
do it yourself, by using the Lagrange multiplier method on the sum
with the constraint Sum{pi}=1.  But the book is a good read if you can
get hold of it.

-- 
* Harald Hanche-Olsen     <URL:http://www.math.ntnu.no/~hanche/>
- "There arises from a bad and unapt formation of words
   a wonderful obstruction to the mind."  - Francis Bacon