🚀 go-pugleaf

"Simon" <s.salm@cable.a2000.nl> writes:

>To all

>X is a discrete random variable that takes on one of the values x1, x2,
>..... , xn, with respective probabilities p1, p2, .... ,pn.

>The entropy of X is defined as:  H(X) = -Sum{pi*log(pi)} in which log means
>log with base 2 and Sum means summation over all i.

>Please, could someone explain to me why H(X) is maximized when all of the pi
>are equal??
>(It is quite easy to prove when n=2)(But what to do when n>2???)

Are you familiar with the method of Lagrange multipliers? Roughly
speaking it says that if H(p1,p2,..pn) has a maximum on the (hyper)surface
F(p1,p2,...pn)=0 then at that point the gradients of G and F are linerly
dependent.

Applying this with the entropy H (considered a function of p1,...,pn)
and F(p1,...pn) being p1 + p2 + ... +pn -1, we get
grad H = (log(p1) - 1 ,...,log(pn) - 1) = k*grad F = (k,k,...,k).
at the maximum.

--
Dan Luecking                           Dept. of Mathematical Sciences     
luecking@comp.uark.edu                 University of Arkansas      
http://comp.uark.edu/~luecking/        Fayetteville, AR 72101