Article View: de.sci.mathematik
Article #9Re: entropy
From: Robert
Date: Wed, 13 Oct 1999 00:00
Date: Wed, 13 Oct 1999 00:00
20 lines
1483 bytes
1483 bytes
Simon wrote:
> To all
>
> X is a discrete random variable that takes on one of the values x1, x2,
> ..... , xn, with respective probabilities p1, p2, .... ,pn.
>
> The entropy of X is defined as: H(X) = -Sum{pi*log(pi)} in which log means
> log with base 2 and Sum means summation over all i.
>
> Please, could someone explain to me why H(X) is maximized when all of the pi
> are equal??
> (It is quite easy to prove when n=2)(But what to do when n>2???)
>
differentiation with respect to pi yields Log(pi)+1=0, i=1...n, So you can see
immediately that the pi have to be equal. Sum(pi)=1 leads to the values of pi,
pi=1/n.
Technical Details
Message-ID:
<3804B633.DD151EF8@gte.tuwien.ac.at>Path:
arc1-ca.pugleaf.net!unknown!not-for-mailReferences:
<7u0bf5$4k3$1@tesla.a2000.nl>Show Full Headers
Complete Article Headers
X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 10d67e,a1b4de0cf05f260e X-Google-Attributes: gid10d67e,public X-Google-Thread: 1032cb,a1b4de0cf05f260e X-Google-Attributes: gid1032cb,public X-Google-Thread: 10d29f,a1b4de0cf05f260e X-Google-Attributes: gid10d29f,public From: Robert <robert@gte.tuwien.ac.at> Subject: Re: entropy Date: 1999/10/13 Message-ID: <3804B633.DD151EF8@gte.tuwien.ac.at> X-Deja-AN: 536306871 Content-Transfer-Encoding: 7bit Approved: Daniel Grayson, dan@math.uiuc.edu, moderator for sci.math.research References: <7u0bf5$4k3$1@tesla.a2000.nl> X-Accept-Language: en Content-Type: text/plain; charset=us-ascii Organization: Vienna University of Technology, Austria Mime-Version: 1.0 Newsgroups: de.sci.mathematik,aus.mathematics,sci.math.research Originator: dan@orion.math.uiuc.edu