entropy

Article View: de.sci.mathematik

Article #8

From: Simon
Date: Wed, 13 Oct 1999 00:00

15 lines
1402 bytes

To all

X is a discrete random variable that takes on one of the values x1, x2,
..... , xn, with respective probabilities p1, p2, .... ,pn.

The entropy of X is defined as:  H(X) = -Sum{pi*log(pi)} in which log means
log with base 2 and Sum means summation over all i.

Please, could someone explain to me why H(X) is maximized when all of the pi
are equal??
(It is quite easy to prove when n=2)(But what to do when n>2???)

Thanks in advance

Simon

Technical Details

Message-ID: <7u0bf5$4k3$1@tesla.a2000.nl>

Path: arc1-ca.pugleaf.net!unknown!not-for-mail

Show Full Headers

Complete Article Headers

X-Google-Language: ENGLISH,ASCII-7-bit
X-Google-Thread: 10d67e,a1b4de0cf05f260e,start
X-Google-Attributes: gid10d67e,public
X-Google-Thread: 1032cb,a1b4de0cf05f260e,start
X-Google-Attributes: gid1032cb,public
X-Google-Thread: 10d29f,a1b4de0cf05f260e,start
X-Google-Attributes: gid10d29f,public
From: "Simon" <s.salm@cable.a2000.nl>
Subject: entropy
Date: 1999/10/13
Message-ID: <7u0bf5$4k3$1@tesla.a2000.nl>
X-Deja-AN: 536223599
Approved: Daniel Grayson, dan@math.uiuc.edu, moderator for sci.math.research
X-Trace: tesla.a2000.nl 939766053 4739 62.108.13.92 (12 Oct 1999 22:07:33 GMT)
Organization: A2000 Kabeltelevisie en Telecommunicatie
X-MSMail-Priority: Normal
X-Priority: 3
NNTP-Posting-Date: 12 Oct 1999 22:07:33 GMT
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Newsgroups: de.sci.mathematik,aus.mathematics,sci.math.research
Originator: dan@orion.math.uiuc.edu
X-Complaints-To: abuse@a2000.nl