Among probability distributions
which are nonzero over a
*finite* range of values
, the maximum-entropy
distribution is the *uniform* distribution. To show this, we
must maximize the entropy,

(D.33) |

with respect to , subject to the constraints

Using the method of *Lagrange multipliers* for optimization in
the presence of constraints [#!GillMurrayAndWright81!#], we may form the
*objective function*

(D.34) |

and differentiate with respect to (and renormalize by dropping the factor multiplying all terms) to obtain

(D.35) |

Setting this to zero and solving for gives

(D.36) |

(Setting the partial derivative with respect to to zero merely restates the constraint.)

Choosing to satisfy the constraint gives , yielding

(D.37) |

That this solution is a maximum rather than a minimum or inflection point can be verified by ensuring the sign of the second partial derivative is negative for all :

(D.38) |

Since the solution spontaneously satisfied , it is a maximum.

[How to cite this work] [Order a printed hardcopy] [Comment on this page via email]

Copyright ©

Center for Computer Research in Music and Acoustics (CCRMA), Stanford University