Problem bank

Problem 263 of 333MediumCalculusP263

Maximum-entropy distribution

  1. Among all probability distributions (p1,…,pn)(p_1, \dots, p_n) on {1,…,n}\{1, \dots, n\}, which one maximizes the entropy H=−∑ipiln⁡piH = -\sum_i p_i \ln p_i? What changes if you also require the mean ∑ii pi\sum_i i\,p_i to equal a given mm? Describe the answer for a six-sided die constrained to have mean 4.5.