🚀 go-pugleaf

Suppose you have a parametric model for the probability density
function of a discrete random variable, and a set of observed values.
Call the model pdf p(x), and call the observed values x_1,...x_N.  The
goal is to find parameter values that make the model pdf p(x_i)
approximate the "true" but unknown pdf q(x_i).

One way to fit the model to the data would be to find parameter values
that minimize

                    -sum(log p(x_i))

i.e., to minimize the total self-information of the observed points
with respect to the model.  This "works" because for a given true pdf
q(x_i) and for all valid model pdfs p(x_i),

         E[-log(p(x_i))] = -sum( q(x_i) log p(x_i) )

is minimized when p = q.

I'd like a pointer to any papers that discuss this or similar
approaches to parametric pdf fitting.

Kris Popat
MIT Rm E15-391  Cambridge, MA  02139