8
$\begingroup$

I have a set of data points $X_i, y_i$ where $x$ are the independent variables and I believe each $y_i$ can be modeled as being drawn from a exponential distributions with parameters $\lambda_i$.

If I use $X_i$ to predict $\lambda_i$, how can I evaluate the quality of my predicted distributions with respect to the observations $y_i$?

Edit: This is essentially the same question as How to evaluate quality of probability estimator for Bernoulli experiments? but in a continuous context rather than a binomial context. It's not obvious to me what to use in this case instead of cross-entropy.

$\endgroup$
7
$\begingroup$

The standard approach to this is using the log-likelihood of the exponential distribution. This is actually exactly how the cross-entropy is derived, it is the log-likelihood of the Bernoulli distribution.

In the case of an exponential distribution, the pdf is:

$$ f(y; \lambda) = \lambda e^{-\lambda y} $$

So the log-likelihood is:

$$ LL(\lambda_i; y_i ) = \log(f(y_i; \lambda_i)) = \log(\lambda_i) - \lambda_i y_i$$

So, if $y_i$ are your true values, and $\lambda_i$ are your predictions, an exponential model would minimize:

$$ LL(\{\lambda_i\}; \{y_i\}) = \sum_i \log(\lambda_i) -\lambda_i y_i$$

Fitting models by maximizing the log-likelihood in this way leads to the the theory of generalized linear models; the exponential model is a special case.

$\endgroup$
3
$\begingroup$

The standard way to assess predictive distributions is via scoring rules. The log-likelihood that Matthew Drury recommends is one example, it's the logarithmic scoring rule. There are also others. Merkle & Steyvers (2013, Decision Analysis) discuss how different scoring rules hang together, and how to choose one.

More information can be found in the tag wiki, and we have a number of questions carrying the tag.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.