3
$\begingroup$

I have to come up with a way to measure the 'quality' of a distribution for a research project.

We collect data over a a period of time $t_0$ through $t_1$ and then estimate the distribution that fits this data. At a later time, $t_2$, we collect a single piece of data. Is there a way to measure the likelihood of the initial estimated distribution being 'correct', given this new data point?

We assume that the data points are independent random variables.

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ I assume you mean the data points independent conditioned on the generating process. If they're totally independent, then collecting data for a period $(t_0, t_1)$ tells you nothing about any other period! $\endgroup$
    – Neil G
    Commented Jul 21, 2015 at 3:39

3 Answers 3

Reset to default
3
$\begingroup$

Call your original sample $X = \{x_1, \ldots, x_n \}$. If you have a density estimate $\hat{f}(x|X)$ computed from your data, then the likelihood of your new data point $u$ is just $\mathcal{L}(u)=\hat{f}(u)$. Higher values of $\mathcal{L}$ imply it's more plausible that the new data point was obtained from the same underlying distribution of the original sample.

Improve this answer
$\endgroup$
1
$\begingroup$

Not without some additional assumptions.

Imagine the distribution the data are drawn from happens to consist of $n+1$ possible equally likely values, and that in $(t_0,t_1)$ you observe $n$ of those values. At time $t_2$ you observe the $(n+1)$th possible value for that distribution.

It doesn't matter how far away that last point is from the rest of the data, the observed sample is no more "weird" than it taking any of the other values for the last point; indeed the MLE of the distribution (the empirical cdf) is the closest possible to the true distribution.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the answer! I see your point there... Can you give an example of what additional assumptions might help out? E.g., If we assume it's a normal distribution? I was trying to use Bayes' Theorem to find the probability of it being the distribution given the data point but fell down at the point where I need the probability of that distribution... $\endgroup$
    – Pete Ludlow
    Commented Jul 21, 2015 at 3:03
  • $\begingroup$ What does "more weird" mean? Surely some values for the point at $t_2$ have higher surprisal than others given the predictive distribution. Similarly, some distributions are more likely than others given the value for the point at $t_2$. $\endgroup$
    – Neil G
    Commented Jul 21, 2015 at 3:38
1
$\begingroup$

Let your inferred distribution from the data collected in $(t_0, t_1)$ have parameters $\theta$. The likelihood over $\theta$ is proportional to the density of the point observed at $t_2$ given $\theta$:

$$L(\theta \mid x_{t_2}) = f(x_{t_2} \mid \theta).$$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.