8
$\begingroup$

Right now I am totally confused as to the difference between these two distributions. I think theoretical means that a given distribution that we already know its all information. However, for the empirical distribution, we also know all information about it. What is the exactly difference between them?

For in example,

In R, dnorm(): Obtain the density values for the theoretical normal distribution; why it isn't an empirical normal distribution?

In R, density(): fit an empirical density curve to a set of values; why in this case, it uses "empirical"?

$\endgroup$

3 Answers 3

12
$\begingroup$

In an nutshell, when you know what the distribution is and its parameters, you can construct the theoretical distribution.

So, in the case of R, the dnorm command returns the Standard Normal distribution. That is the distribution whose probability density function is: $$ f(x|\mu, \sigma) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac{(x - \mu)^2}{2 \sigma^2}} $$ and where we know $\mu = 0$ and $\sigma = 1$ so we actually have $$ f(x) = \frac{1}{\sqrt{2\pi}}\, e^{-\frac{x^2}{2}} $$ and $$ P(X \leq x) = \int_0^x \frac{1}{\sqrt{2\pi}}\, e^{-\frac{t^2}{2}}\; dt $$

That's because we start knowing everything.

With the EMPIRICAL distribution we start knowing nothing. What we have is a collection of observations, and we want to try and derive some knowledge from that collection. Perhaps we will fit a distribution, perhaps if we have enough observations, we'll just measure from those.

For example, if I have the following 10 numbers, I can create an empirical distribution: ${1, 2, 3, 4, 4, 5, 8, 9, 9, 10}$

Looking at just these numbers, the empirical probability of choosing a 5 or less is 60%, since I have 6 out of 10 observations of 5 or less.

What density does is run through the collection of observations and fit a kernel-smoothed density to them. It isn't normal, binomial, Poisson, Pareto, or anything in particular necessarily. It is a (sometimes) smoothed version of a histogram which can be treated like a density for calculations relating to the observations. We can try and fit theoretical distributions which are "close" in some way to the empirical. These fitted theoretical distributions can then be used as a proxy and we can use their properties for further fun and games.

$\endgroup$
3
  • 1
    $\begingroup$ "when you know ... and its parameters" There is a gray area. When we assume (not necessarily knowing) a certain distribution, but we estimate the parameters based in data, then I would still speak about a theoretical distribution as in the specific hypothetical/theoretical family of curves that are assumed to describe or approximate the population. $\endgroup$ Jan 3 at 14:26
  • 2
    $\begingroup$ So, it might be context dependent what the exact distinction is. $\endgroup$ Jan 3 at 14:32
  • $\begingroup$ @SextusEmpiricus, fair point, but one can say that once parameters have been estimated, we now use them and deal with the theroetical distribution given fitted parameters which "most closely" fits an empirical distribution for some definition of "most close". $\endgroup$
    – Avraham
    Jan 3 at 15:44
2
$\begingroup$

Simply put, an empirical distribution changes w.r.t. to the empirical sample, whereas a theoretical distribution doesn't w.r.t. to the sample coming from it.

Or put it another way, an empirical distribution is determined by the sample, whereas a theoretical distribution can determine the sample coming out of it.

$\endgroup$
0
$\begingroup$

Empirical Probability of an event is an "estimate" that the event will happen based on how often the event occurs after collecting data or running an experiment (in a large number of trials). It is based specifically on direct observations or experiences.

Theoretical Probability of an event is the number of ways that the event can occur, divided by the total number of outcomes. It is finding the probability of events that come from a sample space of known equally likely outcomes.

http://www.regentsprep.org/regents/math/algebra/apr5/theoprop.htm

$\endgroup$
1
  • 1
    $\begingroup$ The question asks about empirical distributions rather than probability estimates. To get a sense of what "empirical distribution" actually means, please search our site. $\endgroup$
    – whuber
    Sep 27, 2016 at 21:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.