2
$\begingroup$

Is it correct to think that the true probability of an event cannot be ever known?

When studying probability, in the first lectures, there are those typical exercises which start with sentences like: "One tosses a (fair) coin." or "In a bag there are 4 blue marbles and 6 red marbles.". In the first case we usually say that $P($heads$)=0.5$ and in the latter: $P($blue marble$) = 0.4$.

After a few lectures, there comes the notion of "maximum likelihood estimation" (MLE) and one can observe that in the second case above (with marbles) we just fit a Bernoulli distribution to the data we had (10 marbles: BBBB RRRRRR).

I don't know how to think about the first example above (with coin tossing). But I have the following intuition (which would help me a lot I guess) and I hope you can help me to understand if I am correct or not:

  1. The true probability of an event cannot be ever known.

  2. In the first case (with coin tossing) the probability was estimated from context/text. Can we say that I applied the closed world assumption?

  3. In the second case (with marbles) the probability was estimated from data (using MLE).

EDIT:

As a conclusion, the probability cannot be known, but can be assumed to be some number. My two examples were not examples of estimating probabilities (because there was NO data: I incorrectly said that the marbles were data...), but of making someone to think of a specific assumption: equally likely outcomes assumption.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ So you know that the bag contains 4 blue and 6 des marbles AND your Data was BBBB RRRRRR ? $\endgroup$
    – Sebastian
    Commented Aug 14, 2018 at 8:52
  • $\begingroup$ Yes. I was just trying to say that the 4 blue and 6 red marbles were my data. I mean I can consider them data, can't I? $\endgroup$
    – user_anon
    Commented Aug 14, 2018 at 8:56
  • $\begingroup$ In your examples, the writer wants you to consider that the true probabilities are known. In real life, probability is never known. There is no fair coin and the painting of the marbles have an unknown and very very very slight effect on the propensity you have to catch them. $\endgroup$
    – Arnaud
    Commented Aug 14, 2018 at 9:27
  • $\begingroup$ So, the second example has nothing to do with maximum likelihood estimation? $\endgroup$
    – user_anon
    Commented Aug 14, 2018 at 9:33
  • 1
    $\begingroup$ There's a distinction between a model of a situation and the situation itself. In the case of a coin we assume P(H)=1/2 not because people have tossed coins thousands of times and discovered them to give very close to fair results, but from symmetry considerations. Similarly, when drawing marbles from a bag we assume (again via symmetry considerations) that on any draw the remaining marbles are all equally likely to be drawn. These models (invocations of some kind of symmetry in these two cases) will be plausible idealizations of reality (we can improve the quality of these with care). $\endgroup$
    – Glen_b
    Commented Aug 14, 2018 at 15:13

2 Answers 2

Reset to default
-1
$\begingroup$

The true probability of an event cannot be ever known.

Right and wrong, depending on the assumption / the point in time. Since a binary valued random variable is in any case Bernoulli distributed and you do not have a choice of how to model that, let us move away from that example and consider instead the following: We wonder whether or not the following data is normally distributed and if so, what mean it has:

-0.33,1.4,0.64,-0.11,0.51,0.4,1.66,0.28,0.51,0.35,-0.38,0.1,1.64,-0.88,0.12,1.36,-0.23,-1.05,-0.87,-0.39

Right now you have the choice of modelling that with a t-distribution, a normal distribution and so on. Furthermore it is data from the real world, hence yes: we can never be absolutely, 100% sure that this data even was produced from a random variable. Maybe all the concepts of probability do not apply here because there is simply no rule behind how this data was generated and maybe the next number that this process would have generated is 100001344.99 ( a number that would not fit at all into the pattern). But the true question here is: does it really matter? The anser is no: We simply try to model this data with different distribution and 'do the best we can'. In the end (in the real world) we want to make optimize something, reduce costs, reduce waste or so. So, if we were able to do that by using a (maybe somewhat inadequate) model, do we care as long as we can make "good money" out of it? I highly doubt that :-)

On the other hand, once you have selected a model (and therefore explicitly assumed that the data was in fact generated by a random variable and that this random variable in fact has a normal distribution with some unknown parameters) then you can compute everything you want (like $P[X > 0]$) explicitly!

On questions 2) and 3):

We always get insights about the data by using two sources:

1) context / business knowledge / experience from the past

2) the data

We use 1) to select the model and then we use 2) to adjust the parameters of the model. Examples for 1):

  • Any random variable with only two outcomes is Bernoulli distributed.
  • We know that the data is the hapiness index of different people. We know from earlier experiments that most of the people are "neutrally happy" (i.e. data gathers around 0) and that only very few are extraordinarily happy (little data that is very positive) and only very few are extraordinarily unhappy (little data that is very negative).

Clearly, the coice of the model influences what you get out of the experiment and if everybody uses the normal distribution because, hey, everybody before me did that so I am doing it as well, then we may keep ourselves from getting valuable new insights (by trying new distributions/models maybe?).

Improve this answer
$\endgroup$
4
  • $\begingroup$ As a conclusion, the probability cannot be known, but can be assumed to be some number. My two examples were not examples of estimating probabilities (because there was NO data: I incorrectly said that the marbles were data...), but of making someone to think of a specific assumption: equally likely outcomes assumption. Right? $\endgroup$
    – user_anon
    Commented Aug 14, 2018 at 10:32
  • $\begingroup$ Yes, you are right. In order to fit a distribution you need 'much' empirical evidence (mostly in form of data). $\endgroup$
    – Fabian Werner
    Commented Aug 14, 2018 at 11:18
  • $\begingroup$ I have a related question: when assuming something (eg equally likely outcomes), we "are" bayesians or frequentists? I thought the latter, but our assumption is subjective I guess so we might be bayesians... $\endgroup$
    – user_anon
    Commented Aug 16, 2018 at 7:16
  • 1
    $\begingroup$ To make a very long and mathematically not always precise story short: Lets say $\Theta$ is your model parameter and $X$ is the data (probably involving input features, true answers and so on) then frequentists maximize $p(x|\theta)$. Bayesians also put the assumption of the form of $p(\theta)$ into the whole model and then optimize $p(\theta|x) = p(x|\theta)p(\theta)/p(x)$. Since you have not yet stated something about $p(\theta)$ I guess that you still follow the frequentists approach so far... $\endgroup$
    – Fabian Werner
    Commented Aug 16, 2018 at 7:40
1
$\begingroup$
  1. The true probability of an event cannot be ever known.

Let's start by saying that a model is always an approximation, so in this sense your sentence is correct. If you are dealing with data measured from a real world events, then any distribution you will attribute to it you will obtain an error.

  1. In the first case (with coin tossing) the probability was estimated from context/text. Can we say that I applied the closed world assumption?

In this case it is not an estimation, but an assumption. The coin is an abstract event that we are using to represent a probability and it is described by some hypothesis that are given.

I am not familiar with the CWA, but I don't think it relates well with stocastic processes as a concept.

  1. In the second case (with marbles) the probability was estimated from data (using MLE).

In the case of marbles, the settings of the experiments are not data, but assumptions. The data would be the outcomes of your tries.

In MLE your assumption is the model you are trying to fit on the data you have. You don't need to know about the number of marbles in the pocket when using MLE.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.