Questions tagged [probability]
A probability provides a quantitative description of the likely occurrence of a particular event.
1,407
questions
179
votes
6
answers
116k
views
Can a probability distribution value exceeding 1 be OK?
On the Wikipedia page about naive Bayes classifiers, there is this line:
$p(\mathrm{height}|\mathrm{male}) = 1.5789$ (A probability distribution over 1 is OK. It is the area under the bell curve ...
- 4,689
26
votes
1
answer
7k
views
Is accuracy an improper scoring rule in a binary classification setting?
I have recently been learning about proper scoring rules for probabilistic classifiers. Several threads on this website have made a point of emphasizing that accuracy is an improper scoring rule and ...
- 261
295
votes
16
answers
104k
views
Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?
It seems that through various related questions here, there is consensus that the "95%" part of what we call a "95% confidence interval" refers to the fact that if we were to exactly replicate our ...
- 13k
54
votes
5
answers
32k
views
Generic sum of Gamma random variables
I have read that the sum of Gamma random variables with the same scale parameter is another Gamma random variable. I've also seen the paper by Moschopoulos describing a method for the summation of a ...
- 1,117
643
votes
12
answers
466k
views
What is the difference between "likelihood" and "probability"?
The wikipedia page claims that likelihood and probability are distinct concepts.
In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a ...
- 7,291
32
votes
3
answers
3k
views
I've heard that ratios or inverses of random variables often are problematic, in not having expectations. Why is that?
The title is the question. I am told that ratios and inverses of random variables often are problematic. What is meant is that expectation often do not exist. Is there a simple, general explication of ...
- 71k
40
votes
4
answers
18k
views
Functions of Independent Random Variables
Is the claim that functions of independent random variables are themselves independent, true?
I have seen that result often used implicitly in some proofs, for example in the proof of independence ...
- 19.2k
21
votes
6
answers
13k
views
$P[X=x]=0$ when $X$ is a continuous variable
I know that for any continuous variable $P[X=x]=0$.
But I can't visualize that if $P[X=x]=0$, there is an infinite number of possible $x$'s. And also why do their probabilities get infinitely small ?
- 1,527
61
votes
4
answers
65k
views
How to generate correlated random numbers (given means, variances and degree of correlation)?
I'm sorry if this seems a bit too basic, but I guess I'm just looking to confirm understanding here. I get the sense I'd have to do this in two steps, and I've started trying to grok correlation ...
- 721
54
votes
4
answers
83k
views
How does linear regression use the normal distribution?
In linear regression, each predicted value is assumed to have been picked from a normal distribution of possible values. See below.
But why is each predicted value assumed to have come from a normal ...
- 13.2k
66
votes
14
answers
11k
views
What is the most surprising characterization of the Gaussian (normal) distribution?
A standardized Gaussian distribution on $\mathbb{R}$ can be defined by giving explicitly its density:
$$ \frac{1}{\sqrt{2\pi}}e^{-x^2/2}$$
or its characteristic function.
As recalled in this ...
30
votes
5
answers
12k
views
Time taken to hit a pattern of heads and tails in a series of coin-tosses
Inspired by Peter Donnelly's talk at TED, in which he discusses how long it would take for a certain pattern to appear in a series of coin tosses, I created the following script in R. Given two ...
- 403
54
votes
8
answers
29k
views
How to tell the probability of failure if there were no failures?
I was wondering if there is a way to tell the probability of something failing (a product) if we have 100,000 products in the field for 1 year and with no failures? What is the probability that one of ...
- 541
34
votes
4
answers
10k
views
Can anyone clarify the concept of a "sum of random variables"
In my probability class the terms "sums of random variables" is constantly used. However, I'm stuck on what exactly that means?
Are we talking about the sum of a bunch of realizations from a random ...
- 505
14
votes
3
answers
2k
views
How exactly do Bayesians define (or interpret?) probability?
Part of a series of trying to understand Bayesian vs frequentist: 1 2 3 4 5 6 7
I think I get the difference of how Bayesians and frequentists approach choosing between hypotheses, but I'm not quite ...
- 2,280
47
votes
7
answers
21k
views
Combining probabilities/information from different sources
Lets say I have three independent sources and each of them make predictions for the weather tomorrow. The first one says that the probability of rain tomorrow is 0, then the second one says that the ...
- 591
33
votes
5
answers
3k
views
Wikipedia entry on likelihood seems ambiguous
I have a simple question regarding "conditional probability" and "Likelihood". (I have already surveyed this question here but to no avail.)
It starts from the Wikipedia page on likelihood. They say ...
- 1,567
51
votes
9
answers
39k
views
Bayesian vs frequentist Interpretations of Probability
Can someone give a good rundown of the differences between the Bayesian and the frequentist approach to probability?
From what I understand:
The frequentists view is that the data is a repeatable ...
- 1,435
29
votes
3
answers
17k
views
Why Normalizing Factor is Required in Bayes Theorem?
Bayes theorem goes
$$ P(\textrm{model}|\textrm{data}) = \frac{P(\textrm{model}) \times P(\textrm{data}|\textrm{model})}{P(\textrm{data})} $$
This is all fine. But, I've read somewhere:
Basically, ...
28
votes
3
answers
15k
views
Extreme Value Theory - Show: Normal to Gumbel
The Maximum of $X_1,\dots,X_n. \sim$ i.i.d. Standardnormals converges to the Standard Gumbel Distribution according to Extreme Value Theory.
How can we show that?
We have
$$P(\max X_i \leq x) = P(...
- 1,181
26
votes
4
answers
18k
views
Comparing and contrasting, p-values, significance levels and type I error
I was wondering if anybody could give a concise rundown as to the definitions and uses of p-values, significance level and type I error.
I understand that p-values are defined as "the probability of ...
- 1,435
43
votes
7
answers
20k
views
How often do you have to roll a 6-sided die to obtain every number at least once?
I've just played a game with my kids that basically boils down to: whoever rolls every number at least once on a 6-sided die wins.
I won, eventually, and the others finished 1-2 turns later. Now I'm ...
- 1,668
27
votes
11
answers
48k
views
How to easily determine the results distribution for multiple dice?
I want to calculate the probability distribution for the total of a combination of dice.
I remember that the probability of is the number of combinations that total that number over the total number ...
- 665
34
votes
2
answers
7k
views
How many stickers do I need to complete my FIFA Panini album?
I'm playing the FIFA Panini Online Sticker Album, which is an Internet adaption of the classic Panini albums that are usually published for the soccer world cup, European championship, and possibly ...
- 443
24
votes
6
answers
26k
views
Distribution of ratio between two independent uniform random variables
Supppse $X$ and $Y$ are standard uniformly distributed in $[0, 1]$, and they are independent, what is the PDF of $Z = Y / X$?
The answer from some probability theory textbook is
$$
f_Z(z) =
\begin{...
- 2,598
9
votes
2
answers
3k
views
Risk of extinction of Schrödinger's cats
I am interested in how uncertainty can be accounted for when considering the risk of extinction of a species. Forgive me for extending a rather tired thought experiment, but at least it's familiar ...
- 209
189
votes
4
answers
49k
views
Why do we need sigma-algebras to define probability spaces?
We have a random experiment with different outcomes forming the sample space $\Omega,$ on which we look with interest at certain patterns, called events $\mathscr{F}.$ Sigma-algebras (or sigma-fields) ...
- 25.2k
50
votes
2
answers
69k
views
PP-plots vs. QQ-plots
What is the difference between probability plots, PP-plots and QQ-plots when trying to analyse a fitted distribution to data?
- 631
32
votes
1
answer
14k
views
"Absolutely continuous random variable" vs. "Continuous random variable"?
In the book Limit Theorems of Probability Theory by Valentin V. Petrov, I saw a distinction between the definitions of a distribution being "continuous" and "absolutely continuous",...
- 569
36
votes
5
answers
59k
views
How to sample from a discrete distribution? [duplicate]
Assume I have a distribution governing the possible outcome from a single random variable X.
This is something like [0.1, 0.4, 0.2, 0.3] for X being a value of either 1, 2, 3, 4.
Is it possible to ...
- 503
13
votes
3
answers
27k
views
A normal divided by the $\sqrt{\chi^2(s)/s}$ gives you a t-distribution -- proof
let $Z \sim N(0,1)$ and $W \sim \chi^2(s)$.
If $Z$ and $W$ are independently distributed then the variable $Y = \frac{Z}{\sqrt{W/s}}$ follows a $t$ distribution with degrees of freedom $s$.
I am ...
- 1,431
53
votes
5
answers
37k
views
Probability distribution for different probabilities
If I wanted to get the probability of 9 successes in 16 trials with each trial having a probability of 0.6 I could use a binomial distribution. What could I use if each of the 16 trials has a ...
- 733
17
votes
2
answers
3k
views
Is joint normality a necessary condition for the sum of normal random variables to be normal?
In comments following this answer of mine to a related question, Users ssdecontrol and Glen_b asked
whether joint normality of $X$ and $Y$ is necessary for asserting the
normality of the sum $X+Y$? ...
- 43.4k
41
votes
15
answers
7k
views
The Monty Hall Problem - where does our intuition fail us?
From Wikipedia :
Suppose you're on a game show, and
you're given the choice of three
doors: Behind one door is a car;
behind the others, goats. You pick a
door, say No. 1, and the host, who
...
- 755
16
votes
6
answers
4k
views
Probability theory books for self-study
Are there any good books that explain important concepts of probability theory like probability distribution functions and cumulative distribution functions?
Please, avoid referring books like "...
7
votes
3
answers
1k
views
$X_i, X_j$ independent when $i≠j$, but $X_1, X_2, X_3$ dependent?
I've seen the statement:
It's possible that random variables $X_i, X_j$ are independent for $i≠j$, but $X_1, X_2, X_3$ are dependent.
I haven't been able to find examples of this though.
Any ...
- 3,319
74
votes
11
answers
11k
views
Is there any *mathematical* basis for the Bayesian vs frequentist debate?
It says on Wikipedia that:
the mathematics [of probability] is largely independent of any interpretation of probability.
Question: Then if we want to be mathematically correct, shouldn't we disallow ...
- 5,979
22
votes
3
answers
24k
views
Normalizing constant in Bayes theorem
I read that in Bayes rule, the denominator $\Pr(\textrm{data})$ of
$$\Pr(\text{parameters} \mid \text{data}) = \frac{\Pr(\textrm{data} \mid \textrm{parameters}) \Pr(\text{parameters})}{\Pr(\text{...
- 374
15
votes
1
answer
2k
views
Odds made simple
I am having some trouble in understanding odds, and I would like just a basic explanation for how to interpret them.
I have found various posts related to odds but most of them are more complex than ...
- 151
128
votes
9
answers
100k
views
Numerical example to understand Expectation-Maximization
I am trying to get a good grasp on the EM algorithm, to be able to implement and use it. I spent a full day reading the theory and a paper where EM is used to track an aircraft using the position ...
- 2,533
29
votes
4
answers
14k
views
How to derive the probabilistic interpretation of the AUC?
Why is the area under the ROC curve the probability that a classifier will rank a randomly chosen "positive" instance (from the retrieved predictions) higher than a randomly chosen "positive" one (...
- 291
11
votes
2
answers
15k
views
How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?
I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
- 229
4
votes
1
answer
2k
views
Intuition behind Weight of Evidence and Information Value formula
In credit scoring models, we use Weight of Evidence to create bins for continuous variables and Information value to filter out important variables.
\begin{align}
\text{WoE:} \qquad &\ln \frac{\...
67
votes
4
answers
20k
views
What is so cool about de Finetti's representation theorem?
From Theory of Statistics by Mark J. Schervish (page 12):
Although DeFinetti's representation theorem 1.49 is central to motivating parametric models, it is not actually used in their ...
- 14k
38
votes
4
answers
14k
views
What is the difference between the vertical bar and semi-colon notations?
What is the difference in meaning between the notation $P(z;d,w)$ and $P(z|d,w)$ which are commonly used in many books and papers?
- 4,307
37
votes
4
answers
10k
views
Intuition for Conditional Expectation of $\sigma$-algebra
Let $(\Omega,\mathscr{F},\mu)$ be a probability space, given a random variable $\xi:\Omega \to \mathbb{R}$ and a $\sigma$-algebra $\mathscr{G}\subseteq \mathscr{F}$ we can construct a new random ...
- 1,725
27
votes
4
answers
28k
views
"The total area underneath a probability density function is 1" - relative to what?
Conceptually I grasp the meaning of the phrase "the total area underneath a PDF is 1". It should mean that the chances of the outcome being in the total interval of possibilities is 100%.
But I ...
- 934
15
votes
4
answers
678
views
If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$
Question
If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$.
Attempt: Please check if the below is correct.
Let ...
- 283
9
votes
2
answers
11k
views
Transformation Chi-squared to Normal distribution
The relationship between the standard normal and the chi-squared distributions is well known. I was wondering though, is there a transformation that can lead from a $\chi^2 (1)$
back to a standard ...
- 19.2k
9
votes
2
answers
2k
views
What is $P(X_1>X_2 , X_1>X_3,... , X_1>X_n)$?
All $X$ are mutually independent and from normal distributions, each with its own mean and variance. If it's easier, $P(X_1 \geq X_i \forall i \in \{1, ..., n\})$ is fine although I suspect it's the ...
- 193