# Questions tagged [probability]

A probability provides a quantitative description of the likely occurrence of a particular event.

1,407 questions
Filter by
Sorted by
Tagged with
116k views

### Can a probability distribution value exceeding 1 be OK?

On the Wikipedia page about naive Bayes classifiers, there is this line: $p(\mathrm{height}|\mathrm{male}) = 1.5789$ (A probability distribution over 1 is OK. It is the area under the bell curve ...
• 4,689
7k views

### Is accuracy an improper scoring rule in a binary classification setting?

I have recently been learning about proper scoring rules for probabilistic classifiers. Several threads on this website have made a point of emphasizing that accuracy is an improper scoring rule and ...
• 261
104k views

### Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

It seems that through various related questions here, there is consensus that the "95%" part of what we call a "95% confidence interval" refers to the fact that if we were to exactly replicate our ...
32k views

### Generic sum of Gamma random variables

I have read that the sum of Gamma random variables with the same scale parameter is another Gamma random variable. I've also seen the paper by Moschopoulos describing a method for the summation of a ...
• 1,117
466k views

### What is the difference between "likelihood" and "probability"?

The wikipedia page claims that likelihood and probability are distinct concepts. In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a ...
• 7,291
3k views

### I've heard that ratios or inverses of random variables often are problematic, in not having expectations. Why is that?

The title is the question. I am told that ratios and inverses of random variables often are problematic. What is meant is that expectation often do not exist. Is there a simple, general explication of ...
18k views

### Functions of Independent Random Variables

Is the claim that functions of independent random variables are themselves independent, true? I have seen that result often used implicitly in some proofs, for example in the proof of independence ...
• 19.2k
13k views

### $P[X=x]=0$ when $X$ is a continuous variable

I know that for any continuous variable $P[X=x]=0$. But I can't visualize that if $P[X=x]=0$, there is an infinite number of possible $x$'s. And also why do their probabilities get infinitely small ?
• 1,527
65k views

### How to generate correlated random numbers (given means, variances and degree of correlation)?

I'm sorry if this seems a bit too basic, but I guess I'm just looking to confirm understanding here. I get the sense I'd have to do this in two steps, and I've started trying to grok correlation ...
83k views

### How does linear regression use the normal distribution?

In linear regression, each predicted value is assumed to have been picked from a normal distribution of possible values. See below. But why is each predicted value assumed to have come from a normal ...
• 13.2k
11k views

### What is the most surprising characterization of the Gaussian (normal) distribution?

A standardized Gaussian distribution on $\mathbb{R}$ can be defined by giving explicitly its density: $$\frac{1}{\sqrt{2\pi}}e^{-x^2/2}$$ or its characteristic function. As recalled in this ...
12k views

### Time taken to hit a pattern of heads and tails in a series of coin-tosses

Inspired by Peter Donnelly's talk at TED, in which he discusses how long it would take for a certain pattern to appear in a series of coin tosses, I created the following script in R. Given two ...
• 403
29k views

### How to tell the probability of failure if there were no failures?

I was wondering if there is a way to tell the probability of something failing (a product) if we have 100,000 products in the field for 1 year and with no failures? What is the probability that one of ...
• 541
10k views

### Can anyone clarify the concept of a "sum of random variables"

In my probability class the terms "sums of random variables" is constantly used. However, I'm stuck on what exactly that means? Are we talking about the sum of a bunch of realizations from a random ...
• 505
2k views

### How exactly do Bayesians define (or interpret?) probability?

Part of a series of trying to understand Bayesian vs frequentist: 1 2 3 4 5 6 7 I think I get the difference of how Bayesians and frequentists approach choosing between hypotheses, but I'm not quite ...
• 2,280
21k views

### Combining probabilities/information from different sources

Lets say I have three independent sources and each of them make predictions for the weather tomorrow. The first one says that the probability of rain tomorrow is 0, then the second one says that the ...
• 591
3k views

### Wikipedia entry on likelihood seems ambiguous

I have a simple question regarding "conditional probability" and "Likelihood". (I have already surveyed this question here but to no avail.) It starts from the Wikipedia page on likelihood. They say ...
• 1,567
39k views

### Bayesian vs frequentist Interpretations of Probability

Can someone give a good rundown of the differences between the Bayesian and the frequentist approach to probability? From what I understand: The frequentists view is that the data is a repeatable ...
• 1,435
17k views

### Why Normalizing Factor is Required in Bayes Theorem?

Bayes theorem goes $$P(\textrm{model}|\textrm{data}) = \frac{P(\textrm{model}) \times P(\textrm{data}|\textrm{model})}{P(\textrm{data})}$$ This is all fine. But, I've read somewhere: Basically, ...
15k views

• 2,598
3k views

### Risk of extinction of Schrödinger's cats

I am interested in how uncertainty can be accounted for when considering the risk of extinction of a species. Forgive me for extending a rather tired thought experiment, but at least it's familiar ...
49k views

### Why do we need sigma-algebras to define probability spaces?

We have a random experiment with different outcomes forming the sample space $\Omega,$ on which we look with interest at certain patterns, called events $\mathscr{F}.$ Sigma-algebras (or sigma-fields) ...
• 25.2k
69k views

### PP-plots vs. QQ-plots

What is the difference between probability plots, PP-plots and QQ-plots when trying to analyse a fitted distribution to data?
• 631
14k views

### "Absolutely continuous random variable" vs. "Continuous random variable"?

In the book Limit Theorems of Probability Theory by Valentin V. Petrov, I saw a distinction between the definitions of a distribution being "continuous" and "absolutely continuous",...
• 569
59k views

### How to sample from a discrete distribution? [duplicate]

Assume I have a distribution governing the possible outcome from a single random variable X. This is something like [0.1, 0.4, 0.2, 0.3] for X being a value of either 1, 2, 3, 4. Is it possible to ...
• 503
27k views

### A normal divided by the $\sqrt{\chi^2(s)/s}$ gives you a t-distribution -- proof

let $Z \sim N(0,1)$ and $W \sim \chi^2(s)$. If $Z$ and $W$ are independently distributed then the variable $Y = \frac{Z}{\sqrt{W/s}}$ follows a $t$ distribution with degrees of freedom $s$. I am ...
• 1,431
37k views

### Probability distribution for different probabilities

If I wanted to get the probability of 9 successes in 16 trials with each trial having a probability of 0.6 I could use a binomial distribution. What could I use if each of the 16 trials has a ...
• 733
3k views

### Is joint normality a necessary condition for the sum of normal random variables to be normal?

In comments following this answer of mine to a related question, Users ssdecontrol and Glen_b asked whether joint normality of $X$ and $Y$ is necessary for asserting the normality of the sum $X+Y$? ...
• 43.4k
7k views

### The Monty Hall Problem - where does our intuition fail us?

From Wikipedia : Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who ...
4k views

### Probability theory books for self-study

Are there any good books that explain important concepts of probability theory like probability distribution functions and cumulative distribution functions? Please, avoid referring books like "...
1k views

### $X_i, X_j$ independent when $i≠j$, but $X_1, X_2, X_3$ dependent?

I've seen the statement: It's possible that random variables $X_i, X_j$ are independent for $i≠j$, but $X_1, X_2, X_3$ are dependent. I haven't been able to find examples of this though. Any ...
• 3,319
11k views

### Is there any *mathematical* basis for the Bayesian vs frequentist debate?

It says on Wikipedia that: the mathematics [of probability] is largely independent of any interpretation of probability. Question: Then if we want to be mathematically correct, shouldn't we disallow ...
• 5,979
24k views

### Normalizing constant in Bayes theorem

I read that in Bayes rule, the denominator $\Pr(\textrm{data})$ of \Pr(\text{parameters} \mid \text{data}) = \frac{\Pr(\textrm{data} \mid \textrm{parameters}) \Pr(\text{parameters})}{\Pr(\text{...
• 374
2k views

I am having some trouble in understanding odds, and I would like just a basic explanation for how to interpret them. I have found various posts related to odds but most of them are more complex than ...
• 151
100k views

### Numerical example to understand Expectation-Maximization

I am trying to get a good grasp on the EM algorithm, to be able to implement and use it. I spent a full day reading the theory and a paper where EM is used to track an aircraft using the position ...
• 2,533
14k views

### How to derive the probabilistic interpretation of the AUC?

Why is the area under the ROC curve the probability that a classifier will rank a randomly chosen "positive" instance (from the retrieved predictions) higher than a randomly chosen "positive" one (...
• 291
15k views

### How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
• 229
2k views

### Intuition behind Weight of Evidence and Information Value formula

In credit scoring models, we use Weight of Evidence to create bins for continuous variables and Information value to filter out important variables. \begin{align} \text{WoE:} \qquad &\ln \frac{\...
20k views

### What is so cool about de Finetti's representation theorem?

From Theory of Statistics by Mark J. Schervish (page 12): Although DeFinetti's representation theorem 1.49 is central to motivating parametric models, it is not actually used in their ...
• 14k
14k views

### What is the difference between the vertical bar and semi-colon notations?

What is the difference in meaning between the notation $P(z;d,w)$ and $P(z|d,w)$ which are commonly used in many books and papers?
• 4,307
10k views

### Intuition for Conditional Expectation of $\sigma$-algebra

Let $(\Omega,\mathscr{F},\mu)$ be a probability space, given a random variable $\xi:\Omega \to \mathbb{R}$ and a $\sigma$-algebra $\mathscr{G}\subseteq \mathscr{F}$ we can construct a new random ...
• 1,725
28k views

### "The total area underneath a probability density function is 1" - relative to what?

Conceptually I grasp the meaning of the phrase "the total area underneath a PDF is 1". It should mean that the chances of the outcome being in the total interval of possibilities is 100%. But I ...
• 934
678 views

### If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$

Question If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$. Attempt: Please check if the below is correct. Let ...
• 283
The relationship between the standard normal and the chi-squared distributions is well known. I was wondering though, is there a transformation that can lead from a $\chi^2 (1)$ back to a standard ...
### What is $P(X_1>X_2 , X_1>X_3,... , X_1>X_n)$?
All $X$ are mutually independent and from normal distributions, each with its own mean and variance. If it's easier, $P(X_1 \geq X_i \forall i \in \{1, ..., n\})$ is fine although I suspect it's the ...