New answers tagged probability
2
votes
Least squares estimator for parameter p in binomial distribution
As has been pointed out in comments, least squares and maximum likelihood don't always coincide. However, for a Binomial distribution, they do. We have the least squares problem:
$$\min_{\hat{p}} \...
- 35.3k
1
vote
Accepted
Gambler's ruin with partial gains
Formulating the Markov chain: Although you didn't explicitly specify it, I will assume that the gambler instantly cashes in their tokens for a dollar as soon as they have $R$ tokens (i.e., they don't ...
- 107k
2
votes
Accepted
Does an expectation for a Markov chain simplify like this?
You don't need the Markov property. For any two jointly defined random variables $X$ and $Y$,
$$E_X[x]=E_{X,Y}[x]$$
The marginal distribution of $X$ is just the distribution of $X$.
Now just take $X$ ...
- 27.7k
1
vote
Does probability calibration descrease model prediction variance?
It could be that these items are always likely to be spam and all should have spam probability scores above $0.9$ or even $0.99$. We would hope for the calibration to correct this, and such a result ...
- 43.3k
1
vote
How can I use the Central Limit Theorem to calculate the distribution of $\bar{X}$?
If $X_i$ are iid with mean $\mu_X$ and variance $\sigma^2_X$ then
$\sum X_i$ has mean $n \mu_X$ and variance $n\sigma^2_X$
$\bar X =\frac1n\sum X_i$ has mean $\mu_X$ and variance $\frac1n\sigma^2_X$
...
- 34k
0
votes
How can I use the Central Limit Theorem to calculate the distribution of $\bar{X}$?
There's probably an easier way to prove this, simply based on linearity of expectation and variance, and the fact that Gaussians are defined by these two moments, however:
CLT implies convergence by ...
- 2,555
1
vote
Inversion and correlation of Binomial random variable
This approach (inverting the transformation, finding the Jacobian, etc.) is applicable when $A$ and $B$ are jointly continuous random variables, and $f_{A,B}$ is their joint density function. In this ...
- 565
0
votes
Need help on conditional probability problem
By the law of total probability, P(cough) = P(cough|covid)*P(covid) + P(cough|¬covid)*P(¬covid). It should be no trouble for you to continue from here.
- 306
0
votes
Using machine learning to compare the probability of success of two different treatment strategies
What you're referring to is estimating an optimal treatment rule. Assuming you can satisfy certain causal assumptions, one approach for this is using a counterfactual framework. Denote the set of all ...
- 79
0
votes
Does large mutual information (between observations and parameter) imply the existence of a good estimator?
Yes. For instance, the error of an optimal estimator (i.e. maximum a posteriori estimator) may be upper bounded in terms of $H(X|Y)$. If we let the random variable $\hat{X}$ describing our estimator ...
- 163
0
votes
Using machine learning to compare the probability of success of two different treatment strategies
Following my comments, I have made an example to show my suggested approach. First I simulate some data with a binary outcome, and two continues predictors and a binary variable for strategy:
...
- 611
2
votes
What are $\rho$-, $\beta$-, and $\alpha$-mixing conditions?
There is an excellent survey on various notions of mixing. I can personally recommend:-)
R.C. Bradley, "Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions"
- 121
1
vote
Testing for statistical correlation using textual data
You can use an autoregressive model with a binary covariate indicating pre or post announcement. A simple AR(1) model can be written as follows:
$$y_{t} = \alpha + \beta_{1}y_{t-1} + \beta_{2}X_{t} + \...
- 611
1
vote
Testing for statistical correlation using textual data
Unfortunately, the assumptions required of simple statistical tests would fail for your premise.
The most simple examples (a 2-group binomial test as in prop.test or fisher
s exact test fisher.test in ...
- 11
1
vote
Probability of one horse finishing ahead of another
Let $p_i$, $i=1,2,\dots,n$ denote the probabilities that each horse wins given in the problem statement. A model that leads to simple calculations is to assume that some monotonically decreasing ...
- 9,331
2
votes
Accepted
Show the Binomial distribution approaches a Normal distribution (using characteristic function)
Your idea is good. But it's easier to take logarithms and just look at what results. You will eventually want to expand the log characteristic function (aka the cumulant generating function) as a ...
- 303k
1
vote
Discrete uniform converges weakly to continuous uniform
By the definition of expectation:
\begin{align*}
& E[f(X_n)] = \sum_{i = 0}^{n - 1}f(i/n)n^{-1} =
\sum_{i = 0}^{n - 1}\int_{i/n}^{(i + 1)/n}f(i/n)dx, \\
& E[f(X)] = \int_0^1 f(x)dx = \sum_{i =...
- 7,346
0
votes
Show the Binomial distribution approaches a Normal distribution (using characteristic function)
Apply the expension of exponential and logrithm. The CF of $(X_n - np) / \sqrt{np(1-p)}$ is
$$
\begin{align}
f(t) &= \left(pe^{it\frac{1}{\sqrt{np(1-p)}}} -p + 1 \right)^ne^{-it\frac{np}{\sqrt{np(...
- 136
7
votes
Probability of one horse finishing ahead of another
This is similar to Christian Hennig's answer.
If you make the strong assumption that the probabilities are in effect weights, and the first horse is sampled with probabilities proportional to the ...
- 34k
1
vote
Accepted
MLE for categorical distribution
A categorical random variable is such that has $k$ possible values that are exclusive. For example, say that there are three possible ways you could pick to go to work: bike, car, and public ...
Tim♦
- 125k
6
votes
Probability of one horse finishing ahead of another
The given information is not enough to compute these probabilities in general, because it may be that for example Horse C is of a kind that it either wins or gets frustrated and finishes last. But it ...
- 16.1k
0
votes
Intuition behind conditioning Y on X in the front-door adjustment formula
To answer your question straight,
$$P(y | \textit{do}(X = x)) = \sum_z P(z | x) \sum_{x'} P(y|x', z)P(x').$$
can't be rewritten as
$$P(y | \textit{do}(X = x)) = \sum_z P(z | x) \sum_{x'} P(y|z)P(x')$$
...
- 349
1
vote
Accepted
Non-IID Uniform Distribution
\begin{align*}
F_{W}(w) = P(W\le w) = 1- [(1-P(A< w) )(1-P(B< w ))]
= 1- \left[ \left( 1 - \frac{w}{2}\right) \left( 1 - \frac{w-1}{2}\right) \right]
= 1 - \left[ \left( \frac{2-u}{2}\right) \...
- 136
2
votes
Accepted
Probably that an item was not selected in a sample
This "without replacement" sample has a hypergeometric distribution, but it may be more obvious if you say there are ${765 \choose 119}$ ways of choosing all non-blues (ignoring order) out ...
- 34k
0
votes
Accepted
How to marginalise this?
The marginalization is just another form of applying the law of total probability:
\begin{align}
& p(a, b) \\
=& p(a)p(b|a) \tag{1} \\
=& p(a)\sum_c p(b, c|a) \tag{2} \\
=& p(a)\sum_c ...
- 7,346
2
votes
How to think of a sub-sigma algebra as a collection of random variables?
$\newcommand{\classG}{\mathcal{G}}$
To facilitate discussion, let's first settle the probability space to be $(\Omega, \mathcal{H}, P)$, and let $\mathcal{G}$ be a sub-$\sigma$-algebra of $\mathcal{H}$...
- 7,346
2
votes
Accepted
Proving estimator consistency
The estimator $\hat{\sigma}^2_N$ is consistent if it converges in probability to $\sigma^2$. To prove consistency it is sufficient to show that $E[(\hat{\sigma}^2_N - \sigma^2)^2]$ goes to $0$ as $N \...
-1
votes
Find conditional expectation given a discrete random variable whose range is N
This question is another clear case of applying identity: $E[f(X,Y)|Y=y]=E[f(X,y)|Y=y]$.
$$E[X_N|N=n]=E[X_n|N=n]=E[X_n]=\mu _n$$.
In the same way, for the variance we have:
$$Var[X_N|N=n]=E[(X_N-\mu_N)...
1
vote
Accepted
Estimating the parameter in a mixed population
If I'm not mistaken, you have categorical outcomes, one of O1, O2, O3, O4, for each individual. Such data could be handled using multiple logistic regression, available as function ...
- 16.1k
2
votes
Random forest probability meaning
There are many interpretations of probability. Skimming through them would be a good starting point to understand the concept better.
The decision tree makes probabilistic predictions by calculating ...
Tim♦
- 125k
0
votes
Definition and Interpretation of Likelihood for non-PhD's
The likelihood does give us what can often be equated with 'plausibility', but it is important to say that it is the relative plausibility according to the statistical model. And it is probably useful ...
- 11.7k
0
votes
Definition and Interpretation of Likelihood for non-PhD's
Your interpretations 1 and 2 are both wrong.
Bayesians knew all along you have to multiply the likelihood by a prior to get a posterior probability distribution. The problem is there is no ...
- 56.2k
3
votes
How Do I Know If A Markov Chain Follows The Markov Property?
You can easily test this by doing multinomial regression.
To fit the null hypothesis of a first order Markov chain you would include the previous state as a covariate in the model. You then estimate ...
- 9,331
1
vote
Proof of inequality $P(X \geq \lambda) \leq \frac{\sigma^2}{\sigma^2 + \lambda^2}$
When $E[X=0]$ then the variance is
$$\sigma^2 = \int_{-\infty}^{\infty} x^2 f(x) dx$$
We can consider some parts of this integral seperately. Consider the area 1 and area 2 in the image below.
$$\...
- 58.4k
4
votes
How Do I Know If A Markov Chain Follows The Markov Property?
An initial and simple test for this would be to see if the data show evidence of the weather being affected by the weather two days ago when you're already conditioning on the weather one day ago. To ...
- 107k
3
votes
Accepted
Why choose a Dirichlet distribution in Latent Dirichlet Allocation (LDA)?
Two properties that make the Dirichlet distribution a suitable default choice are:
Flexibility: it has full support on the parameter space which is the set possible values of the K-dimensional vector ...
- 457
2
votes
Marginal distribution of normal random variable with a normal mean
You have
\begin{align}
X\mid M & \sim \operatorname N(M, \sigma^2) \\[6pt]
M & \sim \operatorname N(\theta,s^2)
\end{align}
Consequently
$$(X-M)\mid M\sim\operatorname N(0,\sigma^2).$$
Observe ...
- 7,484
5
votes
Accepted
Proof of inequality $P(X \geq \lambda) \leq \frac{\sigma^2}{\sigma^2 + \lambda^2}$
This inequality is called Cantelli's inequality (Probability and Measure by Patrick Billingsley, Exercise 5.5). To prove it, note that for $x > 0$, $\{X \geq \lambda\} \subseteq \{(X + x)^2 \geq (\...
- 7,346
0
votes
Poisson Distribution - Conditional Probability Question
Conditional probability is a measure of a set (N==1) occupies upon the (N>=1) partition (rather than the whole unity, as it is for unconditional probability). Understanding this fact makes ...
- 306
4
votes
Poisson Distribution - Conditional Probability Question
The question has asked, given $N\sim\textrm{Poi}(\lambda), $ to find the value of $\lambda$ such that $\mathbb P[N=1|N\geq 1]= 0.4.$
The solution is nothing but applying the definition of conditional ...
- 3,739
3
votes
Accepted
In statistics how does one find the mean of a function w.r.t the uniform probability measure?
Note: I'm going to use the notation $\mathscr{X}$ for the set that you call $X$ and I'm going to define a random vector $\mathbf{X}$ as a uniform random vector on that set. It is better to avoid ...
- 107k
1
vote
Accepted
Finding the distribution of $X$ conditional on the event $\{X > \psi\}$
$
f(x; \theta, \lambda) = \frac{\lambda \theta^{\lambda}}{x^{\lambda+1}}
$
So,
$$
F(x) = \int_\theta^x f(t)\,dt = \lambda \theta^\lambda \left[\frac{-1}{\lambda}t^{-\lambda}\right]_\theta^x = \theta^\...
- 1,460
11
votes
CDF of max of $n$ cauchy variates
You ask about the distribution of the maximum order statistic. There are many similar questions, for other distribution, so follow the outlines of for instance Distribution of extreme values, case of ...
- 70k
2
votes
Equivalence of Tightness of Seqeuence of CDFs
Assuming Durrett's definition, choose $M$ so that for all $\liminf \mu_n([-M,M]) \geq \frac{\epsilon}{2}$ for a fixed $\epsilon > 0$.
Then eventually, $a_n = \inf_{m \geq n} \mu_m([-M,M]) \geq 1-\...
- 462
3
votes
Accepted
What is the reasoning behind the string of equality $P(x_{(n)} \le t) = P(X_i \le t, \ \text{for each $i$}) = \{\Phi(t - \theta)\}^n$?
Preliminary note: There is inconsistency in the use of lower/upper case notation for random variables in the question. I have chosen to correct this in the answer so my capitalisation is different to ...
- 107k
1
vote
What is the reasoning behind the string of equality $P(x_{(n)} \le t) = P(X_i \le t, \ \text{for each $i$}) = \{\Phi(t - \theta)\}^n$?
The more general relation is
$$ \mathbb P[X_{i;n}\leq t]=\sum_{k=i}^n{{n}\choose {k}}F^k(t)[1-F(t)]^{n-k}$$
which has been derived in this CV post.
Breaking it down for $i=n,$\begin{align}\mathbb P[X_{...
- 3,739
0
votes
Likelihood function for Type I censoring
$\bullet$ Conditionally when $\delta = 0,~ T= C_r~\textrm{a.s.}$ and hence $\mathbb P[T = C_r|\delta = 0] = 1. $
$\bullet$ Since, conditionally $\delta = 1\implies X\leq C_r$ and $T = t \leq C_r$ by ...
- 3,739
2
votes
Suppose $X \mid Y$ and $Y$ are normally distributed. Does it follow that $Y \mid X$ is normally distributed?
Here is another counterexample which gives closed-form distributions of $X, Y, X|Y = y$ and $Y|X = x$.
Let $Y, Z \text{ i.i.d. } \sim N(0, 1)$, and define $X = \frac{Z}{Y}$. Then for $y \neq 0$ (the ...
- 7,346
6
votes
Suppose $X \mid Y$ and $Y$ are normally distributed. Does it follow that $Y \mid X$ is normally distributed?
Here I will augment the excellent answer by whuber by showing the mathematical form of your general model and the sufficient conditions that imply a normal distribution for $Y|X$. Consider the ...
- 107k
7
votes
Suppose $X \mid Y$ and $Y$ are normally distributed. Does it follow that $Y \mid X$ is normally distributed?
whuber shows, by means of a counterexample, that the product of a Gaussian $X|Y$ times a Gaussian r.v. $Y$ does not necessarily lead to a joint Gaussian distribution, which in this case certainly ...
- 6,509
Top 50 recent answers are included