New answers tagged

2 votes

Least squares estimator for parameter p in binomial distribution

As has been pointed out in comments, least squares and maximum likelihood don't always coincide. However, for a Binomial distribution, they do. We have the least squares problem: $$\min_{\hat{p}} \...
  • 35.3k
1 vote
Accepted

Gambler's ruin with partial gains

Formulating the Markov chain: Although you didn't explicitly specify it, I will assume that the gambler instantly cashes in their tokens for a dollar as soon as they have $R$ tokens (i.e., they don't ...
  • 107k
2 votes
Accepted

Does an expectation for a Markov chain simplify like this?

You don't need the Markov property. For any two jointly defined random variables $X$ and $Y$, $$E_X[x]=E_{X,Y}[x]$$ The marginal distribution of $X$ is just the distribution of $X$. Now just take $X$ ...
1 vote

Does probability calibration descrease model prediction variance?

It could be that these items are always likely to be spam and all should have spam probability scores above $0.9$ or even $0.99$. We would hope for the calibration to correct this, and such a result ...
  • 43.3k
1 vote

How can I use the Central Limit Theorem to calculate the distribution of $\bar{X}$?

If $X_i$ are iid with mean $\mu_X$ and variance $\sigma^2_X$ then $\sum X_i$ has mean $n \mu_X$ and variance $n\sigma^2_X$ $\bar X =\frac1n\sum X_i$ has mean $\mu_X$ and variance $\frac1n\sigma^2_X$ ...
  • 34k
0 votes

How can I use the Central Limit Theorem to calculate the distribution of $\bar{X}$?

There's probably an easier way to prove this, simply based on linearity of expectation and variance, and the fact that Gaussians are defined by these two moments, however: CLT implies convergence by ...
  • 2,555
1 vote

Inversion and correlation of Binomial random variable

This approach (inverting the transformation, finding the Jacobian, etc.) is applicable when $A$ and $B$ are jointly continuous random variables, and $f_{A,B}$ is their joint density function. In this ...
0 votes

Need help on conditional probability problem

By the law of total probability, P(cough) = P(cough|covid)*P(covid) + P(cough|¬covid)*P(¬covid). It should be no trouble for you to continue from here.
  • 306
0 votes

Using machine learning to compare the probability of success of two different treatment strategies

What you're referring to is estimating an optimal treatment rule. Assuming you can satisfy certain causal assumptions, one approach for this is using a counterfactual framework. Denote the set of all ...
0 votes

Does large mutual information (between observations and parameter) imply the existence of a good estimator?

Yes. For instance, the error of an optimal estimator (i.e. maximum a posteriori estimator) may be upper bounded in terms of $H(X|Y)$. If we let the random variable $\hat{X}$ describing our estimator ...
  • 163
0 votes

Using machine learning to compare the probability of success of two different treatment strategies

Following my comments, I have made an example to show my suggested approach. First I simulate some data with a binary outcome, and two continues predictors and a binary variable for strategy: ...
  • 611
2 votes

What are $\rho$-, $\beta$-, and $\alpha$-mixing conditions?

There is an excellent survey on various notions of mixing. I can personally recommend:-) R.C. Bradley, "Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions"
  • 121
1 vote

Testing for statistical correlation using textual data

You can use an autoregressive model with a binary covariate indicating pre or post announcement. A simple AR(1) model can be written as follows: $$y_{t} = \alpha + \beta_{1}y_{t-1} + \beta_{2}X_{t} + \...
  • 611
1 vote

Testing for statistical correlation using textual data

Unfortunately, the assumptions required of simple statistical tests would fail for your premise. The most simple examples (a 2-group binomial test as in prop.test or fisher s exact test fisher.test in ...
1 vote

Probability of one horse finishing ahead of another

Let $p_i$, $i=1,2,\dots,n$ denote the probabilities that each horse wins given in the problem statement. A model that leads to simple calculations is to assume that some monotonically decreasing ...
  • 9,331
2 votes
Accepted

Show the Binomial distribution approaches a Normal distribution (using characteristic function)

Your idea is good. But it's easier to take logarithms and just look at what results. You will eventually want to expand the log characteristic function (aka the cumulant generating function) as a ...
  • 303k
1 vote

Discrete uniform converges weakly to continuous uniform

By the definition of expectation: \begin{align*} & E[f(X_n)] = \sum_{i = 0}^{n - 1}f(i/n)n^{-1} = \sum_{i = 0}^{n - 1}\int_{i/n}^{(i + 1)/n}f(i/n)dx, \\ & E[f(X)] = \int_0^1 f(x)dx = \sum_{i =...
  • 7,346
0 votes

Show the Binomial distribution approaches a Normal distribution (using characteristic function)

Apply the expension of exponential and logrithm. The CF of $(X_n - np) / \sqrt{np(1-p)}$ is $$ \begin{align} f(t) &= \left(pe^{it\frac{1}{\sqrt{np(1-p)}}} -p + 1 \right)^ne^{-it\frac{np}{\sqrt{np(...
  • 136
7 votes

Probability of one horse finishing ahead of another

This is similar to Christian Hennig's answer. If you make the strong assumption that the probabilities are in effect weights, and the first horse is sampled with probabilities proportional to the ...
  • 34k
1 vote
Accepted

MLE for categorical distribution

A categorical random variable is such that has $k$ possible values that are exclusive. For example, say that there are three possible ways you could pick to go to work: bike, car, and public ...
  • 125k
6 votes

Probability of one horse finishing ahead of another

The given information is not enough to compute these probabilities in general, because it may be that for example Horse C is of a kind that it either wins or gets frustrated and finishes last. But it ...
0 votes

Intuition behind conditioning Y on X in the front-door adjustment formula

To answer your question straight, $$P(y | \textit{do}(X = x)) = \sum_z P(z | x) \sum_{x'} P(y|x', z)P(x').$$ can't be rewritten as $$P(y | \textit{do}(X = x)) = \sum_z P(z | x) \sum_{x'} P(y|z)P(x')$$ ...
  • 349
1 vote
Accepted

Non-IID Uniform Distribution

\begin{align*} F_{W}(w) = P(W\le w) = 1- [(1-P(A< w) )(1-P(B< w ))] = 1- \left[ \left( 1 - \frac{w}{2}\right) \left( 1 - \frac{w-1}{2}\right) \right] = 1 - \left[ \left( \frac{2-u}{2}\right) \...
  • 136
2 votes
Accepted

Probably that an item was not selected in a sample

This "without replacement" sample has a hypergeometric distribution, but it may be more obvious if you say there are ${765 \choose 119}$ ways of choosing all non-blues (ignoring order) out ...
  • 34k
0 votes
Accepted

How to marginalise this?

The marginalization is just another form of applying the law of total probability: \begin{align} & p(a, b) \\ =& p(a)p(b|a) \tag{1} \\ =& p(a)\sum_c p(b, c|a) \tag{2} \\ =& p(a)\sum_c ...
  • 7,346
2 votes

How to think of a sub-sigma algebra as a collection of random variables?

$\newcommand{\classG}{\mathcal{G}}$ To facilitate discussion, let's first settle the probability space to be $(\Omega, \mathcal{H}, P)$, and let $\mathcal{G}$ be a sub-$\sigma$-algebra of $\mathcal{H}$...
  • 7,346
2 votes
Accepted

Proving estimator consistency

The estimator $\hat{\sigma}^2_N$ is consistent if it converges in probability to $\sigma^2$. To prove consistency it is sufficient to show that $E[(\hat{\sigma}^2_N - \sigma^2)^2]$ goes to $0$ as $N \...
-1 votes

Find conditional expectation given a discrete random variable whose range is N

This question is another clear case of applying identity: $E[f(X,Y)|Y=y]=E[f(X,y)|Y=y]$. $$E[X_N|N=n]=E[X_n|N=n]=E[X_n]=\mu _n$$. In the same way, for the variance we have: $$Var[X_N|N=n]=E[(X_N-\mu_N)...
1 vote
Accepted

Estimating the parameter in a mixed population

If I'm not mistaken, you have categorical outcomes, one of O1, O2, O3, O4, for each individual. Such data could be handled using multiple logistic regression, available as function ...
2 votes

Random forest probability meaning

There are many interpretations of probability. Skimming through them would be a good starting point to understand the concept better. The decision tree makes probabilistic predictions by calculating ...
  • 125k
0 votes

Definition and Interpretation of Likelihood for non-PhD's

The likelihood does give us what can often be equated with 'plausibility', but it is important to say that it is the relative plausibility according to the statistical model. And it is probably useful ...
  • 11.7k
0 votes

Definition and Interpretation of Likelihood for non-PhD's

Your interpretations 1 and 2 are both wrong. Bayesians knew all along you have to multiply the likelihood by a prior to get a posterior probability distribution. The problem is there is no ...
  • 56.2k
3 votes

How Do I Know If A Markov Chain Follows The Markov Property?

You can easily test this by doing multinomial regression. To fit the null hypothesis of a first order Markov chain you would include the previous state as a covariate in the model. You then estimate ...
  • 9,331
1 vote

Proof of inequality $P(X \geq \lambda) \leq \frac{\sigma^2}{\sigma^2 + \lambda^2}$

When $E[X=0]$ then the variance is $$\sigma^2 = \int_{-\infty}^{\infty} x^2 f(x) dx$$ We can consider some parts of this integral seperately. Consider the area 1 and area 2 in the image below. $$\...
4 votes

How Do I Know If A Markov Chain Follows The Markov Property?

An initial and simple test for this would be to see if the data show evidence of the weather being affected by the weather two days ago when you're already conditioning on the weather one day ago. To ...
  • 107k
3 votes
Accepted

Why choose a Dirichlet distribution in Latent Dirichlet Allocation (LDA)?

Two properties that make the Dirichlet distribution a suitable default choice are: Flexibility: it has full support on the parameter space which is the set possible values of the K-dimensional vector ...
  • 457
2 votes

Marginal distribution of normal random variable with a normal mean

You have \begin{align} X\mid M & \sim \operatorname N(M, \sigma^2) \\[6pt] M & \sim \operatorname N(\theta,s^2) \end{align} Consequently $$(X-M)\mid M\sim\operatorname N(0,\sigma^2).$$ Observe ...
5 votes
Accepted

Proof of inequality $P(X \geq \lambda) \leq \frac{\sigma^2}{\sigma^2 + \lambda^2}$

This inequality is called Cantelli's inequality (Probability and Measure by Patrick Billingsley, Exercise 5.5). To prove it, note that for $x > 0$, $\{X \geq \lambda\} \subseteq \{(X + x)^2 \geq (\...
  • 7,346
0 votes

Poisson Distribution - Conditional Probability Question

Conditional probability is a measure of a set (N==1) occupies upon the (N>=1) partition (rather than the whole unity, as it is for unconditional probability). Understanding this fact makes ...
  • 306
4 votes

Poisson Distribution - Conditional Probability Question

The question has asked, given $N\sim\textrm{Poi}(\lambda), $ to find the value of $\lambda$ such that $\mathbb P[N=1|N\geq 1]= 0.4.$ The solution is nothing but applying the definition of conditional ...
  • 3,739
3 votes
Accepted

In statistics how does one find the mean of a function w.r.t the uniform probability measure?

Note: I'm going to use the notation $\mathscr{X}$ for the set that you call $X$ and I'm going to define a random vector $\mathbf{X}$ as a uniform random vector on that set. It is better to avoid ...
  • 107k
1 vote
Accepted

Finding the distribution of $X$ conditional on the event $\{X > \psi\}$

$ f(x; \theta, \lambda) = \frac{\lambda \theta^{\lambda}}{x^{\lambda+1}} $ So, $$ F(x) = \int_\theta^x f(t)\,dt = \lambda \theta^\lambda \left[\frac{-1}{\lambda}t^{-\lambda}\right]_\theta^x = \theta^\...
  • 1,460
11 votes

CDF of max of $n$ cauchy variates

You ask about the distribution of the maximum order statistic. There are many similar questions, for other distribution, so follow the outlines of for instance Distribution of extreme values, case of ...
2 votes

Equivalence of Tightness of Seqeuence of CDFs

Assuming Durrett's definition, choose $M$ so that for all $\liminf \mu_n([-M,M]) \geq \frac{\epsilon}{2}$ for a fixed $\epsilon > 0$. Then eventually, $a_n = \inf_{m \geq n} \mu_m([-M,M]) \geq 1-\...
  • 462
3 votes
Accepted

What is the reasoning behind the string of equality $P(x_{(n)} \le t) = P(X_i \le t, \ \text{for each $i$}) = \{\Phi(t - \theta)\}^n$?

Preliminary note: There is inconsistency in the use of lower/upper case notation for random variables in the question. I have chosen to correct this in the answer so my capitalisation is different to ...
  • 107k
1 vote

What is the reasoning behind the string of equality $P(x_{(n)} \le t) = P(X_i \le t, \ \text{for each $i$}) = \{\Phi(t - \theta)\}^n$?

The more general relation is $$ \mathbb P[X_{i;n}\leq t]=\sum_{k=i}^n{{n}\choose {k}}F^k(t)[1-F(t)]^{n-k}$$ which has been derived in this CV post. Breaking it down for $i=n,$\begin{align}\mathbb P[X_{...
  • 3,739
0 votes

Likelihood function for Type I censoring

$\bullet$ Conditionally when $\delta = 0,~ T= C_r~\textrm{a.s.}$ and hence $\mathbb P[T = C_r|\delta = 0] = 1. $ $\bullet$ Since, conditionally $\delta = 1\implies X\leq C_r$ and $T = t \leq C_r$ by ...
  • 3,739
2 votes

Suppose $X \mid Y$ and $Y$ are normally distributed. Does it follow that $Y \mid X$ is normally distributed?

Here is another counterexample which gives closed-form distributions of $X, Y, X|Y = y$ and $Y|X = x$. Let $Y, Z \text{ i.i.d. } \sim N(0, 1)$, and define $X = \frac{Z}{Y}$. Then for $y \neq 0$ (the ...
  • 7,346
6 votes

Suppose $X \mid Y$ and $Y$ are normally distributed. Does it follow that $Y \mid X$ is normally distributed?

Here I will augment the excellent answer by whuber by showing the mathematical form of your general model and the sufficient conditions that imply a normal distribution for $Y|X$. Consider the ...
  • 107k
7 votes

Suppose $X \mid Y$ and $Y$ are normally distributed. Does it follow that $Y \mid X$ is normally distributed?

whuber shows, by means of a counterexample, that the product of a Gaussian $X|Y$ times a Gaussian r.v. $Y$ does not necessarily lead to a joint Gaussian distribution, which in this case certainly ...
  • 6,509

Top 50 recent answers are included