7

The image below is how I look at confidence intervals. It is an adaptation from an image in the answer to the question 'The basic logic of constructing a confidence interval', which is itself an adaptation of "The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial C. J. Clopper and E. S. Pearson Biometrika Vol. 26, No. 4 (Dec., ...


6

You can find some useful discussion of the "rule of three", including derivations and simulation analysis, in Javonovic and Levy (1997). With modern computing technology there is really no reason to use a "rule of thumb" like this instead of a good confidence interval formula what respects the support of the parameter of interest. For ...


6

Making the substitution $x = \frac{n}{2}-1$, you essentially want to control $$1 - \frac{\Gamma(x+1)}{\Gamma(x+\frac{1}{2}) \sqrt{x + \frac{1}{2}}}$$ as $x \to \infty$. Gautschi's inequality (applied with $s=\frac{1}{2}$) implies $$ 1 - \sqrt{\frac{x+1}{x+\frac{1}{2}}} <1 - \frac{\Gamma(x+1)}{\Gamma(x+\frac{1}{2}) \sqrt{x + \frac{1}{2}}} < 1 - \sqrt{\...


6

The default approach for analyzing expressions involving Gamma functions is Stirling's asymptotic expansion $$\log \Gamma(z) = \frac{1}{2}\log(2\pi) + \left(z - \frac{1}{2}\right)\log(z) - z + \frac{1}{12z} - \frac{1}{360z^3} + \cdots$$ (and usually you don't even need that final term). This gives us some intuition about how $\Gamma$ behaves and a basis for ...


5

This is a standard statistical inference problem involving the classical occupancy distribution (see e.g., O'Neill 2019). Since $R$ is the number of repeated balls, the number of distinct balls selected in the sample is given by: $$K = N-R \ \sim \ \text{Occ}(N, M).$$ The probability mass function for this random variable is: $$p(K=k|N,M) = \frac{(N)_k \...


5

If you get no successes in (say) $n = 100$ trials, then you might use a Jeffreys Confidence interval for the true binomial $p.$ This style of CI has very good frequentist properties, but its motivation is in terms of a Bayesian argument using the non-informative prior distribution $\mathsf{Beta}(.5,.5).$ For $n = 100$ it gives the 95% CI shown in the R code ...


4

Note, for the below proof to work you need to assume that the function is $g$ monotonic. (and also note that for non-monotonic functions there might not be always proof possible) Proof using chain rule Let's consider for simplicity the likelihood function as a function of a single variable: $$\mathcal{L}(\theta \vert x_1,x_2, \dots, x_n) = h(\theta)$$ If ...


4

The independence of $X$ and $Y$ makes this problem straightforward. To make the notation easier, assume $\mu=0$. Then $Cov (X,Y)=S_{XY}=E[(X-\mu_X)(Y-\mu_Y)]=E[XY]$, and the estimator $\hat S_{XY} = \frac{1}{n}\sum_{i=1}^n x_iy_i$ has expectation zero, so $Var (\hat S_{XY}) = E[\hat S_{XY}^2]$. \begin{align} Var(\hat S_{XY}) &= E[\hat S_{XY}^2] \\ &...


3

Your explanation for completeness is correct. For sufficiency, I think you need to be a little more careful with the probability mass function for $X$. What you wrote down, $f_\theta(x)$, is the p.m.f of $X^2$, not $X$ itself. It should be simple enough to show that, $$f_\theta(x) = \left(\frac{2}{3}\right)^{\frac{x(x+1)}{2}}\left(\frac{1}{3}\right)^{\frac{x(...


2

I think your likelihood expression has reversed $x=R$ and $m=M$ in $S_2(x,m)$ but no matter - this is a constant with respect to $N$ and so can be ignored. What you want is the integer $N$ which maximises $\frac{N!}{N^M \; (N-R)!}$. So you want the largest $N$ where $\frac{N!}{N^M \; (N-R)!} \ge \frac{(N-1)!}{(N-1)^M \; (N-1-R)!} $, i.e. where $N\left(\...


2

Try an algebraic example. The OLS estimator is: $$ \hat{\beta}_{OLS} = (X^TX)^{-1}X^Ty $$ Now scale $X$ and $y$ by a scalar $\textrm{a}$: $$ \hat{\beta}^{scaled}_{OLS} = ((aX)^T(aX))^{-1}(aX)^T(ay) \\ =(X^TX a^2)^{-1}X^Tya^2 \\ =(X^TX)^{-1}X^Ty \underbrace{\frac{a^2}{a^2}}_{=1} = \hat{\beta}_{OLS} $$ Note that if only $X$ is scaled, then the estimator is ...


1

With continuous distributions you are right to say that all samples would have probability zero, yet some will have larger density that others. The logic is the same as with discrete distributions, you choose the values of the parameters which would give the observed sample maximum density. Why does it work so well? Not always does, in fact optimality ...


1

OLS simply works with whatever units you provide in the data. The contribution of predictor $i$ to the linear predictor is $\beta_i x_i$. Any changes in the scale in which $x_i$ is expressed will just result in a proportional change in the value returned for $\beta_i$. For example, say one of your predictors is a distance, measured in miles, and the outcome ...


1

Your response variable is binary - fitting a linear regression model, lm(), to a binary variable is not the best thing you can do. As you can see from your own response, a linear regression model can produce "probabilities" whose values are non-sensical (e.g., > 1). By definition, probabilities can only take values between 0 and 1. Why not use ...


1

Denote the lowered points of the blood pressure by the two drugs as $X_1$ and $X_2$ respectively, where both $X_1$ and $X_2$ are normal distribution with sample means $\bar{X}_1=11$ and $\bar{X}_2=12$, and STD $\sigma_1=6$ and $\sigma_2=8$ respectively. Obviously, the difference between $\bar X_1$ and $\bar X_2$ is also normal distribution with variance ...


1

Suppose the age when children learn to read is distributed normally, with mean $m$ and standard deviation $s$. Then the fraction of children who can read at age $a$ is $\Phi((a-m)/s)$. (Here, $\Phi$ is the cumulative distribution function for the standard normal, and we'll also want its inverse $\Phi^{-1}$ which calculates quantiles or $z$-scores.) So for ...


1

In addition to @abstrusiosity's analytic solution, you can apply the bootstrap here. This has the advantage of working even when the two samples aren't independent (the true covariance isn't $0$) library(tidyverse) # Simulate data m = 0 s = 2 n = 100 X = rnorm(n, m, s) Y = rnorm(n, m, s) (expected_se = (s**2) / sqrt(n)) # @abstrusiosity's solution # [1] 0....


1

That's an interesting one :-) Now if I understood your question right, then the trick is to think in terms of functions and not focus just on COV. Though, the following things are initially important: It's normal distributed. Covariance is (just) a function as any other. If you need the standard deviation for the result of the COV function, you ...


1

Here is what I believe might be a counterexample if the intuition were a general claim, or at least a result that seems to indicate that the answer to 2. might be "not really". The measure of the precision of an estimator of a certain moment that I use here is the variance. It is well known that the variance of the sample variance, when sampling ...


1

Comment: Using R to visualize the speed of convergence. n = seq(5,300,by=5) c = 4*n*(1-sqrt(2/(n-1))*gamma(n/2)/gamma((n-1)/2)) plot(n,c); abline(h=1, col="green2", lwd=2)


1

It's not functional forms themselves are inappropriate, but the assumption part. It's a rigid approach (the parameters are often allowed to vary, but it's quite restrictive compared to other approaches). While we have functional forms for a lot of things, many other distributions are not even expressible by (our human) functions. If you need to assume a ...


1

That's right. In the case of ordinary lieast squares (OLS), $Y - \hat{Y} = (I-X(X'X)^{-1}X')Y$ and the fact that $$(I-X(X'X)^{-1}X')X = 0$$ implies that $Y - \hat{Y}$ is orthogonal to any linear combination of the $X$ (and hence to $\hat{Y} = X\hat{\beta}$). However, it is no longer true that $$(I-X(X'X+ \lambda I)^{-1}X')X = 0$$ so orthogonality to $\hat{Y}...


1

I believe there is a simple enough Bayesian solution here. I haven't tried to implement any of this though! Without measurement error, this is a simple Beta-Bernoulli model: $$ \begin{align} p_i &\sim \text{Beta}(\alpha_i, \beta_i)\\ y_{i,j} &\sim \text{Bernoulli}(p_i) \end{align} $$ where $p_i$ is the proportion of people who like movie $i$, and ...


1

One additional example of non-uniqueness of MLE estimator: When you need to estimate the location parameter $\mu$ of the Laplace distribution through ML, you need a value $\hat{\mu}$ such that: $$ \sum_{i=1}^n \frac{|x_i - \hat{\mu}|}{x_i - \hat{\mu}} = 0$$ That is, an estimate $\hat{\mu}$ such that the number of observations below and above $\hat{\mu}$ are ...


Only top voted, non community-wiki answers of a minimum length are eligible