Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

1,371 questions
537 views

124 views

MLE, regularity conditions, finite and infinite parameter spaces

The problem I have is in figuring out why the MLE is no longer consistent in countable parameter spaces under conditions specified below. The set up is as follows: we are consider a parameters space ...
151 views

Time evolution of a Bayesian posterior

I have a question regarding the time evolution of a quantity related to a Bayesian posterior. Suppose we have binary parameter space $\{ s_1, s_2 \}$ with prior $(p, 1-p)$, The data generating ...
238 views

Reference Request: Information Geometry for Ridge Regression

I am reading the book "regression estimators" by Gruber 2010 where he uses this technique to compare Ridge Regressors, however he concentrates on deriving the mathematical results without giving any ...
83 views

French website Providing Instruction/Tutorials on Statistical Theory

This is somewhat of an odd question for CV, but since it's a question about statistical education, I think it falls within the scope of CV. Several years ago I stumbled across a French website that ...
114 views

571 views

Are there unbiased, non-linear estimators with lower variance than the OLS estimator?

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$ The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-...
215 views

Intuitive explanation for Marchenko-Pastur law

I am looking for an intuitive reasoning behind the Marchenko Pastur law, which is described as a law of large numbers analog for random matrices. I know the law gives the probability density function ...
590 views

quantile of standardized t distribution

How to show that, for any given left tail probability, the corresponding quantile of standardized t distribution is increasing in degree of freedom for left tail probability less than 0.5? This is ...
219 views

I'm not asking for a conjugate prior. Is there a distribution $p(x|y)$ that satisfies $\int p(x|y)Beta(y|a,b) dy = Beta(x| a', b')$?

I know the result of integrating a Gaussian against another Gaussian is still Gaussian, $$\int N(x|\mu_y,\sigma_y)N(y|\mu,\sigma) dy = N(x|\mu',\sigma')\quad.$$ Can I get the same form for Beta ...
2k views

Help with a proof of Bayes classifier optimality

I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier. I've never worked through a ...
512 views

2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
259 views

What properties of a likelihood function are required for quasi-likelihood estimation?

Quasi-likelihood seems like a great way to use Iteratively Weighted Least Squares to fit linear models with a very general class of likelihoods. But what is that class? Obviously the distribution ...
232 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Theoremm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
295 views

The distribution of the scalar product of multivariate complex normal vectors

Assume two independent random complex vectors (with real and imaginary parts):  \vec{\dot{Z}}=(\dot{z}_1,\dot{z}_2,\ldots,\dot{z}_n)~~~~~~~~~~ \begin{cases} \Re(\dot{z}_i)&=x_i \\ \Im(\dot{z}_i)...
282 views

Likelihood ratio test: $f(x)=2x$ vs $f(x)=3x^2$: $2n$ degrees of freedom?

Suppose $X_1, . . . , X_n$ are i.i.d. with pdf $f(·)$. We want to test the hypotheses \begin{align} H_0 &: f(x) = 2x , \;\, \text{ for } 0 \le x \le 1, \text{ against}, \\ H_1 &: f(x) = 3x^2 , ...
I have the following joint density: $p(x_1,x_2,y_1,y_2) \propto \exp\left(−\left(x_1^2+x_2^2+c_1(y_2-y_1)^2+c_2(y_2-y_1)^4\right)\right)$ Can I use Gibbs sampling to sample from that? How can I get ...
At page 85-86 of Young and Smith "Essentials of Statistical Inference" there is an interesting result. If $X$ is a r.v. distributed according to the exponential family and $\phi(x)$ is a bounded (but ...