Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

14
votes
0answers
537 views

Is there a general expression for ancillary statistics in exponential families?

It is known that an i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n}...
11
votes
0answers
2k views

When does a UMP test fail to exist?

I have a sample $X=(X_1, ...,X_n)\sim N(\mu,\sigma^2)$ with $\sigma^2$ known. The hypotheses are $H_0: \mu=\mu_0, H_1:\mu \neq \mu_0$. I know that in such a case an UMP test does not exist and so ...
8
votes
0answers
57 views

What is the correct way to write the elastic net?

I am confused about the correct way to write the elastic net. After reading some research papers there seems to be three forms 1) $\exp\{-\lambda_1|\beta_k|-\lambda_2\beta_k^2\}$ 2) $\exp\{-\frac{(\...
7
votes
0answers
124 views

MLE, regularity conditions, finite and infinite parameter spaces

The problem I have is in figuring out why the MLE is no longer consistent in countable parameter spaces under conditions specified below. The set up is as follows: we are consider a parameters space ...
6
votes
0answers
151 views

Time evolution of a Bayesian posterior

I have a question regarding the time evolution of a quantity related to a Bayesian posterior. Suppose we have binary parameter space $\{ s_1, s_2 \}$ with prior $(p, 1-p)$, The data generating ...
6
votes
0answers
238 views

Reference Request: Information Geometry for Ridge Regression

I am reading the book "regression estimators" by Gruber 2010 where he uses this technique to compare Ridge Regressors, however he concentrates on deriving the mathematical results without giving any ...
5
votes
0answers
83 views

French website Providing Instruction/Tutorials on Statistical Theory

This is somewhat of an odd question for CV, but since it's a question about statistical education, I think it falls within the scope of CV. Several years ago I stumbled across a French website that ...
5
votes
0answers
114 views

Multiple maximum likelihood estimates for discrete parameter

Suppose I have a bivariate likelihood function, $L(\theta ,\lambda |\mathbf{x})$, where $\theta$ can take on continuous values, but $\lambda$ can only take 'count' values $(0,1,2,...)$, and $\mathbf{x}...
5
votes
0answers
219 views

Why can't the complete class theorem be easily generalized to all locally-compact spaces?

So I was reading Christian P. Robert's The Bayesian Choice, going through the constellation of results related to complete class theorems, and I don't see why all of them are necessary. In particular, ...
5
votes
0answers
62 views

Asymptotic joint distribution of the sample medians of a collection and a sub-collection of i.i.d. random variables

Let $X_1,\dots, X_n$ denote i.i.d. random variables, with a smooth density $f$ having unique median $m$. It is known that the sample median $M_n$ of $X_1,\dots, X_n$, $M_n = \operatorname{med}(X_1, \...
5
votes
0answers
571 views

Are there unbiased, non-linear estimators with lower variance than the OLS estimator?

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$ The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-...
5
votes
0answers
215 views

Intuitive explanation for Marchenko-Pastur law

I am looking for an intuitive reasoning behind the Marchenko Pastur law, which is described as a law of large numbers analog for random matrices. I know the law gives the probability density function ...
5
votes
0answers
590 views

quantile of standardized t distribution

How to show that, for any given left tail probability, the corresponding quantile of standardized t distribution is increasing in degree of freedom for left tail probability less than 0.5? This is ...
5
votes
0answers
219 views

I'm not asking for a conjugate prior. Is there a distribution $p(x|y)$ that satisfies $\int p(x|y)Beta(y|a,b) dy = Beta(x| a', b')$?

I know the result of integrating a Gaussian against another Gaussian is still Gaussian, $$\int N(x|\mu_y,\sigma_y)N(y|\mu,\sigma) dy = N(x|\mu',\sigma')\quad.$$ Can I get the same form for Beta ...
5
votes
0answers
2k views

Help with a proof of Bayes classifier optimality

I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier. I've never worked through a ...
5
votes
0answers
512 views

Conditional expectation everywhere non-zero while unconditional one zero?

I have a real-valued random variable $X$ that takes on positive and negative values, and $$E(X)=0 \tag{A}$$ There is also another real-valued random variable $Y$, not independent from $X$, neither $...
5
votes
0answers
177 views

Distribution of $X'X$ if $X\in\mathbb{R}^{T \times N}$ and $X_i'\sim N(\mu,\sigma^2I_N)$

Let $x_i\in\mathbb{R}^N$ be multivariate normal distributed with mean vector $\mu\in\mathbb{R}^N$ and no correlation: $x_i\sim N(\mu,\sigma^2 I_N)$. Given $T$ iid samples, define the matrix $$X:=\left(...
5
votes
0answers
806 views

Proof of Kolmogorov-Smirnov test

Could someone provide me a reference, preferably a book, where I can find detailed proofs and explanations of the Kolmogorov-Smirnov test (including the two-sample variant) and the derivation of the $...
5
votes
0answers
105 views

Relation between asymptotic relative efficiency for tests and estimators

The asymptotic relative efficiency for unbiased estimators is the limit of the ratio of the variances as the $n\rightarrow \infty$. Is there a relation to asymptotic relative efficiency according to ...
5
votes
0answers
252 views

Restriction matrix for a VAR

In New Introduction to Multiple Time Series Analysis by Luetkepohl (2005), section 5.2.1, it says that one can specify linear restraints for a VAR, $Y = \beta X + U$, in the form $$ \operatorname{vec}{...
5
votes
0answers
619 views

Sum of independent Wishart with same degrees of freedom but different scale matrices

Is there any result showing that a sum of independent Wishart with same degrees of freedom but different scale matrices is a Wishart? For example, if I have two random variables: $$ Y \sim W_p(n,\...
5
votes
0answers
147 views

Hypothesis test on the Euclidean length of an unknown vector

Question Suppose I observe a vector $\mathbf{x}=[X_1 \ldots X_n]$, where each $X_i=m_i+n_i$, with $n_i$ being an independent zero-mean Gaussian random variable with variance $\sigma^2$ (i.e. $n_i\sim\...
5
votes
0answers
117 views

Orthogonal intersection in a Riemannian manifold

Let $S$ be the set of all probability distributions on $\mathbb{R}$ and $S_n=\{p_\theta\}$ be an $n$ dimensional submanifold of parameterized family of probability distributions on $\mathbb{R}$ where $...
5
votes
0answers
286 views

Derivation of prediction intervals for a normally distributed population with unknown population standard deviation

I have via the ISO standard 16269 found the solution to a problem that I've been working on. Based on a couple of independent samples from a normally distributed population, I would like to determine ...
5
votes
0answers
1k views

Taylor Series and Multivariate Delta Method

I asked this question on https://math.stackexchange.com/ but did not get any answer. Sorry for cross posting. I'm trying to understand delta method for matrices and vectors to find the variance-...
4
votes
0answers
66 views

What is the median of $y_{i}$ given $x_{i}$ for the function $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$

$y_{i}$ is a kx1 matrix, $x_{i}$ is a kxk matrix, $\beta$ is a 1xk matrix of coefficients and $u_{i}$ is a kx1 matrix of error terms. $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$ and $med(u_{i}|x_{i}...
4
votes
0answers
74 views

Why is this statistic F-distributed?

A book I'm reading claims that the statistic: $\frac{(RSS_0 - RSS_1) / (p_1 - p_0)}{RSS_1 / (N - p_1 - 1)}$ has an F distribution. Why is this? I know that an F distribution is something like $\frac{\...
4
votes
0answers
34 views

Uniform distribution on the simplex. - Thomas cover

I'm trying to formulate the solution for the following problem: I was thinking in finding the equivalent distribution on $X_i$ based on $Y_i$, but I think I'm cheating. I think that the autor wants ...
4
votes
0answers
65 views

Proving an inequality for CDF's

I am working on a proof to show that given $x_1, x_2,\ldots,x_k$ random variables with a joint pdf and joint CDF, show that $$ 1-\sum_{i=1}^k \overline{F_i(x_i)} \leq F(x_1,x_2,\ldots,x_k) \leq \min_i ...
4
votes
0answers
40 views

If a statistic can be written as a function of a minimal sufficient statistic almost everywhere, is it minimal sufficient?

I know that if $T(X) = f(W(X))$ for one-to-one $f$, where $W(X)$ is minimal sufficient, then $T(X)$ is also minimal sufficient. But my textbook does not include "almost everywhere" or "almost surely" ...
4
votes
0answers
96 views

Is an inadmissible estimator necessarily dominated by some admissible estimator

Basic example: $X$ has a $p$-variate iid standard Normal distribution; the sample mean is not admissible if $p>2$ and is dominated by the Stein shrinkage estimator. However, the Stein shrinkage ...
4
votes
0answers
174 views

How do I get the p value of AD test using the results of scipy.stats.anderson()

I am conducting an Anderson Darling test for normality using scipy.stats.anderson() command in python. I am getting test statistic, critical values at various significance levels as the output. The ...
4
votes
0answers
65 views

A possible typo in the textbook?

On page 74 of Lehmann's Testing Statistical Hypothesis, the author writes Let $P_0$ and $P_1$ be probability distributions possessing densities $p_0$ and $p_1$ respectively w.r.t. a measure $\mu$ ...
4
votes
0answers
88 views

Maximizing a computationally expensive function

Let $f:[0,1]^{80} \rightarrow [0,1]$ be some function, and say I have a computationally expensive way to calculate $f(x)$ for each $x \in [0,1]^{80}$ (expensive = 40s per query). The goal is to ...
4
votes
0answers
117 views

Finding the Cauchy Principal Value Mean of the pdf for $Z=XY$ where $\mathbb{E}[Y]$ does not exist

Let's first define the Cauchy Principal Value Mean (PVM). For a continuous random variable $V$ with pdf $f_{V}(v)$, the PVM of $f_{V}$ is \begin{equation} \mathrm{PV}(\mathbb{E}[V])=\lim_{a\to\infty}\...
4
votes
0answers
53 views

Why is the $\chi^{2}$ approximation for deviance GLM $\sim \operatorname{Binomial}(n_{i},\pi_{i})$ not valid when $n_{i} = 1$?

I know from McCullagh & Nelder's text (p.118) that the $\chi^{2}$ approximation for deviance for the binomial family is based on a limiting operation in which $n$, the number of observations, is ...
4
votes
0answers
164 views

Calculating Formula with logs and Weighted Average

I am trying to figure out an equation from the following paper by Cadena and Kovak (2016): http://pubs.aeaweb.org/doi/pdfplus/10.1257/app.20140095 on pages 264 and 265. I don't think the context is ...
4
votes
0answers
69 views

Hausdorff (fractal) Dimension of a Stochastic Process

It is well known that Brownian motion (BM) has a Hausdorff dimension (ie fractal dimension) of 2, for topological dimension >= 2. In other words, BM always "behaves like" a plane surface, no matter ...
4
votes
0answers
46 views

Data Transformation Question - Multiplying data proportional to demographics

I have a bunch of data that is tied to demographic variables (Age, Sex, Income, Education, etc.). However, the data is sent by one person in a household for the entire house. It's numerical data and I ...
4
votes
0answers
152 views

Rigorous theory behind overfitting

I am taking an intro to ML class, and in my limited experience, training ML algorithms (validation, overfitting etc.) feels a bit like black magic. For instance, you aren't supposed to touch the test ...
4
votes
0answers
211 views

Find pdf of a multivariate function with log-normal distributions

Consider the following function: $f(x,t) = \dfrac{1}{2} \dfrac{1}{\sqrt{\pi(d_0+\alpha v) t}}\exp\left(-\dfrac{x^2}{4(d_0+\alpha v)t}-\lambda t\right)$ where the parameters $d_0,\alpha,\lambda$ are ...
4
votes
0answers
670 views

Kurtosis, bias, unbiased and statistics

I apologize ahead of time if this is too vague or meta to be a valid question. I've been looking at Algorithms (Sedgewick & Wayne). They define a class stdstats. In that they define min, max, ...
4
votes
0answers
122 views

Truncated trivariate normal - conditional expectation

I am working on a paper in which I'd need to use the two following conditional expectations: $E(X_{1}|a \leq X_{2} \leq b)$ $E(X_{1}|a \leq X_{2} \leq b, a \leq X_{3} \leq b)$ where $X_{1}, X_{2}, ...
4
votes
0answers
2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
4
votes
0answers
259 views

What properties of a likelihood function are required for quasi-likelihood estimation?

Quasi-likelihood seems like a great way to use Iteratively Weighted Least Squares to fit linear models with a very general class of likelihoods. But what is that class? Obviously the distribution ...
4
votes
0answers
232 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Theoremm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
4
votes
0answers
295 views

The distribution of the scalar product of multivariate complex normal vectors

Assume two independent random complex vectors (with real and imaginary parts): $$ \vec{\dot{Z}}=(\dot{z}_1,\dot{z}_2,\ldots,\dot{z}_n)~~~~~~~~~~ \begin{cases} \Re(\dot{z}_i)&=x_i \\ \Im(\dot{z}_i)...
4
votes
0answers
282 views

Likelihood ratio test: $f(x)=2x$ vs $f(x)=3x^2$: $2n$ degrees of freedom?

Suppose $X_1, . . . , X_n$ are i.i.d. with pdf $f(·)$. We want to test the hypotheses \begin{align} H_0 &: f(x) = 2x , \;\, \text{ for } 0 \le x \le 1, \text{ against}, \\ H_1 &: f(x) = 3x^2 , ...
4
votes
0answers
295 views

Gibbs sampling from full conditionals

I have the following joint density: $p(x_1,x_2,y_1,y_2) \propto \exp\left(−\left(x_1^2+x_2^2+c_1(y_2-y_1)^2+c_2(y_2-y_1)^4\right)\right)$ Can I use Gibbs sampling to sample from that? How can I get ...
4
votes
0answers
115 views

Smooth expectations outside the exponential family

At page 85-86 of Young and Smith "Essentials of Statistical Inference" there is an interesting result. If $X$ is a r.v. distributed according to the exponential family and $\phi(x)$ is a bounded (but ...