Questions tagged [mathematical-statistics]
Mathematical theory of statistics, concerned with formal definitions and general results.
1,415
questions with no upvoted or accepted answers
37
votes
2answers
2k views
Probability inequalities
I am looking for some probability inequalities for sums of unbounded random variables. I would really appreciate it if anyone can provide me some thoughts.
My problem is to find an exponential upper ...
14
votes
0answers
542 views
Is there a general expression for ancillary statistics in exponential families?
It is known that an i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n}...
12
votes
0answers
2k views
When does a UMP test fail to exist?
I have a sample $X=(X_1, ...,X_n)\sim N(\mu,\sigma^2)$ with $\sigma^2$ known. The hypotheses are $H_0: \mu=\mu_0, H_1:\mu \neq \mu_0$.
I know that in such a case an UMP test does not exist and so ...
9
votes
1answer
207 views
Topologies for which the ensemble of probability distributions is complete
I have been struggling quite a bit with reconciling my intuitive understanding of probability distributions with the weird properties that almost all topologies on probability distributions possess.
...
8
votes
0answers
59 views
What is the correct way to write the elastic net?
I am confused about the correct way to write the elastic net. After reading some research papers there seems to be three forms
1) $\exp\{-\lambda_1|\beta_k|-\lambda_2\beta_k^2\}$
2) $\exp\{-\frac{(\...
8
votes
1answer
149 views
Do random variables follow the same algebraic rules as ordinary numbers?
In the comments on my answer to a recent question about the sum of random variables, I came across a link to the Wikipedia article on the ratio distribution, and noticed the following peculiar claim ...
8
votes
1answer
980 views
How to show that a sufficient statistic is NOT minimal sufficient?
My homework problem is to give a counterexample where a certain statistic is not in general minimal sufficient. Irrespective of the details of finding a particular counterexample for this particular ...
7
votes
0answers
152 views
MLE, regularity conditions, finite and infinite parameter spaces
The problem I have is in figuring out why the MLE is no longer consistent in countable parameter spaces under conditions specified below.
The set up is as follows: we are consider a parameters space ...
7
votes
1answer
152 views
Finding the distribution of sample range for a Beta population
Let $X_1,X_2,\ldots,X_n$ be i.i.d random variables having density
$$f(x)=2(1-x)\mathbf1_{0<x<1}$$
I am trying to derive the distribution of the sample range $R=X_{(n)}-X_{(1)}$.
The ...
6
votes
1answer
255 views
Expected squared distance from origin of training points vs. test points
This is from Exercise 2.4 (Page 39) of Elements of Statistical Learning:
The edge effect problem discussed on page 23 is not peculiar to uniform sampling from bounded domains. Consider inputs drawn ...
6
votes
1answer
1k views
score function of bivariate/multivariate normal distribution
I tried to obtain the score vector (1st derivative of density function w.r.t. parameters) of multivariate normal distribution.
Density function of multivariate normal distribution:
\begin{align*}
f(x)...
6
votes
0answers
155 views
Time evolution of a Bayesian posterior
I have a question regarding the time evolution of a quantity related to a Bayesian posterior.
Suppose we have binary parameter space $\{ s_1, s_2 \}$ with prior $(p, 1-p)$,
The data generating ...
6
votes
0answers
239 views
Reference Request: Information Geometry for Ridge Regression
I am reading the book "regression estimators" by Gruber 2010 where he uses this technique to compare Ridge Regressors, however he concentrates on deriving the mathematical results without giving any ...
6
votes
0answers
1k views
Taylor Series and Multivariate Delta Method
I asked this question on https://math.stackexchange.com/ but did not get any answer. Sorry for cross posting.
I'm trying to understand delta method for matrices and vectors to find the variance-...
5
votes
1answer
5k views
What does “def” above an equals sign mean?
I am reading this:
https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf
and on equation (17), there is a def on top of the equal sign. What does this mean?
5
votes
0answers
83 views
French website Providing Instruction/Tutorials on Statistical Theory
This is somewhat of an odd question for CV, but since it's a question about statistical education, I think it falls within the scope of CV. Several years ago I stumbled across a French website that ...
5
votes
0answers
126 views
Multiple maximum likelihood estimates for discrete parameter
Suppose I have a bivariate likelihood function, $L(\theta ,\lambda |\mathbf{x})$, where $\theta$ can take on continuous values, but $\lambda$ can only take 'count' values $(0,1,2,...)$, and $\mathbf{x}...
5
votes
0answers
237 views
Why can't the complete class theorem be easily generalized to all locally-compact spaces?
So I was reading Christian P. Robert's The Bayesian Choice, going through the constellation of results related to complete class theorems, and I don't see why all of them are necessary. In particular, ...
5
votes
0answers
62 views
Asymptotic joint distribution of the sample medians of a collection and a sub-collection of i.i.d. random variables
Let $X_1,\dots, X_n$ denote i.i.d. random variables, with a smooth density $f$ having unique median $m$.
It is known that the sample median $M_n$ of $X_1,\dots, X_n$, $M_n = \operatorname{med}(X_1, \...
5
votes
0answers
600 views
Are there unbiased, non-linear estimators with lower variance than the OLS estimator?
Consider an ordinary least squares model,
$$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$
The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-...
5
votes
1answer
59 views
Statistical model with $\Gamma(\alpha_i,1)$ sample
We are given statistical sample of $X=(X_1,X_2,X_3)$, where $X_i\sim\Gamma(\alpha_i,1)$ and independent. Let $Z=X_1+X_2+X_3$ and $T$ three dimensional statistic $T:=(\frac{X_1}{Z},\frac{X_2}{Z},\frac{...
5
votes
1answer
108 views
Binomial confidence intervals - which is correct?
Background: I am working with a data set that requires a transformation. It's prevalence data so I have proportions to deal with, and as the proportions are quite low, I'm using the Freeman-Tukey ...
5
votes
0answers
257 views
Intuitive explanation for Marchenko-Pastur law
I am looking for an intuitive reasoning behind the Marchenko Pastur law, which is described as a law of large numbers analog for random matrices. I know the law gives the probability density function ...
5
votes
0answers
597 views
quantile of standardized t distribution
How to show that, for any given left tail probability, the corresponding quantile of standardized t distribution is increasing in degree of freedom for left tail probability less than 0.5? This is ...
5
votes
0answers
220 views
I'm not asking for a conjugate prior. Is there a distribution $p(x|y)$ that satisfies $\int p(x|y)Beta(y|a,b) dy = Beta(x| a', b')$?
I know the result of integrating a Gaussian against another Gaussian is still Gaussian, $$\int N(x|\mu_y,\sigma_y)N(y|\mu,\sigma) dy = N(x|\mu',\sigma')\quad.$$
Can I get the same form for Beta ...
5
votes
0answers
173 views
Cox's Theorem: ignorance, objective priors, and the Mind Projection Fallacy
I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
5
votes
1answer
374 views
Why low rank expansions can exploit the redundancy that exist between different feature channels and filters?
I read Jaderberg et al., 2014 paper about Speeding up Convolutional Neural Network with Low Rank Expansions. In the introduction, it is written in bold font:
Our key insight is to exploit the ...
5
votes
1answer
103 views
What is an example of data where the permutation test succeeds but a normal t-test fails?
In literature, I normally see authors use a two sample permutation test on normal data to show that it works as well as the two sample t-test. However, the real power for permutation tests should be ...
5
votes
0answers
2k views
Help with a proof of Bayes classifier optimality
I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier.
I've never worked through a ...
5
votes
1answer
1k views
Basu's Theorem Proof
I am having trouble with the proof of Basu's theorem... specifically, I'm not sure about the $\theta$s in the expectations below:
Let $T$ be a complete sufficient statistic. Let $V$ be an ancillary ...
5
votes
0answers
125 views
Truncated trivariate normal - conditional expectation
I am working on a paper in which I'd need to use the two following conditional expectations:
$E(X_{1}|a \leq X_{2} \leq b)$
$E(X_{1}|a \leq X_{2} \leq b, a \leq X_{3} \leq b)$
where $X_{1}, X_{2}, ...
5
votes
1answer
130 views
How to prove the properties of penalized likelihood estimator in Fan and Li (2001) paper
I'm reading through Fan and Li's paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties". In Page 2 near bottom right corner, they proposed three properties that a ...
5
votes
0answers
544 views
Conditional expectation everywhere non-zero while unconditional one zero?
I have a real-valued random variable $X$ that takes on positive and negative values, and
$$E(X)=0 \tag{A}$$
There is also another real-valued random variable $Y$, not independent from $X$, neither $...
5
votes
0answers
178 views
Distribution of $X'X$ if $X\in\mathbb{R}^{T \times N}$ and $X_i'\sim N(\mu,\sigma^2I_N)$
Let $x_i\in\mathbb{R}^N$ be multivariate normal distributed with mean vector $\mu\in\mathbb{R}^N$ and no correlation: $x_i\sim N(\mu,\sigma^2 I_N)$.
Given $T$ iid samples, define the matrix
$$X:=\left(...
5
votes
0answers
823 views
Proof of Kolmogorov-Smirnov test
Could someone provide me a reference, preferably a book, where I can find detailed proofs and explanations of the Kolmogorov-Smirnov test (including the two-sample variant) and the derivation of the $...
5
votes
0answers
105 views
Relation between asymptotic relative efficiency for tests and estimators
The asymptotic relative efficiency for unbiased estimators is the limit of the ratio of the variances as the $n\rightarrow \infty$. Is there a relation to asymptotic relative efficiency according to ...
5
votes
1answer
253 views
minimizer weighted linear regression
In a regression problem, with $y=X\theta+\epsilon$ and $X$ is an $n$ by $p$ matrix
the ‘weighted least squares estimate is the minimizer $\theta^{*}$ of $f(\theta)=\sum_{i=1}^{n}\omega_{i}(y_i-x_i^{'}...
5
votes
0answers
261 views
Restriction matrix for a VAR
In New Introduction to Multiple Time Series Analysis by Luetkepohl (2005), section 5.2.1, it says that one can specify linear restraints for a VAR, $Y = \beta X + U$, in the form
$$
\operatorname{vec}{...
5
votes
0answers
626 views
Sum of independent Wishart with same degrees of freedom but different scale matrices
Is there any result showing that a sum of independent Wishart with same degrees of freedom but different scale matrices is a Wishart?
For example, if I have two random variables:
$$
Y \sim W_p(n,\...
5
votes
0answers
147 views
Hypothesis test on the Euclidean length of an unknown vector
Question
Suppose I observe a vector $\mathbf{x}=[X_1 \ldots X_n]$, where each $X_i=m_i+n_i$, with $n_i$ being an independent zero-mean Gaussian random variable with variance $\sigma^2$ (i.e. $n_i\sim\...
5
votes
0answers
117 views
Orthogonal intersection in a Riemannian manifold
Let $S$ be the set of all probability distributions on $\mathbb{R}$ and $S_n=\{p_\theta\}$ be an $n$ dimensional submanifold of parameterized family of probability distributions on $\mathbb{R}$ where $...
5
votes
0answers
289 views
Derivation of prediction intervals for a normally distributed population with unknown population standard deviation
I have via the ISO standard 16269 found the solution to a problem that I've been working on. Based on a couple of independent samples from a normally distributed population, I would like to determine ...
4
votes
0answers
50 views
A consistent estimator with infinite expectation?
Typical (or common) approaches to prove an estimator is consistent require finite mean and variance. The proofs usually follow from concentration bounds, e.g. Markov, Chebyshev, etc.
I'm wondering ...
4
votes
0answers
67 views
What is the median of $y_{i}$ given $x_{i}$ for the function $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$
$y_{i}$ is a kx1 matrix, $x_{i}$ is a kxk matrix, $\beta$ is a 1xk matrix of coefficients and $u_{i}$ is a kx1 matrix of error terms.
$y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$
and $med(u_{i}|x_{i}...
4
votes
0answers
75 views
Why is this statistic F-distributed?
A book I'm reading claims that the statistic:
$\frac{(RSS_0 - RSS_1) / (p_1 - p_0)}{RSS_1 / (N - p_1 - 1)}$ has an F distribution. Why is this? I know that an F distribution is something like $\frac{\...
4
votes
0answers
35 views
Uniform distribution on the simplex. - Thomas cover
I'm trying to formulate the solution for the following problem:
I was thinking in finding the equivalent distribution on $X_i$ based on $Y_i$, but I think I'm cheating. I think that the autor wants ...
4
votes
0answers
68 views
Proving an inequality for CDF's
I am working on a proof to show that given $x_1, x_2,\ldots,x_k$ random variables with a joint pdf and joint CDF, show that $$ 1-\sum_{i=1}^k \overline{F_i(x_i)} \leq F(x_1,x_2,\ldots,x_k) \leq \min_i ...
4
votes
0answers
41 views
If a statistic can be written as a function of a minimal sufficient statistic almost everywhere, is it minimal sufficient?
I know that if $T(X) = f(W(X))$ for one-to-one $f$, where $W(X)$ is minimal sufficient, then $T(X)$ is also minimal sufficient. But my textbook does not include "almost everywhere" or "almost surely" ...
4
votes
0answers
107 views
Is an inadmissible estimator necessarily dominated by some admissible estimator
Basic example: $X$ has a $p$-variate iid standard Normal distribution; the sample mean is not admissible if $p>2$ and is dominated by the Stein shrinkage estimator.
However, the Stein shrinkage ...
