Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

1,814 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
14
votes
0answers
633 views

Is there a general expression for ancillary statistics in exponential families?

An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{...
9
votes
0answers
312 views

MLE, regularity conditions, finite and infinite parameter spaces

The problem I have is in figuring out why the MLE is no longer consistent in countable parameter spaces under conditions specified below. The set up is as follows: we are consider a parameters space ...
8
votes
0answers
822 views

Are there unbiased, non-linear estimators with lower variance than the OLS estimator?

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$ The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-...
7
votes
1answer
7k views

What does “def” above an equals sign mean?

I am reading this: https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf and on equation (17), there is a def on top of the equal sign. What does this mean?
7
votes
0answers
1k views

Taylor Series and Multivariate Delta Method

I asked this question on https://math.stackexchange.com/ but did not get any answer. Sorry for cross posting. I'm trying to understand delta method for matrices and vectors to find the variance-...
7
votes
1answer
189 views

Finding the distribution of sample range for a Beta population

Let $X_1,X_2,\ldots,X_n$ be i.i.d random variables having density $$f(x)=2(1-x)\mathbf1_{0<x<1}$$ I am trying to derive the distribution of the sample range $R=X_{(n)}-X_{(1)}$. The usual way I ...
6
votes
1answer
85 views

Asymptotic joint distribution of the sample medians of a collection and a sub-collection of i.i.d. random variables

Let $X_1,\dots, X_n$ denote i.i.d. random variables, with a smooth density $f$ having unique median $m$. It is known that the sample median $M_n$ of $X_1,\dots, X_n$, $M_n = \operatorname{med}(X_1, \...
6
votes
1answer
393 views

Expected squared distance from origin of training points vs. test points

This is from Exercise 2.4 (Page 39) of Elements of Statistical Learning: The edge effect problem discussed on page 23 is not peculiar to uniform sampling from bounded domains. Consider inputs drawn ...
6
votes
0answers
177 views

Time evolution of a Bayesian posterior

I have a question regarding the time evolution of a quantity related to a Bayesian posterior. Suppose we have binary parameter space $\{ s_1, s_2 \}$ with prior $(p, 1-p)$, The data generating ...
6
votes
0answers
433 views

Intuitive explanation for Marchenko-Pastur law

I am looking for an intuitive reasoning behind the Marchenko Pastur law, which is described as a law of large numbers analog for random matrices. I know the law gives the probability density function ...
6
votes
0answers
756 views

quantile of standardized t distribution

How to show that, for any given left tail probability, the corresponding quantile of standardized t distribution is increasing in degree of freedom for left tail probability less than 0.5? This is ...
6
votes
0answers
206 views

Cox's Theorem: ignorance, objective priors, and the Mind Projection Fallacy

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
6
votes
0answers
929 views

Proof of Kolmogorov-Smirnov test

Could someone provide me a reference, preferably a book, where I can find detailed proofs and explanations of the Kolmogorov-Smirnov test (including the two-sample variant) and the derivation of the $...
6
votes
1answer
309 views

minimizer weighted linear regression

In a regression problem, with $y=X\theta+\epsilon$ and $X$ is an $n$ by $p$ matrix the ‘weighted least squares estimate is the minimizer $\theta^{*}$ of $f(\theta)=\sum_{i=1}^{n}\omega_{i}(y_i-x_i^{'}\...
6
votes
0answers
254 views

Reference Request: Information Geometry for Ridge Regression

I am reading the book "regression estimators" by Gruber 2010 where he uses this technique to compare Ridge Regressors, however he concentrates on deriving the mathematical results without ...
6
votes
0answers
125 views

Orthogonal intersection in a Riemannian manifold

Let $S$ be the set of all probability distributions on $\mathbb{R}$ and $S_n=\{p_\theta\}$ be an $n$ dimensional submanifold of parameterized family of probability distributions on $\mathbb{R}$ where $...
5
votes
0answers
30 views

Simulate correlate random variables with given marginal distribution where one is always larger

Is it possible to simulate pairs of random variables with a given marginal distribution and population correlation where one random variable is larger than the other? More formally, I need to simulate ...
5
votes
0answers
225 views

Strange connection between Bernouilli, Uniform and Geometric distributions

Final update on 11/29/2019: I have worked on this a bit more, and wrote an article summarizing all the main findings. You can read it here. Let us consider $Z = X_1 + X_1 X_2 + X_1 X_2 X_3 +\cdots$ ...
5
votes
2answers
79 views

How to win this dice probability game?

The game is a variation of Pig. Here is how the game works: There are about 20 players. Each round, a single six sided die is rolled. All players add that rolled number to their "bank." However, if a ...
5
votes
0answers
86 views

French website Providing Instruction/Tutorials on Statistical Theory

This is somewhat of an odd question for CV, but since it's a question about statistical education, I think it falls within the scope of CV. Several years ago I stumbled across a French website that ...
5
votes
1answer
755 views

How do I get the p value of AD test using the results of scipy.stats.anderson()

I am conducting an Anderson Darling test for normality using scipy.stats.anderson() command in python. I am getting test statistic, critical values at various significance levels as the output. The ...
5
votes
0answers
438 views

Does the definition of regular estimator depend on the rate of convergence? If not, should it?

The definition of regular estimator in my lecture notes is: Let $X_1^{(n)}, \dots, X_n^{(n)} \overset{iid}{\sim} P_n \sim \mathcal{P}(\Theta)$ where $\mathcal{P}(\Theta)$ is a regular parametric ...
5
votes
0answers
316 views

Why can't the complete class theorem be easily generalized to all locally-compact spaces?

So I was reading Christian P. Robert's The Bayesian Choice, going through the constellation of results related to complete class theorems, and I don't see why all of them are necessary. In particular, ...
5
votes
0answers
1k views

Calculating a confidence interval for a weighted sample

In a nutshell, I'd like to compute a confidence interval for some weighted sample day where the final value I'm seeking is a sum of different weighted samples. Illustrated here with a toy example set ...
5
votes
0answers
128 views

Finding the Cauchy Principal Value Mean of the pdf for $Z=XY$ where $\mathbb{E}[Y]$ does not exist

Let's first define the Cauchy Principal Value Mean (PVM). For a continuous random variable $V$ with pdf $f_{V}(v)$, the PVM of $f_{V}$ is \begin{equation} \mathrm{PV}(\mathbb{E}[V])=\lim_{a\to\infty}\...
5
votes
0answers
600 views

Under what conditions will a Bayesian posterior fail to converge to a point mass?

Let's say you have a Bayesian model: $$\theta' \sim g(\theta|\mu) $$ $$ y \sim p(y|\theta')$$ And we have some data ($n$ data points) $\mathbf{y}_n$, which we will use to perform inference on $\...
5
votes
0answers
60 views

Why is the $\chi^{2}$ approximation for deviance GLM $\sim \operatorname{Binomial}(n_{i},\pi_{i})$ not valid when $n_{i} = 1$?

I know from McCullagh & Nelder's text (p.118) that the $\chi^{2}$ approximation for deviance for the binomial family is based on a limiting operation in which $n$, the number of observations, is ...
5
votes
0answers
234 views

I'm not asking for a conjugate prior. Is there a distribution $p(x|y)$ that satisfies $\int p(x|y)Beta(y|a,b) dy = Beta(x| a', b')$?

I know the result of integrating a Gaussian against another Gaussian is still Gaussian, $$\int N(x|\mu_y,\sigma_y)N(y|\mu,\sigma) dy = N(x|\mu',\sigma')\quad.$$ Can I get the same form for Beta ...
5
votes
0answers
142 views

Cox's Theorem: the necessity of (un)countably additivity

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
5
votes
1answer
389 views

Why low rank expansions can exploit the redundancy that exist between different feature channels and filters?

I read Jaderberg et al., 2014 paper about Speeding up Convolutional Neural Network with Low Rank Expansions. In the introduction, it is written in bold font: Our key insight is to exploit the ...
5
votes
1answer
112 views

What is an example of data where the permutation test succeeds but a normal t-test fails?

In literature, I normally see authors use a two sample permutation test on normal data to show that it works as well as the two sample t-test. However, the real power for permutation tests should be ...
5
votes
0answers
937 views

Kurtosis, bias, unbiased and statistics

I apologize ahead of time if this is too vague or meta to be a valid question. I've been looking at Algorithms (Sedgewick & Wayne). They define a class stdstats. In that they define min, max, ...
5
votes
0answers
3k views

Help with a proof of Bayes classifier optimality

I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier. I've never worked through a ...
5
votes
0answers
777 views

Conditional expectation everywhere non-zero while unconditional one zero?

I have a real-valued random variable $X$ that takes on positive and negative values, and $$E(X)=0 \tag{A}$$ There is also another real-valued random variable $Y$, not independent from $X$, neither $...
5
votes
0answers
185 views

Distribution of $X'X$ if $X\in\mathbb{R}^{T \times N}$ and $X_i'\sim N(\mu,\sigma^2I_N)$

Let $x_i\in\mathbb{R}^N$ be multivariate normal distributed with mean vector $\mu\in\mathbb{R}^N$ and no correlation: $x_i\sim N(\mu,\sigma^2 I_N)$. Given $T$ iid samples, define the matrix $$X:=\left(...
5
votes
0answers
110 views

Relation between asymptotic relative efficiency for tests and estimators

The asymptotic relative efficiency for unbiased estimators is the limit of the ratio of the variances as the $n\rightarrow \infty$. Is there a relation to asymptotic relative efficiency according to ...
5
votes
0answers
286 views

Restriction matrix for a VAR

In New Introduction to Multiple Time Series Analysis by Luetkepohl (2005), section 5.2.1, it says that one can specify linear restraints for a VAR, $Y = \beta X + U$, in the form $$ \operatorname{vec}{...
5
votes
0answers
662 views

Sum of independent Wishart with same degrees of freedom but different scale matrices

Is there any result showing that a sum of independent Wishart with same degrees of freedom but different scale matrices is a Wishart? For example, if I have two random variables: $$ Y \sim W_p(n,\...
5
votes
0answers
148 views

Hypothesis test on the Euclidean length of an unknown vector

Question Suppose I observe a vector $\mathbf{x}=[X_1 \ldots X_n]$, where each $X_i=m_i+n_i$, with $n_i$ being an independent zero-mean Gaussian random variable with variance $\sigma^2$ (i.e. $n_i\sim\...
5
votes
0answers
316 views

Derivation of prediction intervals for a normally distributed population with unknown population standard deviation

I have via the ISO standard 16269 found the solution to a problem that I've been working on. Based on a couple of independent samples from a normally distributed population, I would like to determine ...
4
votes
1answer
129 views

Is there a commonly-accepted/used notion of parametric statistical model equivalence?

To fix notation, let a set of possible data $X$ and a set of admissible parameter values $\Theta$ be given. Let $\mathscr P(X)$ be the set of probability distributions on $X$. A parametric ...
4
votes
1answer
67 views

Which is a better estimator, averaged functions vs. A function of an average?

Problem: Assume that we want to estimate $f(\theta)$ with a pre-specified strictly increasing function $f$ and a parameter $\theta$. Let $\hat{\theta}_1$ and $\hat{\theta}_2$ be unbiased estimators ...
4
votes
0answers
87 views

Deriving spectral measure

While reading this book, I got stuck on page 266 where the authors found the spectral measure $F(du)$ of the generalized covariance function $K(h) = \Gamma(-\alpha/2) |h|^{\alpha}, ~0<\alpha<2.$ ...
4
votes
0answers
66 views

A consistent estimator with infinite expectation?

Typical (or common) approaches to prove an estimator is consistent require finite mean and variance. The proofs usually follow from concentration bounds, e.g. Markov, Chebyshev, etc. I'm wondering ...
4
votes
0answers
113 views

The difference of normal means is also minimax?

Let $X_i \sim N(\xi, \sigma^2)$ and $Y_i \sim N(\eta, \tau^2)$ for known $\sigma^2$ and $\tau^2$. I know that $\bar{X}$ and $\bar{Y}$ are minimax under squared error loss since their variance is ...
4
votes
0answers
69 views

What is the median of $y_{i}$ given $x_{i}$ for the function $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$

$y_{i}$ is a kx1 matrix, $x_{i}$ is a kxk matrix, $\beta$ is a 1xk matrix of coefficients and $u_{i}$ is a kx1 matrix of error terms. $y_i=\max\{0, x_{i}^{\prime}\beta + u_{i}\}$ and $med(u_{i}|x_{i}...
4
votes
0answers
83 views

Why is this statistic F-distributed?

A book I'm reading claims that the statistic: $\frac{(RSS_0 - RSS_1) / (p_1 - p_0)}{RSS_1 / (N - p_1 - 1)}$ has an F distribution. Why is this? I know that an F distribution is something like $\frac{\...
4
votes
0answers
38 views

Uniform distribution on the simplex. - Thomas cover

I'm trying to formulate the solution for the following problem: I was thinking in finding the equivalent distribution on $X_i$ based on $Y_i$, but I think I'm cheating. I think that the autor wants ...
4
votes
0answers
86 views

Incorrect computation in Knight and Fu (2000)?

I'm currently reading Knight and Fu's 2000 paper on the asymptotics of "Bridge" estimators with a particular focus on LASSO as a special case. In the proof of theorem 2, they make the claim that under ...

1
2 3 4 5
37