# Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

2,269 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
882 views

An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{... 11 votes 0 answers 2k views ### What does it mean to take the expectation with respect to a probability distribution? I see this expectation in a lot of machine learning literature: $$\mathbb{E}_{p(\mathbf{x};\mathbf{\theta})}[f(\mathbf{x};\mathbf{\phi})] = \int p(\mathbf{x};\mathbf{\theta}) f(\mathbf{x};\mathbf{\phi}... 9 votes 1 answer 12k views ### What does "def" above an equals sign mean? I am reading this: https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf and on equation (17), there is a def on top of the equal sign. What does this mean? 9 votes 0 answers 2k views ### Taylor Series and Multivariate Delta Method I asked this question on https://math.stackexchange.com/ but did not get any answer. Sorry for cross posting. I'm trying to understand delta method for matrices and vectors to find the variance-... 8 votes 0 answers 777 views ### Intuitive explanation for Marchenko-Pastur law I am looking for an intuitive reasoning behind the Marchenko Pastur law, which is described as a law of large numbers analog for random matrices. I know the law gives the probability density function ... 8 votes 0 answers 1k views ### Proof of Kolmogorov-Smirnov test Could someone provide me a reference, preferably a book, where I can find detailed proofs and explanations of the Kolmogorov-Smirnov test (including the two-sample variant) and the derivation of the ... 8 votes 0 answers 142 views ### Orthogonal intersection in a Riemannian manifold Let S be the set of all probability distributions on \mathbb{R} and S_n=\{p_\theta\} be an n dimensional submanifold of parameterized family of probability distributions on \mathbb{R} where ... 8 votes 1 answer 242 views ### Finding the distribution of sample range for a Beta population Let X_1,X_2,\ldots,X_n be i.i.d random variables having density$$f(x)=2(1-x)\mathbf1_{0<x<1}$$I am trying to derive the distribution of the sample range R=X_{(n)}-X_{(1)}. The usual way I ... 7 votes 0 answers 367 views ### Sklar’s Extension Theorem and support restrictions This question is about an application of the Sklar's Extension Theorem, whose proof can be found in Sklar, A. (1996), "Random variables, distribution functions, and copulas: A personal look ... 7 votes 0 answers 109 views ### Proof: Nearest Neighbor classifier achieves Bayes rate asymptotically on countable domains I am trying to understand in which situations the 1-NN classifier asymptotically attains the Bayes error rate. My intuition is that if the domain is countable, then 1-NN will asymptotically do as well ... 7 votes 0 answers 51 views ### Simulate correlate random variables with given marginal distribution where one is always larger Is it possible to simulate pairs of random variables with a given marginal distribution and population correlation where one random variable is larger than the other? More formally, I need to simulate ... 7 votes 0 answers 128 views ### How do we call a more extreme case of fat tails than a power law? According to Wikipedia the most extreme case of a fat tail follows a power law: The most extreme case of a fat tail is given by a distribution whose tail decays like a power law. That is, if the ... 7 votes 0 answers 105 views ### Inequalities on Fisher Information / expected second derivative? Under some regularity conditions we can compute fisher information as - \mathbb{E}_{\theta_0} [\frac{\partial}{\partial \theta^2} \ln f(x;\theta_0)] I was wondering if there are some kind of ... 7 votes 0 answers 314 views ### Strange connection between Bernouilli, Uniform and Geometric distributions Final update on 11/29/2019: I have worked on this a bit more, and wrote an article summarizing all the main findings. You can read it here. Let us consider Z = X_1 + X_1 X_2 + X_1 X_2 X_3 +\cdots ... 7 votes 0 answers 97 views ### French website Providing Instruction/Tutorials on Statistical Theory This is somewhat of an odd question for CV, but since it's a question about statistical education, I think it falls within the scope of CV. Several years ago I stumbled across a French website that ... 7 votes 0 answers 224 views ### Time evolution of a Bayesian posterior I have a question regarding the time evolution of a quantity related to a Bayesian posterior. Suppose we have binary parameter space \{ s_1, s_2 \} with prior (p, 1-p), The data generating ... 7 votes 0 answers 234 views ### Cox's Theorem: the necessity of (un)countably additivity I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ... 7 votes 0 answers 234 views ### Cox's Theorem: ignorance, objective priors, and the Mind Projection Fallacy I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ... 7 votes 0 answers 3k views ### Help with a proof of Bayes classifier optimality I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always {\le} any other classifier. I've never worked through a ... 7 votes 1 answer 390 views ### minimizer weighted linear regression In a regression problem, with y=X\theta+\epsilon and X is an n by p matrix the ‘weighted least squares estimate is the minimizer \theta^{*} of f(\theta)=\sum_{i=1}^{n}\omega_{i}(y_i-x_i^{'}\... 7 votes 0 answers 3k views ### Deriving the maximum likelihood for a generative classification model for K classes In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for K classes defined by the prior class ... 7 votes 0 answers 275 views ### Reference Request: Information Geometry for Ridge Regression I am reading the book "regression estimators" by Gruber 2010 where he uses this technique to compare Ridge Regressors, however he concentrates on deriving the mathematical results without ... 7 votes 0 answers 162 views ### Hypothesis test on the Euclidean length of an unknown vector Question Suppose I observe a vector \mathbf{x}=[X_1 \ldots X_n], where each X_i=m_i+n_i, with n_i being an independent zero-mean Gaussian random variable with variance \sigma^2 (i.e. n_i\sim\... 6 votes 0 answers 169 views ### Distribution that doesn't belong to any maximum domain of attraction? Question Does there exist a (non-degenerate) distribution that does NOT belong to any maximum domain of attraction? That is: Does there exist any non-degenerate probability distribution function F ... 6 votes 0 answers 80 views ### Symbolic Formulae for Linear Mixed Models I would like to understand how to create a good formula for a linear mixed model, using the Machines data set from the package ... 6 votes 0 answers 126 views ### Does Fisher scoring always outperform Newton optimization? My understanding is that Fisher scoring has several advantages over Newton raphson optimization such as Computational efficiency: if certain conditions are met (example:During MLE estimation, if link ... 6 votes 0 answers 268 views ### Why is X not an identifiable statistical model In my textbook, Identifiablity is defined as so: For any \theta_1, \theta_2 \in \Theta , if \theta_1 \neq \theta_2 \Rightarrow \Bbb P_{\theta_1} \neq \Bbb P_{\theta_2} , where \Bbb P_{\theta} ... 6 votes 2 answers 107 views ### How to win this dice probability game? The game is a variation of Pig. Here is how the game works: There are about 20 players. Each round, a single six sided die is rolled. All players add that rolled number to their "bank." However, if a ... 6 votes 0 answers 517 views ### Why we really need the concept of "Local" Rademacher complexity? Recently, I have been studying High-Dimensional Statistics: A Non-Asymptotic Viewpoint written by Martin J. Wainwright. In this book, the author uses a special complexity measure which is called Local ... 5 votes 0 answers 59 views ### Nuisance parameters and o_p(n^{-1/4}) convergence: citation I'm looking for an original reference to a proof idea. Suppose we have n iid observations (X_i,Y_i) and an estimating function$$\bar U(\beta;\alpha)=\frac{1}{n}U(\beta;\alpha; X_i,Y_i)$$where we ... 5 votes 0 answers 293 views ### Is an inadmissible estimator necessarily dominated by some admissible estimator Basic example: X has a p-variate iid standard Normal distribution; the sample mean is not admissible if p>2 and is dominated by the Stein shrinkage estimator. However, the Stein shrinkage ... 5 votes 1 answer 2k views ### How do I get the p value of AD test using the results of scipy.stats.anderson() I am conducting an Anderson Darling test for normality using scipy.stats.anderson() command in python. I am getting test statistic, critical values at various ... 5 votes 0 answers 496 views ### Why can't the complete class theorem be easily generalized to all locally-compact spaces? So I was reading Christian P. Robert's The Bayesian Choice, going through the constellation of results related to complete class theorems, and I don't see why all of them are necessary. In particular, ... 5 votes 0 answers 265 views ### Covariance of order statistics convergence? Suppose I have a sample (X_1 \dots X_n) and (Y_1 \dots Y_n), all of which are N(0,1) random variables. I am interested in the asymptotic behaviour of$$\frac{1}{n} \sum_{i=0}^n X_{(i)}Y_{(i)} $$... 5 votes 0 answers 260 views ### Combine absolute and relative difference into one metric Imagine we're looking to buy some commodity, say a table, and we want to make a good deal. We know the "true" price of the tables at offer and we're interested in both the absolute price difference ... 5 votes 0 answers 2k views ### Calculating a confidence interval for a weighted sample In a nutshell, I'd like to compute a confidence interval for some weighted sample day where the final value I'm seeking is a sum of different weighted samples. Illustrated here with a toy example set ... 5 votes 0 answers 176 views ### Finding the Cauchy Principal Value Mean of the pdf for Z=XY where \mathbb{E}[Y] does not exist Let's first define the Cauchy Principal Value Mean (PVM). For a continuous random variable V with pdf f_{V}(v), the PVM of f_{V} is \begin{equation} \mathrm{PV}(\mathbb{E}[V])=\lim_{a\to\infty}\... 5 votes 0 answers 310 views ### Variance of quotient of Poisson random variable and sum of the Poisson sample Let$$Y_1\sim \operatorname{Poisson}(\lambda_1)\\Y_2\sim \operatorname{Poisson}(\lambda_2),$$where Y_1 and Y_2 are independent, and \lambda_1, \lambda_2>0. What is the variance of$$\frac{... 5 votes 0 answers 863 views ### Under what conditions will a Bayesian posterior fail to converge to a point mass? Let's say you have a Bayesian model: $$\theta' \sim g(\theta|\mu)$$ $$y \sim p(y|\theta')$$ And we have some data ($n$data points)$\mathbf{y}_n$, which we will use to perform inference on$\... 64 views

### Why is the $\chi^{2}$ approximation for deviance GLM $\sim \operatorname{Binomial}(n_{i},\pi_{i})$ not valid when $n_{i} = 1$?

I know from McCullagh & Nelder's text (p.118) that the $\chi^{2}$ approximation for deviance for the binomial family is based on a limiting operation in which $n$, the number of observations, is ...
369 views

### Calculating Formula with logs and Weighted Average

I am trying to figure out an equation from the following paper by Cadena and Kovak (2016): http://pubs.aeaweb.org/doi/pdfplus/10.1257/app.20140095 on pages 264 and 265. I don't think the context is ...
275 views

### I'm not asking for a conjugate prior. Is there a distribution $p(x|y)$ that satisfies $\int p(x|y)Beta(y|a,b) dy = Beta(x| a', b')$?

I know the result of integrating a Gaussian against another Gaussian is still Gaussian, $$\int N(x|\mu_y,\sigma_y)N(y|\mu,\sigma) dy = N(x|\mu',\sigma')\quad.$$ Can I get the same form for Beta ...
413 views

### Why low rank expansions can exploit the redundancy that exist between different feature channels and filters?

I read Jaderberg et al., 2014 paper about Speeding up Convolutional Neural Network with Low Rank Expansions. In the introduction, it is written in bold font: Our key insight is to exploit the ...