Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

1,543 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
0answers
66 views

A possible typo in the textbook?

On page 74 of Lehmann's Testing Statistical Hypothesis, the author writes Let $P_0$ and $P_1$ be probability distributions possessing densities $p_0$ and $p_1$ respectively w.r.t. a measure $\mu$ ...
4
votes
0answers
100 views

Maximizing a computationally expensive function

Let $f:[0,1]^{80} \rightarrow [0,1]$ be some function, and say I have a computationally expensive way to calculate $f(x)$ for each $x \in [0,1]^{80}$ (expensive = 40s per query). The goal is to ...
4
votes
2answers
2k views

Confused about the realizability assumption and equations of upper bound

I'm reading the the first chapter of Understanding machine learning from theory to algorithms and they said that: Let be the set of "bad" hypotheses, that is (e is the accuracy ...
4
votes
0answers
122 views

Finding the Cauchy Principal Value Mean of the pdf for $Z=XY$ where $\mathbb{E}[Y]$ does not exist

Let's first define the Cauchy Principal Value Mean (PVM). For a continuous random variable $V$ with pdf $f_{V}(v)$, the PVM of $f_{V}$ is \begin{equation} \mathrm{PV}(\mathbb{E}[V])=\lim_{a\to\infty}\...
4
votes
0answers
487 views

Under what conditions will a Bayesian posterior fail to converge to a point mass?

Let's say you have a Bayesian model: $$\theta' \sim g(\theta|\mu) $$ $$ y \sim p(y|\theta')$$ And we have some data ($n$ data points) $\mathbf{y}_n$, which we will use to perform inference on $\...
4
votes
0answers
57 views

Why is the $\chi^{2}$ approximation for deviance GLM $\sim \operatorname{Binomial}(n_{i},\pi_{i})$ not valid when $n_{i} = 1$?

I know from McCullagh & Nelder's text (p.118) that the $\chi^{2}$ approximation for deviance for the binomial family is based on a limiting operation in which $n$, the number of observations, is ...
4
votes
1answer
51 views

Need help to understand the log-likelihood annotation?

Trying to know the steps to find the maximum likelihood estimate for the covariance matrix, assuming normal probability distribution, I want to differentiate log-likelihood function but what confuses ...
4
votes
0answers
164 views

Calculating Formula with logs and Weighted Average

I am trying to figure out an equation from the following paper by Cadena and Kovak (2016): http://pubs.aeaweb.org/doi/pdfplus/10.1257/app.20140095 on pages 264 and 265. I don't think the context is ...
4
votes
0answers
73 views

Hausdorff (fractal) Dimension of a Stochastic Process

It is well known that Brownian motion (BM) has a Hausdorff dimension (ie fractal dimension) of 2, for topological dimension >= 2. In other words, BM always "behaves like" a plane surface, no matter ...
4
votes
0answers
46 views

Data Transformation Question - Multiplying data proportional to demographics

I have a bunch of data that is tied to demographic variables (Age, Sex, Income, Education, etc.). However, the data is sent by one person in a household for the entire house. It's numerical data and I ...
4
votes
1answer
61 views

EM algorithm increase after E step?

It might be a silly question, but here it goes. The short version of my question is whether the marginal likelihood calculated after every E steps should be increasing or not. More details: Using ...
4
votes
0answers
171 views

Rigorous theory behind overfitting

I am taking an intro to ML class, and in my limited experience, training ML algorithms (validation, overfitting etc.) feels a bit like black magic. For instance, you aren't supposed to touch the test ...
4
votes
0answers
222 views

Find pdf of a multivariate function with log-normal distributions

Consider the following function: $f(x,t) = \dfrac{1}{2} \dfrac{1}{\sqrt{\pi(d_0+\alpha v) t}}\exp\left(-\dfrac{x^2}{4(d_0+\alpha v)t}-\lambda t\right)$ where the parameters $d_0,\alpha,\lambda$ are ...
4
votes
1answer
79 views

Distribution of ratio of 2 points drawn from normal distribution?

Let's say we have a known normal distribution $N(\mu,\sigma^2)$. I now draw 2 points $p1$ and $p2$ randomly from this Gaussian distribution for every observation, and repeat this process large number ...
4
votes
0answers
761 views

Kurtosis, bias, unbiased and statistics

I apologize ahead of time if this is too vague or meta to be a valid question. I've been looking at Algorithms (Sedgewick & Wayne). They define a class stdstats. In that they define min, max, ...
4
votes
0answers
92 views

A Fisher information metric which doesn't refer to any exponential family

If $\mathcal E$ is some exponential family of distributions, then we can view it as a Riemannian manifold with local metric the Fisher information matrix. We can then define the Fisher metric: the ...
4
votes
0answers
2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
4
votes
0answers
298 views

What properties of a likelihood function are required for quasi-likelihood estimation?

Quasi-likelihood seems like a great way to use Iteratively Weighted Least Squares to fit linear models with a very general class of likelihoods. But what is that class? Obviously the distribution ...
4
votes
0answers
239 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Theoremm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
4
votes
0answers
308 views

The distribution of the scalar product of multivariate complex normal vectors

Assume two independent random complex vectors (with real and imaginary parts): $$ \vec{\dot{Z}}=(\dot{z}_1,\dot{z}_2,\ldots,\dot{z}_n)~~~~~~~~~~ \begin{cases} \Re(\dot{z}_i)&=x_i \\ \Im(\dot{z}_i)...
4
votes
0answers
287 views

Likelihood ratio test: $f(x)=2x$ vs $f(x)=3x^2$: $2n$ degrees of freedom?

Suppose $X_1, . . . , X_n$ are i.i.d. with pdf $f(·)$. We want to test the hypotheses \begin{align} H_0 &: f(x) = 2x , \;\, \text{ for } 0 \le x \le 1, \text{ against}, \\ H_1 &: f(x) = 3x^2 , ...
4
votes
0answers
300 views

Gibbs sampling from full conditionals

I have the following joint density: $p(x_1,x_2,y_1,y_2) \propto \exp\left(−\left(x_1^2+x_2^2+c_1(y_2-y_1)^2+c_2(y_2-y_1)^4\right)\right)$ Can I use Gibbs sampling to sample from that? How can I get ...
4
votes
0answers
116 views

Smooth expectations outside the exponential family

At page 85-86 of Young and Smith "Essentials of Statistical Inference" there is an interesting result. If $X$ is a r.v. distributed according to the exponential family and $\phi(x)$ is a bounded (but ...
4
votes
2answers
1k views

Backpropagation proof and usage confusion

I've been taking Andrew Ng's course on Coursera, and although it has been great so far, I loathe his lack of supplementary documents on proofs. Thankfully, there are some great articles found pretty ...
4
votes
1answer
76 views

Is it possible to have $H_0: \theta\neq \theta_0$ (bilateral hypothesis)?

Can we ever have $H_0: \theta\neq \theta_0$ (bilateral hypothesis)? Are there any theorems that show some sort of most powerful test for this case? And what about $H_0: \theta<\theta_0$?
4
votes
1answer
53 views

Help on statistical modeling of pedestrian flow in subways

I'm a New Yorker and take the subways every day. I have a growing interest in understanding the distribution of paths people take on the subways to work every day. I.e. if there are $n$ subway ...
4
votes
1answer
550 views

Power calculation for cluster-level analysis in cluster randomized trials

I would like to solve for $\pi_1$ in equation 7.14 of Hayes and Moulton's Cluster Randomized Trials. I can't for the life of me remember how to do so. Here is a link to the equation. $$ c = 2\;+\;(z_{\...
4
votes
1answer
202 views

Equivalent definitions of a sufficient statistic

I'm trying to understand the definition of a sufficient statistic for continuous random variables given in Introduction to Mathematical Statistics by Hogg and Craig (7th edition). Let $X_1,X_2,...,...
3
votes
0answers
57 views

Generalized univariate normal distribution with $k+1$ parameters

Final update on 11/28/2019: I have worked on this a bit more, and wrote an article summarizing all the main findings. You can read it here. The goal here is to obtain a highly generic family of ...
3
votes
0answers
30 views

Why are these 2 ARIMA formulations equivalent?

In the "Understanding constants in R" section of his book, Hyndman & Athanasopoulos textbook "Forecasting: Principles and Practice" claims that the following AR processes equations are equivalent: ...
3
votes
1answer
70 views

Finding conditional probability of an individual component of a joint distribution

Assume $Y_1$, $Y_2$, $\ldots$ ,$Y_n$ are random variables over a regular lattice indexed by $i= 1,2,\ldots,n$ where $Y_i\in\{1,2,...,K\}$. Let the probability of a particular configuration $\...
3
votes
0answers
60 views

Does this statistical distance have a name and does the triangle inequality hold?

Let $P$ and $Q$ be two distributions on $\{1,2,\dots,n\}$. Define their distance by $$d(P, Q)=\Pr_{X\sim P,Y\sim Q}[P(X)Q(Y)>P(Y)Q(X)]\,,$$ where $X$ and $Y$ are independent. I could show that $d(...
3
votes
1answer
56 views

Sufficient statistic for Gaussian $AR(1)$

Question Does the Gaussian $AR(1)$ model, with a fixed sample size $T$, have nontrivial sufficient statistics? The model is given by $$ y_t = \rho y_{t-1}, \, t = 1, \cdots, T, \; \epsilon_i \...
3
votes
1answer
38 views

solve an exercise of two samples using Kolmogorov-smirnov

I'm looking for books and information like crazy and I can not find what I need. Well the example proposed is about methods that have been used in literature students and these are the data collected: ...
3
votes
0answers
24 views

Reconstruction from statistics

imagine to have data like the following $\begin{matrix}X1 & X2 & X3 \\\ 1 & 2 & 3\\\ 4 & 5 & 6\\\ 7 & 8 & 9\\\ \dots\end{matrix}$ where each row is a multivariate ...
3
votes
0answers
85 views

The difference of normal means is also minimax?

Let $X_i \sim N(\xi, \sigma^2)$ and $Y_i \sim N(\eta, \tau^2)$ for known $\sigma^2$ and $\tau^2$. I know that $\bar{X}$ and $\bar{Y}$ are minimax under squared error loss since their variance is ...
3
votes
0answers
52 views

What are the main approaches to the foundation of statistics without probability

The frequentist, likelihood and, to an even greater extent, Bayesian approaches to statistics are all based on probability. Without probability, it seems difficult to use a data sample ("seen" cases), ...
3
votes
0answers
61 views

Gaussian process and its limitations

I once saw the following statement on Gaussian process, ...
3
votes
3answers
253 views

Compare two samples with many zeros

We carried out a number of some experiments and got 10 independent 2-samples datasets. Is it possible to show a significant difference between the two samples, if each of them contains more than 75% ...
3
votes
0answers
37 views

Require understanding regarding the concept of restricted estimators

I was reading "The Elements of Statistical Learning Book by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie" where I encountered the following: The part tells us that the RSS criterion will ...
3
votes
1answer
75 views

How to call this frequentist interval estimate that is neither a prediction interval nor a confidence interval

This question is inspired by Confidence Interval on a random quantity?. That question introduces an interesting concept for a type of interval that is neither a prediction nor a confidence interval (...
3
votes
0answers
60 views

Incorrect computation in Knight and Fu (2000)?

I'm currently reading Knight and Fu's 2000 paper on the asymptotics of "Bridge" estimators with a particular focus on LASSO as a special case. In the proof of theorem 2, they make the claim that under ...
3
votes
0answers
48 views

What is the intuition behind taking the sum of square roots, squared

In a recent publication, the authors report the following transformation when aggregating across three different scales: Cognitive style level was used as a control variable and captured as ...
3
votes
0answers
108 views

Check Computation of MME and MLE

Let $X_1$, . . . , $X_n$ be i.i.d random variables having pdf $$f(x\mid\theta) = (\theta+ 1)x^{\theta}I_{(0,1)}(x)$$ where $\theta \gt−1$ (a) Give a MME of $\theta$ based on the first ...
3
votes
0answers
81 views

Optimization textbooks for statistics and data analytics

Any statistical analysis, machine learning or data science involves some sort of optimization at the end of the day. I'm looking for good linear and nonlinear optimization textbooks for self ...
3
votes
0answers
35 views

Regression coefficient estimate bounded as a function of the error covariance

Consider the linear regression model with one non-stochastic predictor: $Y = x \beta + \varepsilon$, where $Y \in \mathbb R^n$, $x \in \mathbb R^n$, $\beta \in \mathbb R$, and $\varepsilon \sim \...
3
votes
0answers
103 views

A theoretical explanation why ridge is superior to lasso in non-sparse models

There are several posts about the comparison of lasso vs. ridge. However I didn't find an explanation to my question. My question is why ridge is generating lower prediction errors in cases where the ...
3
votes
0answers
224 views

How to calculate the output of ns() function in R by hand

ns() function in R can generate the natural cubic spline basis matrix. I checked the reference but still do not know how to calculate such matrix by hand. For ...
3
votes
0answers
34 views

How to prove oracle properties in Fan and Li (2001) paper

I am studying Fan and Li's 2001 paper "Variable selection via nonconcave penalized Likelihood andits oracle properties" but I am having troubles understanding Theorem 1 proof (page 1359). I follow the ...
3
votes
0answers
356 views

Distribution of quadratic form of multivariate normal with linear term

Suppose that $A$ is a symmetric non-random matrix and $X\sim N(\mu,\Sigma)$ and $b \in R^n$ is a non-random vector. Then what is the distribution of $$X^tAX+b^tX \quad ?$$ The distribution without ...