# Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

533 questions
Filter by
Sorted by
Tagged with
10k views

### What is meant by a “random variable”?

What do they mean when they say "random variable"?
54k views

### Maximum Likelihood Estimation (MLE) in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
82k views

### Bottom to top explanation of the Mahalanobis distance?

I'm studying pattern recognition and statistics and almost every book I open on the subject I bump into the concept of Mahalanobis distance. The books give sort of intuitive explanations, but still ...
6k views

### Consider the sum of $n$ uniform distributions on $[0,1]$, or $Z_n$. Why does the cusp in the PDF of $Z_n$ disappear for $n \geq 3$?

I've been wondering about this one for a while; I find it a little weird how abruptly it happens. Basically, why do we need just three uniforms for $Z_n$ to smooth out like it does? And why does the ...
13k views

### Central limit theorem for sample medians

If I calculate the median of a sufficiently large number of observations drawn from the same distribution, does the central limit theorem state that the distribution of medians will approximate a ...
63k views

### Simple algorithm for online outlier detection of a generic time series

I am working with a large amount of time series. These time series are basically network measurements coming every 10 minutes, and some of them are periodic (i.e. the bandwidth), while some other aren'...
6k views

919 views

### Estimating parameters for a binomial

First of all I'd like to precise that I'm not an expert of the subject. Suppose to have two random variables $X$ and $Y$ that are binomial, respectively $X\sim B(n_1,p)$ and $Y\sim B(n_2,p),$ note ...
82k views

### Numerical example to understand Expectation-Maximization

I am trying to get a good grasp on the EM algorithm, to be able to implement and use it. I spent a full day reading the theory and a paper where EM is used to track an aircraft using the position ...
32k views

### How exactly did statisticians agree to using (n-1) as the unbiased estimator for population variance without simulation?

The formula for computing variance has $(n-1)$ in the denominator: $s^2 = \frac{\sum_{i=1}^N (x_i - \bar{x})^2}{n-1}$ I've always wondered why. However, reading and watching a few good videos about "...
30k views

### What is the distribution of the sum of non i.i.d. gaussian variates?

If $X$ is distributed $N(\mu_X, \sigma^2_X)$, $Y$ is distributed $N(\mu_Y, \sigma^2_Y)$ and $Z = X + Y$, I know that $Z$ is distributed $N(\mu_X + \mu_Y, \sigma^2_X + \sigma^2_Y)$ if X and Y are ...
3k views

### Good resources (online or book) on the mathematical foundations of statistics

Before I ask my question, let me give you a bit of background about what I know about statistics so that you have a better sense of the types of resources that I'm looking for. I'm a graduate student ...
16k views

### Asymptotic distribution of sample variance of non-normal sample

This is a more general treatment of the issue posed by this question. After deriving the asymptotic distribution of the sample variance, we can apply the Delta method to arrive at the corresponding ...
4k views

### Moment generating function of the inner product of two gaussian random vectors

Can anybody please suggest how I can compute the moment generating function of the inner product of two Gaussian random vectors, each distributed as $\mathcal N(0,\sigma^2)$, independent of each other?...
13k views

### What is so cool about de Finetti's representation theorem?

From Theory of Statistics by Mark J. Schervish (page 12): Although DeFinetti's representation theorem 1.49 is central to motivating parametric models, it is not actually used in their ...
3k views

### What is the intuition behind defining completeness in a statistic as being impossible to form an unbiased estimator of $0$ from it?

In classical statistics, there is a definition that a statistic $T$ of a set of data $y_1, \ldots, y_n$ is defined to be complete for a parameter $\theta$ it is impossible to form an unbiased ...
6k views

### What is the distribution of $R^2$ in linear regression under the null hypothesis? Why is its mode not at zero when $k>3$?

What is the distribution of the coefficient of determination, or R squared, $R^2$, in linear univariate multiple regression under the null hypothesis $H_0:\beta=0$? How does it depend on the number ...
10k views

### Who Are The Bayesians?

As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
37k views

### KL divergence between two multivariate Gaussians

I'm having trouble deriving the KL divergence formula assuming two multivariate normal distributions. I've done the univariate case fairly easily. However, it's been quite a while since I took math ...
3k views

The likelihood could be defined by several ways, for instance : the function $L$ from $\Theta\times{\cal X}$ which maps $(\theta,x)$ to $L(\theta \mid x)$ i.e. $L:\Theta\times{\cal X} \rightarrow \... 3answers 16k views ### Empirical relationship between mean, median and mode For a unimodal distribution that is moderately skewed, we have the following empirical relationship between the mean, median and mode: $$\text{(Mean - Mode)}\sim 3\,\text{(Mean - Median)}$$ How ... 1answer 10k views ### Distribution of sum of squares error for linear regression? I know that distribution of sample variance $$\sum\frac{(X_i-\bar{X})^2}{\sigma^2}\sim \chi^2_{(n-1)}$$ $$\sum\frac{(X_i-\bar{X})^2}{n-1}\sim \frac{\sigma^2}{n-1}\chi^2_{(n-1)}$$ It's from the ... 2answers 616 views ### Is there an elegant/insightful way to understand this linear regression identity for multiple$R^2$? In linear regression I have come across a delightful result that if we fit the model $$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$ then, if we standardize and centre the$Y$,$X_1$and$X_2$data, $$R^... 2answers 5k views ### Transformation Chi-squared to Normal distribution The relationship between the standard normal and the chi-squared distributions is well known. I was wondering though, is there a transformation that can lead from a \chi^2 (1) back to a standard ... 1answer 20k views ### How are the standard errors computed for the fitted values from a logistic regression? When you predict a fitted value from a logistic regression model, how are standard errors computed? I mean for the fitted values, not for the coefficients (which involves Fishers information matrix). ... 1answer 11k views ### Maximum likelihood estimators for a truncated distribution Consider N independent samples S obtained from a random variable X that is assumed to follow a truncated distribution (e.g. a truncated normal distribution) of known (finite) minimum and maximum ... 3answers 29k views ### Prove F test is equal to T test squared I need to show that F test is equal to T test squared, when the T test is for 2 independent groups and assuming variances are equal. I know that F=\frac{MSB}{MSW}=\frac{SSB/k-1}{SSW/N-K} and I know ... 1answer 1k views ### What is meant by using a probability distribution to model the output data for a regression problem? Often a theoretical text will say something like, 'a probability distribution may be used to model the data' or, 'assume a probability distribution such as normal or Lognormal for the outputs'. ... 1answer 228 views ### Prove that E(X^n)^{1/n} is non-decreasing for non-negative random variables For a nonnegative random variable X, how to prove that E(X^n)^{\frac1n} is nondecreasing in n? 21answers 75k views ### What's the difference between probability and statistics? What's the difference between probability and statistics, and why are they studied together? 6answers 683k views ### What's the difference between variance and standard deviation? I was wondering what the difference between the variance and the standard deviation is. If you calculate the two values, it is clear that you get the standard deviation out of the variance, but what ... 5answers 4k views ### What is the mathematical difference between random- and fixed-effects? I have found a lot on the internet regarding the interpretation of random- and fixed-effects. However I could not get a source pinning down the following: What is the mathematical difference between ... 2answers 773 views ### Constructing a discrete r.v. having as support all the rationals in [0,1] This is the constructivist sequel of this question. If we can't have a discrete uniform random variable having as support all the rationals in the interval [0,1], then the next best thing is: ... 3answers 1k views ### Why does this excerpt say that unbiased estimation of standard deviation usually isn't relevant? I was reading on the computation of the unbiased estimation of standard deviation and the source I read stated (...) except in some important situations, the task has little relevance to ... 3answers 5k views ### Tool for generating correlated data sets Does anyone know of a tool that I can use to generate a set of data with known correlations (and to put the icing on the cake - output this in json,csv,txt or some common format)? I am working on ... 1answer 2k views ### Prove the relation between Mahalanobis distance and Leverage? I have seen formulas on Wikipedia. that relate Mahalanobis distance and Leverage: Mahalanobis distance is closely related to the leverage statistic, h, but has a different scale:$$D^2 = (N - 1)(... 1answer 1k views ### t-distribution confidence intervals for non-Gaussian data but large n I have a question concerning a claim I read in statistics books concerning the applicability of the t-distribution to compute confidence intervals for large$n$if the data is not normally distributed ... 7answers 82k views ### Validation Error less than training error? I found two questions here and here about this issue but there is no obvious answer or explanation yet.I enforce the same problem where the validation error is less than training error in my ... 7answers 27k views ### How does the reparameterization trick for VAEs work and why is it important? How does the reparameterization trick for variational autoencoders (VAE) work? Is there an intuitive and easy explanation without simplifying the underlying math? And why do we need the 'trick'? 14answers 8k views ### Why would parametric statistics ever be preferred over nonparametric? Can someone explain to me why would anyone choose a parametric over a nonparametric statistical method for hypothesis testing or regression analysis? In my mind, it's like going for rafting and ... 4answers 6k views ### Taking the expectation of Taylor series (especially the remainder) My question concerns trying to justify a widely-used method, namely taking the expected value of Taylor Series. Assume we have a random variable$X$with positive mean$\mu$and variance$\sigma^2\$. ...
I am interested in the following one-sided Cantelli's version of the Chebyshev inequality: $$\mathbb P(X - \mathbb E (X) \geq t) \leq \frac{\mathrm{Var}(X)}{\mathrm{Var}(X) + t^2} \,.$$ Basically, ...