# Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

4,680 questions
Filter by
Sorted by
Tagged with
83k views

### Bottom to top explanation of the Mahalanobis distance?

I'm studying pattern recognition and statistics and almost every book I open on the subject I bump into the concept of Mahalanobis distance. The books give sort of intuitive explanations, but still ...
709k views

### What's the difference between variance and standard deviation?

I was wondering what the difference between the variance and the standard deviation is. If you calculate the two values, it is clear that you get the standard deviation out of the variance, but what ...
83k views

### Numerical example to understand Expectation-Maximization

I am trying to get a good grasp on the EM algorithm, to be able to implement and use it. I spent a full day reading the theory and a paper where EM is used to track an aircraft using the position ...
77k views

### What's the difference between probability and statistics?

What's the difference between probability and statistics, and why are they studied together?
69k views

### Why does the Cauchy distribution have no mean?

From the distribution density function we could identify a mean (=0) for Cauchy distribution just like the graph below shows. But why do we say Cauchy distribution has no mean?
10k views

### Who Are The Bayesians?

As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
55k views

### Maximum Likelihood Estimation (MLE) in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
64k views

### Simple algorithm for online outlier detection of a generic time series

I am working with a large amount of time series. These time series are basically network measurements coming every 10 minutes, and some of them are periodic (i.e. the bandwidth), while some other aren'...
28k views

### If mean is so sensitive, why use it in the first place?

It is a known fact that median is resistant to outliers. If that is the case, when and why would we use the mean in the first place? One thing I can think of perhaps is to understand the presence of ...
11k views

### What is meant by a “random variable”?

What do they mean when they say "random variable"?
34k views

### How exactly did statisticians agree to using (n-1) as the unbiased estimator for population variance without simulation?

The formula for computing variance has $(n-1)$ in the denominator: $s^2 = \frac{\sum_{i=1}^N (x_i - \bar{x})^2}{n-1}$ I've always wondered why. However, reading and watching a few good videos about "...
5k views

### Bayesians: slaves of the likelihood function?

In his book "All of Statistics", Prof. Larry Wasserman presents the following Example (11.10, page 188). Suppose that we have a density $f$ such that $f(x)=c\,g(x)$, where $g$ is a known (nonnegative, ...
8k views

### Why would parametric statistics ever be preferred over nonparametric?

Can someone explain to me why would anyone choose a parametric over a nonparametric statistical method for hypothesis testing or regression analysis? In my mind, it's like going for rafting and ...
31k views

### How does the reparameterization trick for VAEs work and why is it important?

How does the reparameterization trick for variational autoencoders (VAE) work? Is there an intuitive and easy explanation without simplifying the underlying math? And why do we need the 'trick'?
90k views

### Validation Error less than training error?

I found two questions here and here about this issue but there is no obvious answer or explanation yet.I enforce the same problem where the validation error is less than training error in my ...
4k views

### What are the breakthroughs in Statistics of the past 15 years?

I still remember the Annals of Statistics paper on Boosting by Friedman-Hastie-Tibshirani, and the comments on that same issues by other authors (including Freund and Schapire). At that time, clearly ...
13k views

### What is so cool about de Finetti's representation theorem?

From Theory of Statistics by Mark J. Schervish (page 12): Although DeFinetti's representation theorem 1.49 is central to motivating parametric models, it is not actually used in their ...
14k views

### Central limit theorem for sample medians

If I calculate the median of a sufficiently large number of observations drawn from the same distribution, does the central limit theorem state that the distribution of medians will approximate a ...
3k views

### Are we exaggerating importance of model assumption and evaluation in an era when analyses are often carried out by laymen

Bottom line, the more I learn about statistics, the less I trust published papers in my field; I simply believe that researchers are not doing their statistics well enough. I'm a layman, so to speak. ...
8k views

### Mathematical Statistics Videos

A question previously sought recommendations for textbooks on mathematical statistics Does anyone know of any good online video lectures on mathematical statistics? The closest that I've found are: ...
8k views

### What is the most surprising characterization of the Gaussian (normal) distribution?

A standardized Gaussian distribution on $\mathbb{R}$ can be defined by giving explicitly its density: $$\frac{1}{\sqrt{2\pi}}e^{-x^2/2}$$ or its characteristic function. As recalled in this ...
23k views

### Mutual information versus correlation

Why and when we should use Mutual Information over statistical correlation measurements such as "Pearson", "spearman", or "Kendall's tau" ?
39k views

### KL divergence between two multivariate Gaussians

I'm having trouble deriving the KL divergence formula assuming two multivariate normal distributions. I've done the univariate case fairly easily. However, it's been quite a while since I took math ...
11k views

### Motivation for Kolmogorov distance between distributions

There are many ways to measure how similar two probability distributions are. Among methods which are popular (in different circles) are: the Kolmogorov distance: the sup-distance between the ...
47k views

### Why is it that natural log changes are percentage changes? What is about logs that makes this so?

Can somebody explain how the properties of logs make it so you can do log linear regressions where the coefficients are interpreted as percentage changes?
7k views

### Taking the expectation of Taylor series (especially the remainder)

My question concerns trying to justify a widely-used method, namely taking the expected value of Taylor Series. Assume we have a random variable $X$ with positive mean $\mu$ and variance $\sigma^2$. ...
7k views

### Correlation does not imply causation; but what about when one of the variables is time?

I know this question has been asked a billion times, so, after looking online, I am fully convinced that correlation between 2 variables does not imply causation. In one of my stats lectures today, we ...
6k views

10k views

### Differences between Bhattacharyya distance and KL divergence

I'm looking for an intuitive explanation for the following questions: In statistics and information theory, what's the difference between Bhattacharyya distance and KL divergence, as measures of the ...
8k views

### Are there any examples of where the central limit theorem does not hold?

Wikipedia says - In probability theory, the central limit theorem (CLT) establishes that, in most situations, when independent random variables are added, their properly normalized sum tends ...
3k views

### Does a sample version of the one-sided Chebyshev inequality exist?

I am interested in the following one-sided Cantelli's version of the Chebyshev inequality: $$\mathbb P(X - \mathbb E (X) \geq t) \leq \frac{\mathrm{Var}(X)}{\mathrm{Var}(X) + t^2} \,.$$ Basically, ...
5k views

### If 'correlation doesn't imply causation', then if I find a statistically significant correlation, how can I prove the causality?

I understand that correlation is not causation. Suppose we get high correlation between two variables. How do you check if this correlation is actually because of causation? Or,under what conditions, ...
3k views

The likelihood could be defined by several ways, for instance : the function $L$ from $\Theta\times{\cal X}$ which maps $(\theta,x)$ to $L(\theta \mid x)$ i.e. $L:\Theta\times{\cal X} \rightarrow \... 1answer 21k views ### How are the standard errors computed for the fitted values from a logistic regression? When you predict a fitted value from a logistic regression model, how are standard errors computed? I mean for the fitted values, not for the coefficients (which involves Fishers information matrix). ... 1answer 12k views ### Differences between a statistical model and a probability model? Applied probability is an important branch in probability, including computational probability. Since statistics is using probability theory to construct models to deal with data, as my understanding, ... 1answer 12k views ### Maximum likelihood estimators for a truncated distribution Consider$N$independent samples$S$obtained from a random variable$X$that is assumed to follow a truncated distribution (e.g. a truncated normal distribution) of known (finite) minimum and maximum ... 3answers 2k views ### Gaussian Ratio Distribution: Derivatives wrt underlying$\mu$'s and$\sigma^2$s I'm working with two independent normal distributions$X$and$Y$, with means$\mu_x$and$\mu_y$and variances$\sigma^2_x$and$\sigma^2_y$. I'm interested in the distribution of their ratio$Z=X/...
The random walk that is defined as $Y_{t} = Y_{t-1} + e_t$, where $e_t$ is white noise. Denotes that the current position is the sum of the previous position + an unpredicted term. You can prove that ...