Questions tagged [intuition]

Questions that seek a conceptual or non-mathematical understanding of statistics.

Filter by
Sorted by
Tagged with
284 votes
15 answers

What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It ...
  • 4,206
1262 votes
27 answers

Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...
  • 12.9k
326 votes
13 answers

How to understand degrees of freedom?

From Wikipedia, there are three interpretations of the degrees of freedom of a statistic: In statistics, the number of degrees of freedom is the number of values in the final calculation of a ...
  • 18.3k
96 votes
10 answers

What is meant by a "random variable"?

What do they mean when they say "random variable"?
  • 2,108
122 votes
14 answers

Maximum Likelihood Estimation (MLE) in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
  • 1,579
61 votes
3 answers

What is the intuition behind conditional Gaussian distributions?

Suppose that $\mathbf{X} \sim N_{2}(\mathbf{\mu}, \mathbf{\Sigma})$. Then the conditional distribution of $X_1$ given that $X_2 = x_2$ is multivariate normally distributed with mean: $$ E[P(X_1 | ...
  • 611
190 votes
8 answers

What intuitive explanation is there for the central limit theorem?

In several different contexts we invoke the central limit theorem to justify whatever statistical method we want to adopt (e.g., approximate the binomial distribution by a normal distribution). I ...
user avatar
625 votes
12 answers

What is the difference between "likelihood" and "probability"?

The wikipedia page claims that likelihood and probability are distinct concepts. In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a ...
164 votes
9 answers

Bottom to top explanation of the Mahalanobis distance?

I'm studying pattern recognition and statistics and almost every book I open on the subject I bump into the concept of Mahalanobis distance. The books give sort of intuitive explanations, but still ...
  • 5,327
541 votes
15 answers

What is the intuition behind beta distribution?

Disclaimer: I'm not a statistician but a software engineer. Most of my knowledge in statistics comes from self-education, thus I still have many gaps in understanding concepts that may seem trivial ...
  • 9,570
274 votes
10 answers

How would you explain covariance to someone who understands only the mean?

...assuming that I'm able to augment their knowledge about variance in an intuitive fashion ( Understanding "variance" intuitively ) or by saying: It's the average distance of the data ...
  • 13.9k
218 votes
17 answers

Intuitive explanation for dividing by $n-1$ when calculating standard deviation?

I was asked today in class why you divide the sum of square error by $n-1$ instead of with $n$, when calculating the standard deviation. I said I am not going to answer it in class (since I didn't ...
  • 20.4k
78 votes
5 answers

Intuition on the Kullback–Leibler (KL) Divergence

I have learned about the intuition behind the KL Divergence as how much a model distribution function differs from the theoretical/true distribution of the data. The source I am reading goes on to say ...
  • 8,155
399 votes
11 answers

Explaining to laypeople why bootstrapping works

I recently used bootstrapping to estimate confidence intervals for a project. Someone who doesn't know much about statistics recently asked me to explain why bootstrapping works, i.e., why is it that ...
  • 4,969
47 votes
3 answers

Intuitive explanation for density of transformed variable?

Suppose $X$ is a random variable with pdf $f_X(x)$. Then the random variable $Y=X^2$ has the pdf $$f_Y(y)=\begin{cases}\frac{1}{2\sqrt{y}}\left(f_X(\sqrt{y})+f_X(-\sqrt{y})\right) & y \ge 0 \\ 0 ...
  • 2,087
87 votes
5 answers

Cross-Validation in plain english?

How would you describe cross-validation to someone without a data analysis background?
  • 12.1k
30 votes
4 answers

Statistical interpretation of Maximum Entropy Distribution

I have used the principle of maximum entropy to justify the use of several distributions in various settings; however, I have yet to be able to formulate a statistical, as opposed to information-...
  • 301
61 votes
7 answers

Intuitive explanation of the bias-variance tradeoff?

I am looking for an intuitive explanation of the bias-variance tradeoff, both in general and specifically in the context of linear regression.
  • 5,421
53 votes
6 answers

Debunking wrong CLT statement

The central limit theorem (CLT) gives some nice properties about converging to a normal distribution. Prior to studying statistics formally, I was under the extremely wrong impression that the CLT ...
  • 35.7k
46 votes
4 answers

What is the difference between finite and infinite variance

What is the difference between finite and infinite variance ? My stats knowledge is rather basic; Wikipedia / Google wasn't much help here.
109 votes
4 answers

Difference between standard error and standard deviation

I'm struggling to understand the difference between the standard error and the standard deviation. How are they different and why do you need to measure the standard error?
  • 1,243
43 votes
2 answers

Can you explain Parzen window (kernel) density estimation in layman's terms?

Parzen window density estimation is described as $$ p(x)=\frac{1}{n}\sum_{i=1}^{n} \frac{1}{h^2} \phi \left(\frac{x_i - x}{h} \right) $$ where $n$ is number of elements in the vector, $x$ is a ...
  • 1,528
59 votes
3 answers

Why does shrinkage work?

In order to solve problems of model selection, a number of methods (LASSO, ridge regression, etc.) will shrink the coefficients of predictor variables towards zero. I am looking for an intuitive ...
109 votes
17 answers

What is the role of the logarithm in Shannon's entropy?

Shannon's entropy is the negative of the sum of the probabilities of each outcome multiplied by the logarithm of probabilities for each outcome. What purpose does the logarithm serve in this equation? ...
  • 2,503
84 votes
3 answers

What is the intuition behind SVD?

I have read about singular value decomposition (SVD). In almost all textbooks it is mentioned that it factorizes the matrix into three matrices with given specification. But what is the intuition ...
48 votes
3 answers

What kind of information is Fisher information?

Suppose we have a random variable $X \sim f(x|\theta)$. If $\theta_0$ were the true parameter, the the likelihood function should be maximized and the derivative equal to zero. This is the basic ...
121 votes
3 answers

Intuitive explanation of unit root

How would you explain intuitively what is a unit root, in the context of the unit root test? I'm thinking in ways of explaining much like I've founded in this question. The case with unit root is ...
  • 2,012
267 votes
11 answers

How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?

Maybe the concept, why it's used, and an example.
186 votes
4 answers

Why do we need sigma-algebras to define probability spaces?

We have a random experiment with different outcomes forming the sample space $\Omega,$ on which we look with interest at certain patterns, called events $\mathscr{F}.$ Sigma-algebras (or sigma-fields) ...
103 votes
9 answers

Is there an intuitive explanation why multicollinearity is a problem in linear regression?

The wiki discusses the problems that arise when multicollinearity is an issue in linear regression. The basic problem is multicollinearity results in unstable parameter estimates which makes it very ...
user avatar
76 votes
6 answers

What is an intuitive explanation for how PCA turns from a geometric problem (with distances) to a linear algebra problem (with eigenvectors)?

I've read a lot about PCA, including various tutorials and questions (such as this one, this one, this one, and this one). The geometric problem that PCA is trying to optimize is clear to me: PCA ...
27 votes
1 answer

Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
  • 33.9k
31 votes
2 answers

What is the intuition behind defining completeness in a statistic as being impossible to form an unbiased estimator of $0$ from it?

In classical statistics, there is a definition that a statistic $T$ of a set of data $y_1, \ldots, y_n$ is defined to be complete for a parameter $\theta$ it is impossible to form an unbiased ...
  • 2,225
41 votes
15 answers

The Monty Hall Problem - where does our intuition fail us?

From Wikipedia : Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who ...
33 votes
2 answers

What is the distribution of $R^2$ in linear regression under the null hypothesis? Why is its mode not at zero when $k>3$?

What is the distribution of the coefficient of determination, or R squared, $R^2$, in linear univariate multiple regression under the null hypothesis $H_0:\beta=0$? How does it depend on the number ...
  • 96.4k
81 votes
4 answers

What's so 'moment' about 'moments' of a probability distribution?

I KNOW what moments are and how to calculate them and how to use the moment generating function for getting higher order moments. Yes, I know the math. Now that I need to get my statistics knowledge ...
  • 13.9k
62 votes
1 answer

Can someone explain the concept of 'exchangeability'?

I see the concept of 'exchangeability' being used in different contexts (e.g., bayesian models) but I have never understood the term very well. What does this concept mean? Under what circumstances ...
  • 815
53 votes
4 answers

Why does the correlation coefficient between X and X-Y random variables tend to be 0.7

Taken from Practical Statistics for Medical Research where Douglas Altman writes in page 285: ...for any two quantities X and Y, X will be correlated with X-Y. Indeed, even if X and Y are ...
  • 1,357
26 votes
5 answers

Seeking certain type of ARIMA explanation

This may be hard to find, but I'd like to read a well-explained ARIMA example that uses minimal math extends the discussion beyond building a model into using that model to forecast specific cases ...
  • 11.7k
31 votes
2 answers

Understanding distance correlation computations

As far as I understood, distance correlation is a robust and universal way to check if there is a relation between two numeric variables. For example, if we have a set of pairs of numbers: ...
  • 365
18 votes
2 answers

Intuition behind Box-Cox transform

For features that are heavily skewed, the Transformation technique is useful to stabilize variance, make the data more normal distribution-like, improve the validity of measures of association. I am ...
  • 2,491
15 votes
1 answer

Odds made simple

I am having some trouble in understanding odds, and I would like just a basic explanation for how to interpret them. I have found various posts related to odds but most of them are more complex than ...
  • 151
127 votes
9 answers

Numerical example to understand Expectation-Maximization

I am trying to get a good grasp on the EM algorithm, to be able to implement and use it. I spent a full day reading the theory and a paper where EM is used to track an aircraft using the position ...
  • 2,503
18 votes
3 answers

Idea and intuition behind quasi maximum likelihood estimation (QMLE)

Question(s): What is the idea and intuition behind quasi maximum likelihood estimation (QMLE; also known as pseudo maximum likelihood estimation, PMLE)? What makes the estimator work when the actual ...
7 votes
1 answer

Choosing the number of bootstrap resamples

Say we have the following sample and we are trying to estimate the variance of the sample mean of the population. X = [0, -1, 2, 10, -3] If I take an increasing ...
  • 3,598
27 votes
3 answers

Why is the CDF of a sample uniformly distributed

I read here that given a sample $ X_1,X_2,...,X_n $ from a continuous distribution with cdf $ F_X $, the sample corresponding to $ U_i = F_X(X_i) $ follows a standard uniform distribution. I have ...
17 votes
2 answers

Kullback-Leibler Divergence for two samples

I tried to implement a numerical estimate of the Kullback-Leibler Divergence for two samples. To debug the implementation draw the samples from two normal distributions $\mathcal N (0,1)$ and $\...
  • 305
11 votes
2 answers

What are the consequences of "copying" a data set for OLS?

Suppose I have a random sample $\lbrace X_i, Y_i\rbrace_{i=1}^n$. Assume this sample is such that the Gauss-Markov assumptions are satisfied such that I can construct an OLS estimator where $$\hat{\...
42 votes
6 answers

Intuitive explanation of convergence in distribution and convergence in probability

What is the intuitive difference between a random variable converging in probability versus a random variable converging in distribution? I've read numerous definitions and mathematical equations, ...
  • 1,173
60 votes
13 answers

Does 10 heads in a row increase the chance of the next toss being a tail?

I assume the following is true: assuming a fair coin, getting 10 heads in a row whilst tossing a coin does not increase the chance of the next coin toss being a tail, no matter what amount of ...
  • 601

2 3 4 5 6