Questions tagged [intuition]
Questions that seek a conceptual or non-mathematical understanding of statistics.
755
questions
2
votes
1
answer
29
views
Intuitive explanation of conformal prediction
I have recently started learning about conformal prediction. I am a programmer without a strong mathematical background, but with a strong intuitive, applied background in statistics. I am trying to ...
- 1,848
0
votes
0
answers
26
views
What does the quality of representation of a variable mean in PCA?
I understand that the quality of representation of an individual by a certain axis is measured by the cosine of the angle between the axis and the individual; the more the vector representing the ...
- 208
2
votes
0
answers
39
views
Intuition behind rank of covariance matrix and testing hypotheses
I am trying to acquire some intuition about testing multivariate hypotheses where the test statistic involves inverse covariance matrix. As an example, suppose we have a $p$-variate random vector that ...
- 61.1k
3
votes
0
answers
40
views
Geometric intuition for how ridge ($L_2$) regularization helps under multicollinearity
We have some nice posts (1, 2 and likely more) illustrating multicollinearity geometrically. Now, ridge regression ($L_2$ regularization) is known to be a remedy of multicollinearity. What is the ...
- 61.1k
2
votes
0
answers
60
views
Understanding intuitive difference between KL divergence and Cross entropy
I know there are related questions already asked, for example this one.
I also know the following:
KL divergence $D_{KL}(P\Vert Q)$ is given as:
$$\begin{align}
D_{KL}(P\Vert Q) & = -\sum_xP(x)\...
- 215
0
votes
1
answer
46
views
Real-World Example of Correlation of Random Variables
I'm encountering a result in research that is counter-intuitive to me. Specifically, I have two matrics, $X, Y$, where $X_i$ is the ith column of matrix $X$.
In my research:
$\Large{\rho}$$ (\sum X_i, ...
2
votes
1
answer
82
views
Can you explain bootstrapping like I’m 5?
I think I have a handle on what bootstrapping is and why we need to use it.
Please confirm if my understanding is correct:
Goal of bootstrapping: To find the SE of a feature’s coefficient that you ...
- 513
1
vote
0
answers
107
views
Concrete example of what Sufficient Statistics is [closed]
Having read articles to try to understand Sufficient Statistics.
Sufficient statistics for layman
A sufficient statistic summarizes all the information contained in a sample so that you would make ...
- 1,186
0
votes
0
answers
54
views
Intuitive statistics book [duplicate]
I am looking for a statistics book that not only gives formulas or proofs but also gives intuitive explanations.
For example, the standard deviation is defined by
${\sigma_x} = \sqrt{\frac{1}{n}{\sum\...
5
votes
2
answers
748
views
Is variance the area under the curve of the distribution of a population?
I am trying to understand what variance is, I already know the "official" definition
"Variance is the average squared deviations from the mean"
But I am trying to give it a visual ...
- 240
0
votes
0
answers
71
views
Interpretation of AUC as probability of a random positive being ranked more highly than a random negative [duplicate]
In this article from Google they claim that
One way of interpreting AUC is as the probability that the model ranks a random positive example more highly than a random negative example.
along with an ...
- 55
3
votes
1
answer
99
views
Difference between likelihood functions for pmf vs pdf
Can someone explain the intuition behind how the likelihood function for a specific value of $\theta$ is different if $f_\theta$ is a pmf vs a pdf?
I thought that it was simply the probability that a ...
- 31
0
votes
0
answers
73
views
Is there a meaningful difference between a change described as a percentage increase one way versus a percentage decrease the other way?
When describing a change in value between two scenarios (A and B) I see two options: (1) report the percent increase/decrease from A to B, or (2) report the percent decrease/increase from B to A. ...
- 2,280
1
vote
0
answers
21
views
How to connect the intuitions to the math of adaptive processes?
Formal Definition
Wikipedia gives the following definition of a process adapted to a filtration:
Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space;
$I$ be an index set with total order $\...
- 6,135
3
votes
0
answers
41
views
Is there an outer product counterpart for the Covariance?
Covariance
The covariance of two quantities $X$ and $Y$ within a population, $Cov(X,Y)$, is symmetric and bilinear. It is also true that $Cov(X,X) \ge 0$. So, clearly $Cov(X,Y)$ qualifies as an inner ...
- 31
1
vote
1
answer
33
views
Linear models when responses have no link
I am studying normal linear regression and wanted to ask a question about its utility when working with independent RV. Suppose that we have for $k \in [1,\dots,n]$, $$Y_k = \beta_0 + \beta_1x_{k1} + \...
- 189
1
vote
0
answers
27
views
Interpretation of covariance and linear dependency [duplicate]
What is the best interpretation of covariance you can give ?
I know that if $X$ and $Y$ are random variables, then if $Cov(X,Y)>0$, then if realizations of $X$ are higher than expected, then ...
- 189
2
votes
1
answer
73
views
Mean of geometric distribution is odds?
Context: I mean the $P(X=k)=(1-p)^k p$ not the $P(Y=k)=(1-p)^{k-1} p$. Apparently the mean of the 1st kind of geometric is $\frac{1-p}{p}$ instead of $\frac{1}{p}$ for the 2nd kind of geometric. I ...
- 2,280
2
votes
1
answer
65
views
What does conditional independence mean semantically?
I've just spent the last 3 hours reading every post, question, Medium article, and textbook entry on conditional independence, and I still don't really understand it. Can somebody explain what it ...
- 237
0
votes
0
answers
37
views
Naive Bayes fits multiple hyperplanes in the case of multiclass classification problems?
Naive Bayes differentiates feature distributions given target labels, and intuitively, it fits a hyperplane to the given data set.
But I do not fully understand whether Naive Bayes fits multiple ...
0
votes
0
answers
131
views
What exactly does the Box-Cox transformation do to a time series?
If I were to try and rephrase the argument in the original Box-Cox paper in my own words, I would say something like the following: given a model
$$
y = x \beta ,
$$
if the residuals do not appear to ...
- 442
2
votes
1
answer
65
views
Why do we need a smaller sample size to detect a smaller proportion?
The plot below shows the sample size needed to detect a proportion with a precision 0.01 for various true proportions:
This assumes an infinite population size, and the confidence intervals are fixed ...
- 141
1
vote
0
answers
180
views
Why PCA is invariant under rotation
Lets say that we have a matrix of variables (the columns are variables and rows are the observations) called X whenre X = [x1, x2, ...., xp] where ...
- 133
1
vote
0
answers
36
views
Does the number of samples, as opposed to the sample size in each sample, matter for the Central Limit Theorem? [closed]
(1)So here is a formula that describes CLT I found at https://en.wikipedia.org/wiki/Central_limit_theorem.
According to the first part of the explanation, n as in Xn describes the number of samples(i....
- 11
1
vote
0
answers
79
views
Why is my intuition about probability in this regard so flawed? [closed]
Take the following example:
Take a sample from 100 people and measure their height. Assume that we know that height is approximately normally distributed, with a sample mean of 175 cm and sample ...
- 111
12
votes
5
answers
2k
views
What's complicated about regression to the mean?
Note: I am a bit of a novice when it comes to statistics and data analysis.
Reading the chapter on regression to the mean in Kahneman's Thinking Fast and Slow, I came across the following passage:
...
- 123
2
votes
0
answers
62
views
Intuition behind log in kl distance
So, let's start stating that I already read both Why KL-Divergence uses "ln" in its formula? and What is the role of the logarithm in Shannon's entropy? ... However, I still have no ...
1
vote
0
answers
132
views
Intuition about the relation between joint distribution, marginal distribution, and conditional distribution
The wording "intuition" might be a bit imprecise. I want to discuss how we visualize in our head going from one to another among the joint PDF, marginal PDF, and conditional PDF.
To make the ...
- 357
12
votes
5
answers
2k
views
Is the exact value of any likelihood meaningless?
While reading about likelihood, I have heard that "the exact value of any likelihood is meaningless" why?
So, because of that we may use the likelihood ratio.
So, my question is, why the ...
- 589
3
votes
1
answer
86
views
What is the intuition behind the odds scale?
What is an intuitive explanation of the odds scale?
In a logistic regression such as $$logit(p) = \beta_0 + \beta_1 x$$
we often interpret $\beta_1$ by looking at the odds ratio, $e^{\beta_1}$, which ...
- 337
1
vote
1
answer
159
views
MLE of the Uniform Distribution
In a uniform distribution where $0\leq X \leq \theta$, the pdf is represented as $f(X|\theta) = \frac{1}{\theta}I(0\leq X \leq \theta)$, and the likelihood is $L(\theta) = \prod\frac{1}{\theta}I(0\leq ...
- 61
4
votes
2
answers
474
views
Intuition for why mean of lognormal distribution depends on variance of normally distributed rv
Let $X\sim\mathcal{N}(\mu,\sigma^2)$, which is a normal distribution. Then, $\text{exp}(X)\sim\text{Lognormal}(\mu,\sigma^2)$, and its mean is
$$
\mathbb{E}[\text{exp}(X)]=\text{exp}\left(\mu+\dfrac{\...
- 143
0
votes
0
answers
28
views
Intuitive effect size (and CI) for one-way comparison of group means?
I'm a big fan of common language effect sizes (e.g. McGraw and Wong's CL or Cohen's U3 - "what proportion of group 1 are higher than the average for group 2"). I've been scrabbling around ...
- 775
1
vote
2
answers
96
views
In a diff-in-diff or regression discontinuity research design, why is it important to describe why the counterfactual is a plausible one?
I've heard it mentioned that in difference-in-differences, regression discontinuity, or even in some other quasi-experimental research designs, that the counterfactual should be explained as a ...
- 111
3
votes
1
answer
226
views
Why is each observation in a sample considered a random variable in linear regression?
I have the following excerpt in my statistics textbook:
I am confused by the sentence: "Another way statisticians treat this model is that, assume $X_1...X_n$ are random variables, we make ...
- 641
2
votes
2
answers
198
views
Why does $\underset{\theta}{\arg\max }p_{\text{model}}(X;{\theta}) =\underset{\theta}{\arg\max} \prod_{i=1}^{m}p_{\text{model}}({x}^{(i)} ;\theta)$?
We have seen some definitions of common estimators and analyzed their properties. But where did these estimators come from? Rather than guessing that some function might make a good estimator and then ...
1
vote
0
answers
107
views
Intuition for bandwidth and degrees of freedom in kernel smoothers
For Kernel smoothers such as local polynomial regression smoothers (the Nadaraya-Watson smoother), we consider $y = m(x) + \epsilon$, for $m(x)$ some smooth function we are trying to estimate and $\...
- 421
0
votes
0
answers
106
views
Can you explain LINEAR in BLUE?
I have hard time understanding the LINEAR part. Found something like this:
Linear property of OLS estimator means that OLS belongs to that class of estimators, which are linear in Y, the dependent ...
- 1
0
votes
2
answers
59
views
Estimated vs. true expected value in $\chi^2$ test of independence of two categorical variables
Consider a $\chi^2$ test of independence of two categorical variables. In the test statistic, we have elements of the form
$$
\frac{(\text{observed}_i-\text{expected}_i)^2}{\text{expected}_i},
$$
or $\...
- 61.1k
1
vote
0
answers
25
views
Intuition Maximum likelihood [duplicate]
Can someone describe in simple words what the Intuition behind maximum likelihood estimation is and why it is so commonly used in statistics? Certainly, there are many other ways to estimate ...
- 19
24
votes
7
answers
3k
views
Explain in layperson's terms why predictive models aren't causally interpretable
Imagine that you are asked to infer some causal effect -- a change in an outcome $y$ in response to some variable $x$. But, the person asking for this directs you to use a predictive model to do so. ...
- 12.8k
1
vote
1
answer
121
views
Two basic questions about icp (iterative closest point) algorithm
I am trying to learn shape analysis and a part is learning icp. I have many confusions but for now I have two basic questions:
Does the point clouds need to have the same number of points for icp?
...
- 43
0
votes
2
answers
335
views
In permutation test, why do we take the proportion of sampled permutations with value equal or larger than the observed value?
The tutorial I followed explains permutation testing in an intuitive way. However, it has confused me in one specific part. Why do we take as p-value the proportion/probability of permutation with ...
- 101
0
votes
0
answers
23
views
Intuition underyling kinds of time series that are typically additive or typically multiplicative with examples
What is the intuition underyling the kinds of time series that are typically additive or typically multiplicative? From what I understand, additive time series are such that variations on the trend ...
- 1,848
3
votes
0
answers
181
views
Why do we try to "Reproduce" Hilbert Spaces in Statistics?
I am trying to better understand why people are interested in "reproducing" Hilbert Spaces in Statistics and Machine Learning.
I (think) understand the general idea behind Hilbert Spaces. ...
- 7,038
1
vote
1
answer
134
views
Practical consequences of wrong interpretation of confidence intervals
Can you give me a simple (but preferably common) example (or R/Python simulation) of what are practical consequences of wrong interpretation of frequentist confidence intervals? Especially when they ...
11
votes
4
answers
2k
views
What is an intuitive interpretation for the softmax transformation?
A recent question on this site asked about the intuition of softmax regression. This has inspired me to ask a corresponding question about the intuitive meaning of the softmax transformation itself. ...
- 110k
1
vote
1
answer
70
views
Pairwise deletion and non-positive definite sample covariance matrix
So I have read that using pairwise deletion can lead to a non-positive definite matrix. However, I tried to find a source which showed this mathematically. Can someone point me to that?
My intuition ...
- 85
4
votes
1
answer
352
views
Intuition for Wilks' theorem
I'm trying to wrap my head around why it is intuitive that (under certain conditions) the likelihood ratio statistic follows a chi-squared distribution, asymptotically.
I've looked at the excellent ...
- 43
0
votes
0
answers
29
views
Different way to do PCA: overall comparison
Given a dataset PCA can be performed via 3 ways:
Eigenvalue decomposition
Singular value decomposition
Non-linear iterative partial least-squares algorithm
Can anyone shed light on comparative study ...
- 43