Questions tagged [intuition]
Questions that seek a conceptual or non-mathematical understanding of statistics.
781
questions
1
vote
1
answer
44
views
Motivation behind definition of PMF of function of $2$ variables
I am really curious to understand what motivates the definition $$p_{g(X,Y)} (g(X,Y)=z) = \sum_{(x,y)\in g^{-1}(\{z\})} p_{X,Y} (x,y)$$
where $g$ is a two variable function, and $X,Y$ are random ...
1
vote
0
answers
35
views
Why intuitively is standard deviation the correct thing to scale to get Central Limit Theorem?
Let me start off by saying I already know all the rigorous formulas, but let me explain why I still feel like something is missing in my understanding. There is no need for any answer going over e.g. ...
0
votes
0
answers
32
views
Looking for an intuitive explanation of D-Criterion for Optimal Design Problem
I know only a little about Fisher information and optimal experimental design, but I'm trying to better understand the subject. If I have an experiment composed of a single detector and my detector ...
0
votes
0
answers
13
views
Question Intuition behind mathematics of activation function in a neural network.
Does this intuition behind why an activation function is used in a neural network make sense mathematically :
For this example lets consider a fully connected (NOT CONVOLUTIONAL) network that ...
19
votes
8
answers
2k
views
What is the intuition behind the idea that for linear regression, the number of observations should exceed the number of parameters?
If a population model has k independent variables and 1 intercept, why are k+1 observations required to perform OLS estimates?
What is the intuition behind this?
7
votes
1
answer
650
views
What is a *likelihood ratio test* for a specific distribution, and how does it relate to hypothesis tests?
I'm just now being introduced to likelihood-ratio tests (LRT), and I am having trouble following the concept and terminology.
For example, I posed a question about determining whether two samples {x} ...
7
votes
2
answers
299
views
Intuition and reasoning why LASSO can only select $n$ features when $n \ll p$
I'm struggling to grasp the intuition behind why LASSO can only select at most $n$ features when $n << p$, where $n$ is the number of samples and $p$ is the number of features.
I've read through ...
1
vote
1
answer
45
views
Is my interpretation of "the probability over data $X_1, ... X_N$ correct?
This may seem like a pretty simple question, but I want to make sure I am getting this right because it seems pretty foundational.
I'm reading this note on conformal prediction. In the very first ...
31
votes
13
answers
7k
views
How to explain 1.5 children?
I teach undergrad stats and every year one student asks "You can't have 1.5 children" (the mean for the dataset). I am flummoxed every time to create a sensical answer. I've tried: "no ...
3
votes
1
answer
113
views
How to explain unbiasedness in basic terms?
If I take some estimator X. Lets say that X is unbiased.
Suppose I have 100 samples and each sample has 5 points. I now calculate the value of X on each sample.
Because X is unbiased, this means that ...
2
votes
3
answers
130
views
Shouldn't we consider the difference in variance between population and a sample while calculating confidence intervals?
To comprehend the concept of confidence intervals, I came up with an example. I want to share it here for your better understanding what my question is all about.
Suppose, we want to figure out what ...
13
votes
1
answer
855
views
Intuitive explanation for the fat tails of the t-distribution
Given some standard assumptions, the test statistic
$$
\frac{\Delta\bar{X}}{\sigma/\sqrt{N}}
$$
is normally distributed if $\sigma$ is known and t-distributed if $\sigma$ has to be estimated from the ...
4
votes
1
answer
125
views
Markov's inequality intuitions
Can someone explain intuitively how Markov's inequality was derived? It seems plausible, but looking a it, I can't 'see' how it's true.
2
votes
0
answers
22
views
Intuition behind testing seasonality hypothesis
In this post to prove the statistical significance of a statement about a seasonality of a timeseries (every april returns are high) the author simulates alternative paths using the Monte Carlo method ...
0
votes
0
answers
46
views
Reference about the comparison between covariance matrices
Suppose we have 2 symmetric matrices $A$ and $B$.
Then, we say that $A \succeq B$ if $A - B$ is a positive semi-definite matrix.
I was wondering about the intuition and interpretation of $A \succeq B$,...
0
votes
0
answers
52
views
How should I interpret parameters of the SARIMA model in time series analysis?
I am a bit confused as to why the SARIMA model requires four parameters beyond the ARIMA model just to remove the seasonal component from a time series.
Obviously $m$ is required to specify the ...
0
votes
0
answers
16
views
Are non-constant polynomial means a special case of seasonality?
In this video, it is said that an otherwise-stationary time series with non-constant linear mean is analyzed by taking the first difference of the time series to produce a new, stationary time series. ...
1
vote
0
answers
60
views
what is the Bernoulli product measure's Radon-Nikodym derivative wrt Lebesgue measure? [closed]
The Bernoulli product measure $\mu$ can be defined for each $p\in (0,1)$ on $\Omega = \{0,1\}^\mathbb N=\{\omega=(\omega_i)|\omega_i\in\{0,1\}, i\in\mathbb N\}=\Pi_{i=1}^\infty \{0,1\}$. The measure $...
2
votes
0
answers
56
views
Intuition behind occurence of non central chi squared distribution in conditional coordinates of a random walk
Description of background
Consider a 2d random walk with drift:
$$X(t) = \sum_{k=1}^t X_k \\
Y(t) = \sum_{k=1}^t Y_k$$
where each $X_k$ and $Y_k$ are independently exponentially distributed with rate ...
5
votes
4
answers
606
views
Provide an intuitive example of the linearity of expectation
Can anyone explain the linearity of expectation in an intuitive way? I have been trying to understand this for far too long now. Please don't use any equations and such, try to use real world examples ...
6
votes
1
answer
2k
views
Intuitive explanation of conformal prediction
I have recently started learning about conformal prediction. I am a programmer without a strong mathematical background, but with a strong intuitive, applied background in statistics. I am trying to ...
2
votes
0
answers
115
views
Intuition behind rank of covariance matrix and testing hypotheses
I am trying to acquire some intuition about testing multivariate hypotheses where the test statistic involves inverse covariance matrix. As an example, suppose we have a $p$-variate random vector that ...
3
votes
0
answers
82
views
Geometric intuition for how ridge ($L_2$) regularization helps under multicollinearity
We have some nice posts (1, 2 and likely more) illustrating multicollinearity geometrically. Now, ridge regression ($L_2$ regularization) is known to be a remedy of multicollinearity. What is the ...
2
votes
0
answers
170
views
Understanding intuitive difference between KL divergence and Cross entropy
I know there are related questions already asked, for example this one.
I also know the following:
KL divergence $D_{KL}(P\Vert Q)$ is given as:
$$\begin{align}
D_{KL}(P\Vert Q) & = -\sum_xP(x)\...
0
votes
1
answer
62
views
Real-World Example of Correlation of Random Variables
I'm encountering a result in research that is counter-intuitive to me. Specifically, I have two matrics, $X, Y$, where $X_i$ is the ith column of matrix $X$.
In my research:
$\Large{\rho}$$ (\sum X_i, ...
3
votes
1
answer
283
views
Can you explain bootstrapping like I’m 5?
I think I have a handle on what bootstrapping is and why we need to use it.
Please confirm if my understanding is correct:
Goal of bootstrapping: To find the SE of a feature’s coefficient that you ...
1
vote
0
answers
123
views
Concrete example of what Sufficient Statistics is [closed]
Having read articles to try to understand Sufficient Statistics.
Sufficient statistics for layman
A sufficient statistic summarizes all the information contained in a sample so that you would make ...
0
votes
0
answers
150
views
Intuitive statistics book [duplicate]
I am looking for a statistics book that not only gives formulas or proofs but also gives intuitive explanations.
For example, the standard deviation is defined by
${\sigma_x} = \sqrt{\frac{1}{n}{\sum\...
8
votes
2
answers
1k
views
Is variance the area under the curve of the distribution of a population?
I am trying to understand what variance is, I already know the "official" definition
"Variance is the average squared deviations from the mean"
But I am trying to give it a visual ...
4
votes
0
answers
320
views
What's the intuition behind the fact that sample mean and sample variance are independent when sampling from a normal population?
Let $X_1, \dotsc,X_n$ be i.i.d. from $N(\mu,\sigma^2)$, then we know that sample mean $\bar X\equiv \frac{1}{n}\sum_{i=1}^nX_i$ and $S^2=\frac{1}{n-1}(X_i-\bar X)^2$ are independent. Obviously, they ...
3
votes
1
answer
267
views
Difference between likelihood functions for pmf vs pdf
Can someone explain the intuition behind how the likelihood function for a specific value of $\theta$ is different if $f_\theta$ is a pmf vs a pdf?
I thought that it was simply the probability that a ...
1
vote
0
answers
38
views
How to connect the intuitions to the math of adaptive processes?
Formal Definition
Wikipedia gives the following definition of a process adapted to a filtration:
Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space;
$I$ be an index set with total order $\...
3
votes
0
answers
160
views
Is there an outer product counterpart for the Covariance?
Covariance
The covariance of two quantities $X$ and $Y$ within a population, $Cov(X,Y)$, is symmetric and bilinear. It is also true that $Cov(X,X) \ge 0$. So, clearly $Cov(X,Y)$ qualifies as an inner ...
1
vote
1
answer
35
views
Linear models when responses have no link
I am studying normal linear regression and wanted to ask a question about its utility when working with independent RV. Suppose that we have for $k \in [1,\dots,n]$, $$Y_k = \beta_0 + \beta_1x_{k1} + \...
0
votes
0
answers
61
views
Intuition on expected value of an estimator [duplicate]
When we don't know what's the mean of a normal distribution we try to estimate it and after a time we get lucky and have the true mean (in a magical way).
What does it mean the expected value of the ...
1
vote
0
answers
40
views
Interpretation of covariance and linear dependency [duplicate]
What is the best interpretation of covariance you can give ?
I know that if $X$ and $Y$ are random variables, then if $Cov(X,Y)>0$, then if realizations of $X$ are higher than expected, then ...
3
votes
1
answer
113
views
Mean of geometric distribution is odds?
Context: I mean the $P(X=k)=(1-p)^k p$ not the $P(Y=k)=(1-p)^{k-1} p$. Apparently the mean of the 1st kind of geometric is $\frac{1-p}{p}$ instead of $\frac{1}{p}$ for the 2nd kind of geometric. I ...
3
votes
1
answer
103
views
What does conditional independence mean semantically?
I've just spent the last 3 hours reading every post, question, Medium article, and textbook entry on conditional independence, and I still don't really understand it. Can somebody explain what it ...
0
votes
0
answers
344
views
What exactly does the Box-Cox transformation do to a time series?
If I were to try and rephrase the argument in the original Box-Cox paper in my own words, I would say something like the following: given a model
$$
y = x \beta ,
$$
if the residuals do not appear to ...
2
votes
1
answer
422
views
Why do we need a smaller sample size to detect a smaller proportion?
The plot below shows the sample size needed to detect a proportion with a precision 0.01 for various true proportions:
This assumes an infinite population size, and the confidence intervals are fixed ...
1
vote
0
answers
503
views
Why PCA is invariant under rotation
Lets say that we have a matrix of variables (the columns are variables and rows are the observations) called X whenre X = [x1, x2, ...., xp] where ...
1
vote
0
answers
198
views
Does the number of samples, as opposed to the sample size in each sample, matter for the Central Limit Theorem? [closed]
(1)So here is a formula that describes CLT I found at https://en.wikipedia.org/wiki/Central_limit_theorem.
According to the first part of the explanation, n as in Xn describes the number of samples(i....
1
vote
0
answers
81
views
Why is my intuition about probability in this regard so flawed? [closed]
Take the following example:
Take a sample from 100 people and measure their height. Assume that we know that height is approximately normally distributed, with a sample mean of 175 cm and sample ...
13
votes
5
answers
2k
views
What's complicated about regression to the mean?
Note: I am a bit of a novice when it comes to statistics and data analysis.
Reading the chapter on regression to the mean in Kahneman's Thinking Fast and Slow, I came across the following passage:
...
2
votes
0
answers
149
views
Intuition behind log in kl distance
So, let's start stating that I already read both Why KL-Divergence uses "ln" in its formula? and What is the role of the logarithm in Shannon's entropy? ... However, I still have no ...
1
vote
0
answers
257
views
Intuition about the relation between joint distribution, marginal distribution, and conditional distribution
The wording "intuition" might be a bit imprecise. I want to discuss how we visualize in our head going from one to another among the joint PDF, marginal PDF, and conditional PDF.
To make the ...
12
votes
5
answers
2k
views
Is the exact value of any likelihood meaningless?
While reading about likelihood, I have heard that "the exact value of any likelihood is meaningless" why?
So, because of that we may use the likelihood ratio.
So, my question is, why the ...
5
votes
1
answer
166
views
What is the intuition behind the odds scale?
What is an intuitive explanation of the odds scale?
In a logistic regression such as $$logit(p) = \beta_0 + \beta_1 x$$
we often interpret $\beta_1$ by looking at the odds ratio, $e^{\beta_1}$, which ...
1
vote
1
answer
222
views
MLE of the Uniform Distribution
In a uniform distribution where $0\leq X \leq \theta$, the pdf is represented as $f(X|\theta) = \frac{1}{\theta}I(0\leq X \leq \theta)$, and the likelihood is $L(\theta) = \prod\frac{1}{\theta}I(0\leq ...
4
votes
2
answers
874
views
Intuition for why mean of lognormal distribution depends on variance of normally distributed rv
Let $X\sim\mathcal{N}(\mu,\sigma^2)$, which is a normal distribution. Then, $\text{exp}(X)\sim\text{Lognormal}(\mu,\sigma^2)$, and its mean is
$$
\mathbb{E}[\text{exp}(X)]=\text{exp}\left(\mu+\dfrac{\...