Questions tagged [intuition]

Questions that seek a conceptual or non-mathematical understanding of statistics.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Question Intuition behind mathematics of activation function in a neural network.

Does this intuition behind why an activation function is used in a neural network make sense mathematically : For this example lets consider a fully connected (NOT CONVOLUTIONAL) network that ...
Stef's user avatar
  • 1
19 votes
8 answers
2k views

What is the intuition behind the idea that for linear regression, the number of observations should exceed the number of parameters?

If a population model has k independent variables and 1 intercept, why are k+1 observations required to perform OLS estimates? What is the intuition behind this?
user57623's user avatar
  • 309
7 votes
1 answer
613 views

What is a *likelihood ratio test* for a specific distribution, and how does it relate to hypothesis tests?

I'm just now being introduced to likelihood-ratio tests (LRT), and I am having trouble following the concept and terminology. For example, I posed a question about determining whether two samples {x} ...
feetwet's user avatar
  • 1,108
7 votes
2 answers
277 views

Intuition and reasoning why LASSO can only select $n$ features when $n \ll p$

I'm struggling to grasp the intuition behind why LASSO can only select at most $n$ features when $n << p$, where $n$ is the number of samples and $p$ is the number of features. I've read through ...
Jacob Bumgarner's user avatar
1 vote
1 answer
45 views

Is my interpretation of "the probability over data $X_1, ... X_N$ correct?

This may seem like a pretty simple question, but I want to make sure I am getting this right because it seems pretty foundational. I'm reading this note on conformal prediction. In the very first ...
paul's user avatar
  • 395
31 votes
13 answers
7k views

How to explain 1.5 children?

I teach undergrad stats and every year one student asks "You can't have 1.5 children" (the mean for the dataset). I am flummoxed every time to create a sensical answer. I've tried: "no ...
RLDavis's user avatar
  • 313
3 votes
1 answer
112 views

How to explain unbiasedness in basic terms?

If I take some estimator X. Lets say that X is unbiased. Suppose I have 100 samples and each sample has 5 points. I now calculate the value of X on each sample. Because X is unbiased, this means that ...
Uk rain troll's user avatar
2 votes
3 answers
129 views

Shouldn't we consider the difference in variance between population and a sample while calculating confidence intervals?

To comprehend the concept of confidence intervals, I came up with an example. I want to share it here for your better understanding what my question is all about. Suppose, we want to figure out what ...
Davie Blain's user avatar
13 votes
1 answer
837 views

Intuitive explanation for the fat tails of the t-distribution

Given some standard assumptions, the test statistic $$ \frac{\Delta\bar{X}}{\sigma/\sqrt{N}} $$ is normally distributed if $\sigma$ is known and t-distributed if $\sigma$ has to be estimated from the ...
monade's user avatar
  • 509
4 votes
1 answer
117 views

Markov's inequality intuitions

Can someone explain intuitively how Markov's inequality was derived? It seems plausible, but looking a it, I can't 'see' how it's true.
jbuddy_13's user avatar
  • 2,808
2 votes
0 answers
22 views

Intuition behind testing seasonality hypothesis

In this post to prove the statistical significance of a statement about a seasonality of a timeseries (every april returns are high) the author simulates alternative paths using the Monte Carlo method ...
gournge's user avatar
  • 21
0 votes
0 answers
42 views

Reference about the comparison between covariance matrices

Suppose we have 2 symmetric matrices $A$ and $B$. Then, we say that $A \succeq B$ if $A - B$ is a positive semi-definite matrix. I was wondering about the intuition and interpretation of $A \succeq B$,...
Yonatan Kurniawan's user avatar
0 votes
0 answers
51 views

How should I interpret parameters of the SARIMA model in time series analysis?

I am a bit confused as to why the SARIMA model requires four parameters beyond the ARIMA model just to remove the seasonal component from a time series. Obviously $m$ is required to specify the ...
user10478's user avatar
  • 111
0 votes
0 answers
16 views

Are non-constant polynomial means a special case of seasonality?

In this video, it is said that an otherwise-stationary time series with non-constant linear mean is analyzed by taking the first difference of the time series to produce a new, stationary time series. ...
user10478's user avatar
  • 111
1 vote
0 answers
60 views

what is the Bernoulli product measure's Radon-Nikodym derivative wrt Lebesgue measure? [closed]

The Bernoulli product measure $\mu$ can be defined for each $p\in (0,1)$ on $\Omega = \{0,1\}^\mathbb N=\{\omega=(\omega_i)|\omega_i\in\{0,1\}, i\in\mathbb N\}=\Pi_{i=1}^\infty \{0,1\}$. The measure $...
fromscratch's user avatar
2 votes
0 answers
55 views

Intuition behind occurence of non central chi squared distribution in conditional coordinates of a random walk

Description of background Consider a 2d random walk with drift: $$X(t) = \sum_{k=1}^t X_k \\ Y(t) = \sum_{k=1}^t Y_k$$ where each $X_k$ and $Y_k$ are independently exponentially distributed with rate ...
Sextus Empiricus's user avatar
5 votes
4 answers
578 views

Provide an intuitive example of the linearity of expectation

Can anyone explain the linearity of expectation in an intuitive way? I have been trying to understand this for far too long now. Please don't use any equations and such, try to use real world examples ...
Jeygopi's user avatar
  • 99
5 votes
1 answer
2k views

Intuitive explanation of conformal prediction

I have recently started learning about conformal prediction. I am a programmer without a strong mathematical background, but with a strong intuitive, applied background in statistics. I am trying to ...
Tripartio's user avatar
  • 2,146
0 votes
0 answers
126 views

What does the quality of representation of a variable mean in PCA?

I understand that the quality of representation of an individual by a certain axis is measured by the cosine of the angle between the axis and the individual; the more the vector representing the ...
Mehdi Charife's user avatar
2 votes
0 answers
112 views

Intuition behind rank of covariance matrix and testing hypotheses

I am trying to acquire some intuition about testing multivariate hypotheses where the test statistic involves inverse covariance matrix. As an example, suppose we have a $p$-variate random vector that ...
Richard Hardy's user avatar
3 votes
0 answers
81 views

Geometric intuition for how ridge ($L_2$) regularization helps under multicollinearity

We have some nice posts (1, 2 and likely more) illustrating multicollinearity geometrically. Now, ridge regression ($L_2$ regularization) is known to be a remedy of multicollinearity. What is the ...
Richard Hardy's user avatar
2 votes
0 answers
165 views

Understanding intuitive difference between KL divergence and Cross entropy

I know there are related questions already asked, for example this one. I also know the following: KL divergence $D_{KL}(P\Vert Q)$ is given as: $$\begin{align} D_{KL}(P\Vert Q) & = -\sum_xP(x)\...
Mahesha999's user avatar
0 votes
1 answer
62 views

Real-World Example of Correlation of Random Variables

I'm encountering a result in research that is counter-intuitive to me. Specifically, I have two matrics, $X, Y$, where $X_i$ is the ith column of matrix $X$. In my research: $\Large{\rho}$$ (\sum X_i, ...
2 votes
1 answer
270 views

Can you explain bootstrapping like I’m 5?

I think I have a handle on what bootstrapping is and why we need to use it. Please confirm if my understanding is correct: Goal of bootstrapping: To find the SE of a feature’s coefficient that you ...
Katsu's user avatar
  • 861
1 vote
0 answers
123 views

Concrete example of what Sufficient Statistics is [closed]

Having read articles to try to understand Sufficient Statistics. Sufficient statistics for layman A sufficient statistic summarizes all the information contained in a sample so that you would make ...
mon's user avatar
  • 1,436
0 votes
0 answers
146 views

Intuitive statistics book [duplicate]

I am looking for a statistics book that not only gives formulas or proofs but also gives intuitive explanations. For example, the standard deviation is defined by ${\sigma_x} = \sqrt{\frac{1}{n}{\sum\...
8 votes
2 answers
1k views

Is variance the area under the curve of the distribution of a population?

I am trying to understand what variance is, I already know the "official" definition "Variance is the average squared deviations from the mean" But I am trying to give it a visual ...
RodParedes's user avatar
3 votes
0 answers
288 views

What's the intuition behind the fact that sample mean and sample variance are independent when sampling from a normal population?

Let $X_1, \dotsc,X_n$ be i.i.d. from $N(\mu,\sigma^2)$, then we know that sample mean $\bar X\equiv \frac{1}{n}\sum_{i=1}^nX_i$ and $S^2=\frac{1}{n-1}(X_i-\bar X)^2$ are independent. Obviously, they ...
ExcitedSnail's user avatar
  • 2,616
3 votes
1 answer
250 views

Difference between likelihood functions for pmf vs pdf

Can someone explain the intuition behind how the likelihood function for a specific value of $\theta$ is different if $f_\theta$ is a pmf vs a pdf? I thought that it was simply the probability that a ...
Eunice Lo's user avatar
1 vote
0 answers
37 views

How to connect the intuitions to the math of adaptive processes?

Formal Definition Wikipedia gives the following definition of a process adapted to a filtration: Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space; $I$ be an index set with total order $\...
Galen's user avatar
  • 8,105
3 votes
0 answers
148 views

Is there an outer product counterpart for the Covariance?

Covariance The covariance of two quantities $X$ and $Y$ within a population, $Cov(X,Y)$, is symmetric and bilinear. It is also true that $Cov(X,X) \ge 0$. So, clearly $Cov(X,Y)$ qualifies as an inner ...
Fermion's user avatar
  • 31
1 vote
1 answer
35 views

Linear models when responses have no link

I am studying normal linear regression and wanted to ask a question about its utility when working with independent RV. Suppose that we have for $k \in [1,\dots,n]$, $$Y_k = \beta_0 + \beta_1x_{k1} + \...
Kilkik's user avatar
  • 305
0 votes
0 answers
60 views

Intuition on expected value of an estimator [duplicate]

When we don't know what's the mean of a normal distribution we try to estimate it and after a time we get lucky and have the true mean (in a magical way). What does it mean the expected value of the ...
Abderrahmen Hamdi's user avatar
1 vote
0 answers
40 views

Interpretation of covariance and linear dependency [duplicate]

What is the best interpretation of covariance you can give ? I know that if $X$ and $Y$ are random variables, then if $Cov(X,Y)>0$, then if realizations of $X$ are higher than expected, then ...
Kilkik's user avatar
  • 305
3 votes
1 answer
113 views

Mean of geometric distribution is odds?

Context: I mean the $P(X=k)=(1-p)^k p$ not the $P(Y=k)=(1-p)^{k-1} p$. Apparently the mean of the 1st kind of geometric is $\frac{1-p}{p}$ instead of $\frac{1}{p}$ for the 2nd kind of geometric. I ...
BCLC's user avatar
  • 2,404
3 votes
1 answer
96 views

What does conditional independence mean semantically?

I've just spent the last 3 hours reading every post, question, Medium article, and textbook entry on conditional independence, and I still don't really understand it. Can somebody explain what it ...
NaiveBae's user avatar
  • 257
0 votes
0 answers
333 views

What exactly does the Box-Cox transformation do to a time series?

If I were to try and rephrase the argument in the original Box-Cox paper in my own words, I would say something like the following: given a model $$ y = x \beta , $$ if the residuals do not appear to ...
Anthony's user avatar
  • 500
2 votes
1 answer
396 views

Why do we need a smaller sample size to detect a smaller proportion?

The plot below shows the sample size needed to detect a proportion with a precision 0.01 for various true proportions: This assumes an infinite population size, and the confidence intervals are fixed ...
Mhairi McNeill's user avatar
1 vote
0 answers
471 views

Why PCA is invariant under rotation

Lets say that we have a matrix of variables (the columns are variables and rows are the observations) called X whenre X = [x1, x2, ...., xp] where ...
Amir Jalilifard's user avatar
1 vote
0 answers
174 views

Does the number of samples, as opposed to the sample size in each sample, matter for the Central Limit Theorem? [closed]

(1)So here is a formula that describes CLT I found at https://en.wikipedia.org/wiki/Central_limit_theorem. According to the first part of the explanation, n as in Xn describes the number of samples(i....
e1phy's user avatar
  • 11
1 vote
0 answers
80 views

Why is my intuition about probability in this regard so flawed? [closed]

Take the following example: Take a sample from 100 people and measure their height. Assume that we know that height is approximately normally distributed, with a sample mean of 175 cm and sample ...
Sen90's user avatar
  • 111
13 votes
5 answers
2k views

What's complicated about regression to the mean?

Note: I am a bit of a novice when it comes to statistics and data analysis. Reading the chapter on regression to the mean in Kahneman's Thinking Fast and Slow, I came across the following passage: ...
ciru_4011's user avatar
  • 133
2 votes
0 answers
144 views

Intuition behind log in kl distance

So, let's start stating that I already read both Why KL-Divergence uses "ln" in its formula? and What is the role of the logarithm in Shannon's entropy? ... However, I still have no ...
Alberto's user avatar
  • 989
1 vote
0 answers
248 views

Intuition about the relation between joint distribution, marginal distribution, and conditional distribution

The wording "intuition" might be a bit imprecise. I want to discuss how we visualize in our head going from one to another among the joint PDF, marginal PDF, and conditional PDF. To make the ...
whoknowsnot's user avatar
12 votes
5 answers
2k views

Is the exact value of any likelihood meaningless?

While reading about likelihood, I have heard that "the exact value of any likelihood is meaningless" why? So, because of that we may use the likelihood ratio. So, my question is, why the ...
Alice's user avatar
  • 640
5 votes
1 answer
164 views

What is the intuition behind the odds scale?

What is an intuitive explanation of the odds scale? In a logistic regression such as $$logit(p) = \beta_0 + \beta_1 x$$ we often interpret $\beta_1$ by looking at the odds ratio, $e^{\beta_1}$, which ...
thomaskeefe's user avatar
1 vote
1 answer
218 views

MLE of the Uniform Distribution

In a uniform distribution where $0\leq X \leq \theta$, the pdf is represented as $f(X|\theta) = \frac{1}{\theta}I(0\leq X \leq \theta)$, and the likelihood is $L(\theta) = \prod\frac{1}{\theta}I(0\leq ...
Dan W's user avatar
  • 183
4 votes
2 answers
849 views

Intuition for why mean of lognormal distribution depends on variance of normally distributed rv

Let $X\sim\mathcal{N}(\mu,\sigma^2)$, which is a normal distribution. Then, $\text{exp}(X)\sim\text{Lognormal}(\mu,\sigma^2)$, and its mean is $$ \mathbb{E}[\text{exp}(X)]=\text{exp}\left(\mu+\dfrac{\...
T_T's user avatar
  • 143
0 votes
0 answers
29 views

Intuitive effect size (and CI) for one-way comparison of group means?

I'm a big fan of common language effect sizes (e.g. McGraw and Wong's CL or Cohen's U3 - "what proportion of group 1 are higher than the average for group 2"). I've been scrabbling around ...
justme's user avatar
  • 775
1 vote
2 answers
143 views

In a diff-in-diff or regression discontinuity research design, why is it important to describe why the counterfactual is a plausible one?

I've heard it mentioned that in difference-in-differences, regression discontinuity, or even in some other quasi-experimental research designs, that the counterfactual should be explained as a ...
simplycoding's user avatar

1
2 3 4 5
16