Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [intuition]

Questions that seek a conceptual or non-mathematical understanding of statistics.

1
vote
0answers
18 views

What scenario corresponds to choosing the “true distribution” $p$ in $\textsf{KL}(p\parallel q)$?

I understand that when you think about changing $q$ in the Kullback-Leibler divergence $\textsf{KL}(p\parallel q)$, this corresponds to trying to find the distribution that minimizes information loss ...
3
votes
0answers
30 views

Which properties yield the exponential family of distributions?

It seems like every resource that discusses exponential families simply defines the family of distributions, explains why it's useful and then derives some of its properties. I have only seen one ...
0
votes
0answers
23 views

Intuition of variance (in the context of linear regression)

I was studying linear regression lately and checking the assumptions for Ordinary Least Squares method for the regression problem. I was not sure about the intuition behind the difference of squares ...
1
vote
4answers
53 views

How do I intuitively understand that independence is always symmetric?

Independence between two events, $A$ and $B$, is a symmetric relation, that is, if $P(A \mid B) = P(A)$, then $P(B \mid A) = P(B)$. The proof is very simple and can be found at the ProofWiki. ...
2
votes
2answers
63 views

Understanding the parameters needed for a distribution in Bayes networks?

Since I have a discriminative mindset hardly can I intuit the so-called parameters needed to specify a distribution in a generative Bayesian Network. I'd like to borrow an example from this blog. If ...
0
votes
0answers
6 views

What's a simple explanation for risk and its formula in survival analysis, weibull regression

I have that if the model is $\ln(\mu_i) = \beta_0 + \beta_1 x_1$ where $x_1 \in \{0,1\}$ and represents tired (or anything suitable, sex, etc). The model also has a shape parameter, $\gamma$. ...
0
votes
0answers
29 views

Idea behind change of basis and how it relates to projecting your points onto principal components

I would like to clarify if my understanding is correct. In the traditional X-Y coordinate system, our choice of basis vectors are $\vec{i} = (1, 0)$ and $\vec{j} = (0, 1)$ and when you I have a point $...
2
votes
1answer
35 views

Explaining modelling in simple terms

One of the most challenging ideas to describe to non-technical/mathematical people is "modelling". I've used quite a few explanations so far but all of them were around specific modelling techniques. ...
0
votes
0answers
17 views

Anyone can explain simply targeted learninig? [duplicate]

I am trying to understand Targeted Learning (Mark van der Laan), can anyone explain this method simply, please?
1
vote
1answer
43 views

Intuition behind the no convergence of the variance of sum of random variables

$$Var[\bar{X}] = \sigma^2/n $$ $$Var [\sum{X}_i] = n\sigma^2$$ $$lim_{n \to \infty} Var[\bar{X}] = 0 $$ wich means at $\infty$ we will always get the same $\bar{X}$ after every simulation. I ...
2
votes
0answers
31 views

Intuition behind White's Estimators/ Heteroscedasticity-consistent Standard Errors

For a medical study I am trying to understand the intuition behind heteroscedasticity-consistent standard errors. I know that it can be used, when in OLS regression residuals are heteroscedastic. By ...
2
votes
0answers
34 views

intuitive explanation for expected value of the square of a uniform variable

I'm confused about something that should be simple. Suppose I have a random uniform variable $X$ on $[0,1]$. It's fairly clear that the expected value of $X$ is 1/2. By integrating $x^2$ on $[0,1]$, I ...
2
votes
2answers
86 views

Intuitive understanding of variance of sum vs variance of difference

$\newcommand{\Var}{\operatorname{Var}}\newcommand{Cov}{\operatorname{Cov}}$Mathematically, $\Var(X + Y) = \Var(X) + \Var(Y) + 2\Cov(X,Y)$ and $\Var(X - Y) = \Var(X) + \Var(Y) - 2\Cov(X,Y)$ This ...
0
votes
0answers
26 views

Is there any intuition behind the writing Jensen-Shannon divergence based on the entropy?

I know that Jensen-Shannon divergence between two distribution $P$ and $Q$ can be written as follow: $JS(P||Q) = H(\frac{P+Q}{2}) - \frac{H(P)}{2} - \frac{H(Q)}{2}$ But is there any intuition behind ...
1
vote
0answers
19 views

How to Calculate Percent of Sample mean falls within Population Mean? [closed]

Following screenshot is from Udacity Statistics Tutorial. Klout score which is Social Media popularity score is distributed as shown in picture. Population mean is 37.72 and Standard Deviation is ...
6
votes
0answers
124 views

What guarantees the existence of a finite representation of the Wold decomposition? Mechanics and Intuition

Every covariance stationary process can be written as a linear, infinite distributed lag of white noise. In other words, every covariance stationary process has a Wold representation. Then we go on to ...
3
votes
2answers
73 views

What does it mean to model data as binomial?

On Wikipedia it says [T]he binomial distribution with parameters $n$ and $p$ is the discrete probability distribution of the number of successes in a sequence of n independent experiments, ...
7
votes
2answers
271 views

Uncorrelatedness + Joint Normality = Independence. Why? Intuition and mechanics

Two variables that are uncorrelated are not necessarily independent, as is simply exemplified by the fact that $X$ and $X^2$ are uncorrelated but not independent. However, two variables that are ...
14
votes
2answers
2k views

Is Cauchy distribution somehow an “unpredictable” distribution?

Is Cauchy distribution somehow an "unpredictable" distribution? I tried doing cs <- function(n) { return(rcauchy(n,0,1)) } in R for a multitude of n values ...
0
votes
0answers
18 views

Gaussianity and Whitening in ICA - The feeling and intuition behind it

I understand what ICA does at a high level but in the cocktail party problem context. All the examples, articles I have read take a similar problem to explain ICA where the aim is to derive the ...
0
votes
1answer
20 views

When and when not to use activation function between input layer and hidden layer? [duplicate]

I am beginner to Neural Network and this question might be very basic and stupid. Expecting some intuitive answer.
0
votes
1answer
61 views

What to conclude for the data-set when the variance for principal components is too low or too high?

I am working on analysing and visualizing a dataset having 12 features and came across PCA. I reduced the dataset to 2 principal components which together explain a variance of 18%. I was able to plot ...
1
vote
1answer
25 views

Sampling from a finite sequence without replacement yields exchangeable sequences?

I have read (and re-read) the wikipedia article on Exchangeability https://en.wikipedia.org/wiki/Exchangeable_random_variables . The disconnect for me is that : after having sampled without ...
1
vote
0answers
305 views

Interpreting Root Mean square Error (RMSE )when dependent variable is log transformed

So I have developed a model to predict the house prices in a county.I log transformed the target variable (home price).I split the data into training and testing sets.I trained the model on the ...
2
votes
0answers
36 views

Why truncated SVD can denoise images

There are a lot of empirical results about that truncated SVD (TSVD) can help denoise the noises of images, but I wonder what is the theoretical support behind that? We know that TSVD is the best low-...
5
votes
1answer
68 views

multivariate Student's t distribution: intuition for non-independence?

Consider a multivariate Student's t distribution, with parameters $\nu$ (d.f.), $\mu$ (location) and $\Sigma$ (shape). Does anyone have a good intuition for the individual components not being ...
6
votes
2answers
124 views

Can an optimal weighted average ever have negative weights?

I've got a few measurements $\vec{x}$ for some real-world value $\hat{x}$. These measurements have some uncertainty, and are correlated. Given these estimates, and their covariances, I want to take ...
2
votes
1answer
104 views

Memoryless property of exponential

Let $X$ be an exponential random variable. I have been asked to evaluate whether each of the following is true or false and am seeking some insight about a solution that was offered. $(a) E(X^2 |X>...
0
votes
1answer
53 views

Why, *intuitively*, in regular parametric problems, does uncertainty go down at a $\sqrt{ n }$ rate on the SE/posterior SD scale?

consider the simplest regular statistical inference problem: $( y_1, \dots, y_n | F ) \sim$ $\text{IID}$ from a cumulative distribution function $F$ on $\mathbb{ R }$ with mean $\mu$ and finite ...
1
vote
1answer
35 views

Can someone explain variance and bias of data in natural language (not mathematically)

I read many tutorial but until now i don't understand what the variance and bias means. So can someone explain me what these words means clearly, not in mathematical language.
10
votes
3answers
450 views

Intuition behind $(X^TX)^{-1}$ in closed form of w in Linear Regression

The closed form of w in Linear regression can be written as $\hat{w}=(X^TX)^{-1}X^Ty$ How can we intuitively explain the role of $(X^TX)^{-1}$ in this equation?
1
vote
1answer
47 views

Layman's explanation on stochastic and statistical models

What's the differences between stochastic models (process) and statistical model (analysis). As I understand, a stochastic model (process) simply means it involves random variables, which is basically ...
0
votes
0answers
52 views

Intuitive explanation of the Wald Chi-Squared Test

I am having trouble understanding what the Wald Chi Squared test is used for intuitively. SPSS uses the Wald Test (type III) in the "Tests of Model Effects" box when I run my GEE models, but I don't ...
1
vote
2answers
165 views

True probability vs estimated probability

Is it correct to think that the true probability of an event cannot be ever known? When studying probability, in the first lectures, there are those typical exercises which start with sentences like: ...
1
vote
0answers
18 views

How can we best utilize the knowledge of P(y=1) in classification? [duplicate]

Premise I saw an interesting example of a machine learning logistic classifier for modeling/predicting sentiment for customer reviews. One of the first things in the example was a note on ...
7
votes
2answers
162 views

Clarification of the intuition behind backpropagation

I've been taking some time to try and understand the computations and mechanics of the machine learning algorithms I use in my day to day life. Studying the backpropagation literature on the CS231n ...
8
votes
1answer
132 views

Intuition (geometric or other) of $Var(X) = Var(E[X|Y]) + E[Var(X|Y)]$

In another installment of intuitions for identities in probability, consider the elementary identity Law of Total Variance $$ \begin{eqnarray} \rm{Var}(X) &=&\rm{E}[\rm{Var}(X|Y)] + \rm{Var}(...
-2
votes
2answers
41 views

My regression/ML algorithm has completed. Can linear algebra still play a part? [closed]

Premise It's a non-argument that linear algebra is a fundamental tool for many fields that use statistics. That being said, how fully one needs to grasp it often varies. Computers are very good at ...
8
votes
1answer
142 views

Intuitive explanation/motivation of stationary distribution of a process

Often, in literature, authors have been interested in finding the stationary distribution of a time-series process. For example, consider the following simple AR($1$) process $\{X_t\}$: $$X_t = \alpha ...
3
votes
1answer
117 views

Requesting Intuitive Explanation to Covariance, Correlation and Standard Deviation

First I would like to state that I am not from a mathematical background. I am studying about change in price of products. So I have to understand about Correlation , Covariance and Standard Deviation ...
0
votes
0answers
33 views

How to formulate a cost function for ROC analysis?

I get the general concept of the ROC as a graphical tool to explore the trade off between false positives and false negatives. However, I am still unclear as to exactly how one formulates the cost ...
0
votes
0answers
26 views

Will a normality test always agree with a qq plot?

I am trying to evaluate normality assessment of a variable using qq plot and/or a formal test. Let's consider the Jarque-Bera normality test. Granted that glancing at qq plots is not intended to be a ...
0
votes
0answers
53 views

Sum of seasonality equal to zero and normality of cycle component

In a time series, why do we require the seasonal component sum to zero over a season? Also, is the mean of the cycle component normalised to zero? I know it is too make the components uniquely ...
2
votes
1answer
93 views

When is it appropriate to apply a transformation?

I understand Box-Cox transformations, when to apply which transform (log vs exponential) and on what type of distribution (left vs right skewed). However, what I don't understand is when is it ...
9
votes
0answers
179 views

Intuitive understanding of the Halmos-Savage theorem

The Halmos-Savage theorem says that for a dominated statistical model $(\Omega, \mathscr A, \mathscr P)$ a statistic $T: (\Omega, \mathscr A, \mathscr P)\to(\Omega', \mathscr A')$ is sufficient if (...
1
vote
1answer
126 views

Covariance versus correlation: which is a “deeper” or more “structural” property of the data?

It might seem obvious that the covariance is a "deeper" property of the data generation process (DGP), since normally the specification of a joint distribution is done in terms of its mean vector and ...
3
votes
2answers
95 views

intuitive interpretation of canonical parameterization of beta distribution

For exponential family, e.g. Beta distirbution, someone argues that the canonical parameterization is better than the traditional $Beta(\alpha,\beta)$ way. The canonical parameters are defined as $n^...
10
votes
3answers
605 views

Intuition behind the formula for the variance of a sum of two variables

I know from previous studies that $Var(A+B) = Var(A) + Var(B) + 2 Cov (A,B)$ However, I don't understand why that is. I can see that the effect will be to 'push up' the variance when A and B covary ...
2
votes
1answer
30 views

Statistical theory proof intuition (UMVU estimators)

I've been working through this problem in Theoretical Statistics by Keener, but could not solve it. I looked up the answer and I do understand why it's correct, but I don't understand what intuition ...
1
vote
0answers
14 views

Process for finding UMVU estimator

I've been working on a problem from Theoretical Statistics: Topics for a Core Course by Keener. I spent a few hours on it making very little progress before caving and looking up the solution. I don't ...