Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Questions that seek a conceptual or non-mathematical understanding of statistics.

0
votes
0answers
14 views

Intuitive explanation of the Wald Chi-Squared Test

I am having trouble understanding what the Wald Chi Squared test is used for intuitively. SPSS uses the Wald Test (type III) in the "Tests of Model Effects" box when I run my GEE models, but I don't ...
1
vote
2answers
50 views

True probability vs estimated probability

Is it correct to think that the true probability of an event cannot be ever known? When studying probability, in the first lectures, there are those typical exercises which start with sentences like: ...
1
vote
0answers
15 views

How can we best utilize the knowledge of P(y=1) in classification? [duplicate]

Premise I saw an interesting example of a machine learning logistic classifier for modeling/predicting sentiment for customer reviews. One of the first things in the example was a note on ...
7
votes
2answers
143 views

Clarification of the intuition behind backpropagation

I've been taking some time to try and understand the computations and mechanics of the machine learning algorithms I use in my day to day life. Studying the backpropagation literature on the CS231n ...
8
votes
1answer
121 views

Intuition (geometric or other) of $Var(X) = Var(E[X|Y]) + E[Var(X|Y)]$

In another installment of intuitions for identities in probability, consider the elementary identity Law of Total Variance $$ \begin{eqnarray} \rm{Var}(X) &=&\rm{E}[\rm{Var}(X|Y)] + \rm{Var}(...
-2
votes
2answers
34 views

My regression/ML algorithm has completed. Can linear algebra still play a part? [closed]

Premise It's a non-argument that linear algebra is a fundamental tool for many fields that use statistics. That being said, how fully one needs to grasp it often varies. Computers are very good at ...
8
votes
1answer
132 views

Intuitive explanation/motivation of stationary distribution of a process

Often, in literature, authors have been interested in finding the stationary distribution of a time-series process. For example, consider the following simple AR($1$) process $\{X_t\}$: $$X_t = \alpha ...
3
votes
1answer
60 views

Requesting Intuitive Explanation to Covariance, Correlation and Standard Deviation

First I would like to state that I am not from a mathematical background. I am studying about change in price of products. So I have to understand about Correlation , Covariance and Standard Deviation ...
0
votes
0answers
14 views

How to formulate a cost function for ROC analysis?

I get the general concept of the ROC as a graphical tool to explore the trade off between false positives and false negatives. However, I am still unclear as to exactly how one formulates the cost ...
0
votes
0answers
23 views

Will a normality test always agree with a qq plot?

I am trying to evaluate normality assessment of a variable using qq plot and/or a formal test. Let's consider the Jarque-Bera normality test. Granted that glancing at qq plots is not intended to be a ...
0
votes
0answers
25 views

Sum of seasonality equal to zero and normality of cycle component

In a time series, why do we require the seasonal component sum to zero over a season? Also, is the mean of the cycle component normalised to zero? I know it is too make the components uniquely ...
1
vote
1answer
62 views

When is it appropriate to apply a transformation?

I understand Box-Cox transformations, when to apply which transform (log vs exponential) and on what type of distribution (left vs right skewed). However, what I don't understand is when is it ...
1
vote
1answer
78 views

Covariance versus correlation: which is a “deeper” or more “structural” property of the data?

It might seem obvious that the covariance is a "deeper" property of the data generation process (DGP), since normally the specification of a joint distribution is done in terms of its mean vector and ...
3
votes
2answers
40 views

intuitive interpretation of canonical parameterization of beta distribution

For exponential family, e.g. Beta distirbution, someone argues that the canonical parameterization is better than the traditional $Beta(\alpha,\beta)$ way. The canonical parameters are defined as $n^...
9
votes
3answers
489 views

Intuition behind the formula for the variance of a sum of two variables

I know from previous studies that $Var(A+B) = Var(A) + Var(B) + 2 Cov (A,B)$ However, I don't understand why that is. I can see that the effect will be to 'push up' the variance when A and B covary ...
2
votes
1answer
26 views

Statistical theory proof intuition (UMVU estimators)

I've been working through this problem in Theoretical Statistics by Keener, but could not solve it. I looked up the answer and I do understand why it's correct, but I don't understand what intuition ...
1
vote
0answers
9 views

Process for finding UMVU estimator

I've been working on a problem from Theoretical Statistics: Topics for a Core Course by Keener. I spent a few hours on it making very little progress before caving and looking up the solution. I don't ...
1
vote
1answer
25 views

In case of Deep NN, why is gradient big in direction in which we want to travel a small distance?

I was going through the intuition behind Momentum and RMSprop technique applied in gradient descent. I read the following statement (from the CE NN lecture notes by G. Hinton) which is not clear to me ...
2
votes
2answers
40 views

Fundamentally, how is “the probability that two randomly selected samples belong to different classes” intuitively useful in any notion of purity?

The Gini impurity measure is defined by $$\sum_{i=1}^m f_i(1 - f_i)$$ This based on the probability of two randomly selected samples belonging to two different classes, one of which is $i$, i.e. $...
3
votes
2answers
141 views

What is Bayesian posterior probability and how is it different to just using a p-value?

I've got an extremely limited understanding of statistics, and every explanation I've found uses further technical terms that I don't understand. I'm trying to figure out: 1) What exactly is ...
5
votes
1answer
204 views

A simple explanation of PACF plot

I am presenting some ACF and PACF plots to colleagues. I can explain how to interpret the plots and how to determine p and q based on what the plots look like, but I cannot find a simple intuitive ...
1
vote
1answer
24 views

Standard Deviation of a Proportion

It's easy to explain why the standard deviation of a data set is computed the way it is. If you can compute an average, it is, in some sense simply an average deviation of each score from the mean for ...
0
votes
1answer
106 views

What is the correct way to scale data, apply PCA and fit a Multivariate Normal Distribution for anomaly detection?

I want to train an anomaly detection model in python. I have a training data set with some 30,000 observations, 700 of which are anomalies, and I can distinguish between normal and anomalous cases (I ...
1
vote
0answers
156 views

What is the intuition behind the KMO formula?

In answer to a different question about data assumptions of factor analysis rolando2 writes: There is another condition that is sometimes treated as an "assumption": that the zero-order (vanilla) ...
0
votes
1answer
17 views

How can I get the optimal perturbation of a trained model?

I get stuck while reading Goodfellow's paper on adversarial networks. In the explanation of the Figure 2 he stated that: b) The sign of the weights of a logistic regression model trained on ...
79
votes
8answers
11k views

Line of best fit does not look like a good fit. Why?

Have a look at this Excel graph: The 'common sense' line-of-best-fit would appear be an almost vertical line straight through the center of the points (edited by hand in red). However the linear ...
9
votes
2answers
1k views

What is the intuitive meaning behind plugging a random variable into its own pdf or cdf?

A pdf is usually written as $f(x|\theta)$, where the lowercase $x$ is treated as a realization or outcome of the random variable $X$ which has that pdf. Similarly, a cdf is written as $F_X(x)$, which ...
2
votes
0answers
87 views

GLM: link function and error distribution relationship, intuitive explanation

The very concise explanation of GLMs in my course describes the link function without saying much about the error distribution. From what I read in other places such distribution seems much more ...
1
vote
1answer
204 views

How is PCA applied to new data?

I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these ...
2
votes
3answers
237 views

Intuitive meaning of vector multiplication with covariance matrix

I often see multiplications with covariance matrices in literature. However I never really understood what is achieved by multiplication with the covariance matrix. Given $\Sigma * r = s$ with $\...
29
votes
12answers
4k views

What is the intuition behind the formula for conditional probability?

The formula for the conditional probability of $\text{A}$ happening given that $\text{B}$ has happened is:$$ P\left(\text{A}~\middle|~\text{B}\right)=\frac{P\left(\text{A} \cap \text{B}\right)}{P\left(...
11
votes
2answers
311 views

Intuition about parameter estimation in mixed models (variance parameters vs. conditional modes)

I have read many times that random effects (BLUPs/conditional modes for, say, subjects) are not parameters of a linear mixed effects model but instead can be derived from the estimated variance/...
5
votes
0answers
105 views

Is there any geometric intuition on least absolute deviation regression?

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.) Is there similar geometric intuition for least ...
1
vote
1answer
466 views

How does a one-class SVM model work?

I am working on a problem involving outliers detection and I found that it was possible to perform this using one-class SVM. I have been googling it and reading some blogs and papers, but I have a ...
2
votes
1answer
198 views

what is the intuition of why randomforest often works well?

What is the intuition behind why randomForest works well with prediction on lots of applied datasets? No fancy math required for an answer-but if you have some that helps intuition great.
0
votes
0answers
38 views

Intuition behind Mantel-Haenszel-weights

I am making a meta-analysis using four big RCTs. The effect measure is binary and the effect sizes are small. I want to use a fixed effects model and from what I can read, the best solution is to use ...
1
vote
1answer
46 views

Correlation for different slopes intuition

I just plotted some very simple R-Code to illustrate how two lines x1 and x2 can correlate perfectly but estimate very different values. I then got curious and ...
1
vote
0answers
22 views

Why do type III sums of squares require othogonal contrasts?

I have read many times that one has to set orthogonal contrast to get correct type III sums of square. E.g. John Fox says To compute Type-III tests using incremental F-tests, one needs contrasts ...
0
votes
0answers
52 views

How to explain differences between values of Kendall's $\tau$ to a layperson?

Suppose I have two data variables $x^{(1)}_t$, $x^{(2)}_t$, and a response variable $y_t$ of interest, all indexed by time in months, and I calculate Kendall's $\tau$ based on $x_t^{(1)}$ and $y_t$, ...
0
votes
1answer
22 views

Why is parameter identification defined on the distribution of observables, rather than just the sample?

I'm struggling to intuitively understand parameter estimation. Specifically, why do we say that a parameter is identified if it is determined by the probability distributions of observables? e.g. we ...
2
votes
2answers
148 views

Can an ARMA(4,1) be the same as an ARMA(3,0)?

I would really appreciate it if someone could explain the reasoning behind whether this can be true or not.
5
votes
1answer
279 views

Intuition for why sum of gaussian RVs is different from gaussian mixture

I know that in the case of Gaussian mixture, the "intuition" is that you're drawing from a PDF which itself is just a sum of weighted Gaussian PDFs. I don't understand the intuition behind how the ...
6
votes
1answer
2k views

A simple & clear explanation of the Gini impurity?

In a context of decision tree splitting, it is not obvious to see why the Gini impurity $$ i(t)=1-\sum\limits_{j=1}^k p^2(j|t) $$ is a measure of node t impurity. Is there an easy explanation of this?
4
votes
0answers
69 views

What is the physical intuition behind the equality $\sum_i (x_i - \bar x)^2 = \sum_i (x_i - \bar x) x_i$?

Suppose that $x_1,\ldots,x_n$ are real numbers, and let $\bar x$ denote the average $\frac{\sum_i x_i}n.$ I know how to prove on paper that the equality $$\sum_i (x_i - \bar x)^2 = \sum_i (x_i - \bar ...
2
votes
1answer
97 views

Interpretation of $\int_{0}^{x} 1-F(t) dt$, in particular when F=Gamma/Erlang dist

I know that $\int_{0}^{\infty} 1-F(t) dt$ is the expectation of a random variable. But what happens when the upper limit is some finite number like so? \begin{align*} \int_{0}^{x} 1-F(t) dt \end{...
0
votes
1answer
42 views

Hypothesis testing with two independent samples test statistics

I'm learning a bit about hypothesis testing with two independent samples (continuous outcome), and I'm just curious about where some of the equations for the test statistics are derived and/or what ...
2
votes
1answer
122 views

What does it mean when we say p(y|x) is fixed?

While reading The Elements of Information Theory on page 59. I got stuck in intuiting this line: If p(y|x) is fixed, then p(y) is a linear function of p(x). $p(y) = \sum_{x\in\mathcal{X}} p(y|x) ...
2
votes
0answers
931 views

Intuitive meaning behind support, confidence, lift and conviction

I'm learning about association rules and came across the common interestingness measures support, confidence, lift and conviction. I'm interested in the intuition behind your decision-making process ...
0
votes
1answer
62 views

What is $X$, it doesn't seem to be a set in the usual mathematical sense, even though it's often referred to as one [duplicate]

It's often used as a "set of values", however, it's not a set as there's no repetition in a set. Also operations such as $X^2$ are carried out, which will apply the operation to each element in the "...
4
votes
1answer
127 views

Intuitive reason why jointly normal and uncorrelated imply independence

It is well-known that if two random variables are jointly normal and uncorrelated, then they are independent. Does anyone have an intuitive reason why this is true? Explanation in terms of data is ...