Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions that seek a conceptual or non-mathematical understanding of statistics.

1
vote
0answers
23 views

Interpreting Root Mean square Error (RMSE )when dependent variable is log transformed

So I have developed a model to predict the house prices in a county.I log transformed the target variable (home price).I split the data into training and testing sets.I trained the model on the ...
2
votes
0answers
25 views

Why truncated SVD can denoise images

There are a lot of empirical results about that truncated SVD (TSVD) can help denoise the noises of images, but I wonder what is the theoretical support behind that? We know that TSVD is the best low-...
5
votes
1answer
56 views

multivariate Student's t distribution: intuition for non-independence?

Consider a multivariate Student's t distribution, with parameters $\nu$ (d.f.), $\mu$ (location) and $\Sigma$ (shape). Does anyone have a good intuition for the individual components not being ...
5
votes
2answers
98 views

Can an optimal weighted average ever have negative weights?

I've got a few measurements $\vec{x}$ for some real-world value $\hat{x}$. These measurements have some uncertainty, and are correlated. Given these estimates, and their covariances, I want to take ...
2
votes
1answer
51 views

Memoryless property of exponential

Let $X$ be an exponential random variable. I have been asked to evaluate whether each of the following is true or false and am seeking some insight about a solution that was offered. $(a) E(X^2 |X>...
0
votes
1answer
47 views

Why, *intuitively*, in regular parametric problems, does uncertainty go down at a $\sqrt{ n }$ rate on the SE/posterior SD scale?

consider the simplest regular statistical inference problem: $( y_1, \dots, y_n | F ) \sim$ $\text{IID}$ from a cumulative distribution function $F$ on $\mathbb{ R }$ with mean $\mu$ and finite ...
1
vote
1answer
34 views

Can someone explain variance and bias of data in natural language (not mathematically)

I read many tutorial but until now i don't understand what the variance and bias means. So can someone explain me what these words means clearly, not in mathematical language.
8
votes
3answers
267 views

Intuition behind $(X^TX)^{-1}$ in closed form of w in Linear Regression

The closed form of w in Linear regression can be written as $\hat{w}=(X^TX)^{-1}X^Ty$ How can we intuitively explain the role of $(X^TX)^{-1}$ in this equation?
1
vote
1answer
40 views

Layman's explanation on stochastic and statistical models

What's the differences between stochastic models (process) and statistical model (analysis). As I understand, a stochastic model (process) simply means it involves random variables, which is basically ...
0
votes
0answers
28 views

Intuitive explanation of the Wald Chi-Squared Test

I am having trouble understanding what the Wald Chi Squared test is used for intuitively. SPSS uses the Wald Test (type III) in the "Tests of Model Effects" box when I run my GEE models, but I don't ...
1
vote
2answers
68 views

True probability vs estimated probability

Is it correct to think that the true probability of an event cannot be ever known? When studying probability, in the first lectures, there are those typical exercises which start with sentences like: ...
1
vote
0answers
15 views

How can we best utilize the knowledge of P(y=1) in classification? [duplicate]

Premise I saw an interesting example of a machine learning logistic classifier for modeling/predicting sentiment for customer reviews. One of the first things in the example was a note on ...
7
votes
2answers
147 views

Clarification of the intuition behind backpropagation

I've been taking some time to try and understand the computations and mechanics of the machine learning algorithms I use in my day to day life. Studying the backpropagation literature on the CS231n ...
8
votes
1answer
121 views

Intuition (geometric or other) of $Var(X) = Var(E[X|Y]) + E[Var(X|Y)]$

In another installment of intuitions for identities in probability, consider the elementary identity Law of Total Variance $$ \begin{eqnarray} \rm{Var}(X) &=&\rm{E}[\rm{Var}(X|Y)] + \rm{Var}(...
-2
votes
2answers
37 views

My regression/ML algorithm has completed. Can linear algebra still play a part? [closed]

Premise It's a non-argument that linear algebra is a fundamental tool for many fields that use statistics. That being said, how fully one needs to grasp it often varies. Computers are very good at ...
8
votes
1answer
136 views

Intuitive explanation/motivation of stationary distribution of a process

Often, in literature, authors have been interested in finding the stationary distribution of a time-series process. For example, consider the following simple AR($1$) process $\{X_t\}$: $$X_t = \alpha ...
3
votes
1answer
69 views

Requesting Intuitive Explanation to Covariance, Correlation and Standard Deviation

First I would like to state that I am not from a mathematical background. I am studying about change in price of products. So I have to understand about Correlation , Covariance and Standard Deviation ...
0
votes
0answers
19 views

How to formulate a cost function for ROC analysis?

I get the general concept of the ROC as a graphical tool to explore the trade off between false positives and false negatives. However, I am still unclear as to exactly how one formulates the cost ...
0
votes
0answers
24 views

Will a normality test always agree with a qq plot?

I am trying to evaluate normality assessment of a variable using qq plot and/or a formal test. Let's consider the Jarque-Bera normality test. Granted that glancing at qq plots is not intended to be a ...
0
votes
0answers
32 views

Sum of seasonality equal to zero and normality of cycle component

In a time series, why do we require the seasonal component sum to zero over a season? Also, is the mean of the cycle component normalised to zero? I know it is too make the components uniquely ...
2
votes
1answer
73 views

When is it appropriate to apply a transformation?

I understand Box-Cox transformations, when to apply which transform (log vs exponential) and on what type of distribution (left vs right skewed). However, what I don't understand is when is it ...
4
votes
0answers
90 views
+50

Intuitive understanding of the Halmos-Savage theorem

The Halmos-Savage theorem says that for a dominated statistical model $(\Omega, \mathscr A, \mathscr P)$ a statistic $T: (\Omega, \mathscr A, \mathscr P)\to(\Omega', \mathscr A')$ is sufficient if for ...
1
vote
1answer
101 views

Covariance versus correlation: which is a “deeper” or more “structural” property of the data?

It might seem obvious that the covariance is a "deeper" property of the data generation process (DGP), since normally the specification of a joint distribution is done in terms of its mean vector and ...
3
votes
2answers
48 views

intuitive interpretation of canonical parameterization of beta distribution

For exponential family, e.g. Beta distirbution, someone argues that the canonical parameterization is better than the traditional $Beta(\alpha,\beta)$ way. The canonical parameters are defined as $n^...
9
votes
3answers
514 views

Intuition behind the formula for the variance of a sum of two variables

I know from previous studies that $Var(A+B) = Var(A) + Var(B) + 2 Cov (A,B)$ However, I don't understand why that is. I can see that the effect will be to 'push up' the variance when A and B covary ...
2
votes
1answer
27 views

Statistical theory proof intuition (UMVU estimators)

I've been working through this problem in Theoretical Statistics by Keener, but could not solve it. I looked up the answer and I do understand why it's correct, but I don't understand what intuition ...
1
vote
0answers
11 views

Process for finding UMVU estimator

I've been working on a problem from Theoretical Statistics: Topics for a Core Course by Keener. I spent a few hours on it making very little progress before caving and looking up the solution. I don't ...
1
vote
1answer
26 views

In case of Deep NN, why is gradient big in direction in which we want to travel a small distance?

I was going through the intuition behind Momentum and RMSprop technique applied in gradient descent. I read the following statement (from the CE NN lecture notes by G. Hinton) which is not clear to me ...
2
votes
2answers
42 views

Fundamentally, how is “the probability that two randomly selected samples belong to different classes” intuitively useful in any notion of purity?

The Gini impurity measure is defined by $$\sum_{i=1}^m f_i(1 - f_i)$$ This based on the probability of two randomly selected samples belonging to two different classes, one of which is $i$, i.e. $...
3
votes
2answers
245 views

What is Bayesian posterior probability and how is it different to just using a p-value?

I've got an extremely limited understanding of statistics, and every explanation I've found uses further technical terms that I don't understand. I'm trying to figure out: What exactly is posterior ...
5
votes
1answer
262 views

A simple explanation of PACF plot

I am presenting some ACF and PACF plots to colleagues. I can explain how to interpret the plots and how to determine p and q based on what the plots look like, but I cannot find a simple intuitive ...
1
vote
1answer
26 views

Standard Deviation of a Proportion

It's easy to explain why the standard deviation of a data set is computed the way it is. If you can compute an average, it is, in some sense simply an average deviation of each score from the mean for ...
0
votes
1answer
157 views

What is the correct way to scale data, apply PCA and fit a Multivariate Normal Distribution for anomaly detection?

I want to train an anomaly detection model in python. I have a training data set with some 30,000 observations, 700 of which are anomalies, and I can distinguish between normal and anomalous cases (I ...
1
vote
0answers
256 views

What is the intuition behind the KMO formula?

In answer to a different question about data assumptions of factor analysis rolando2 writes: There is another condition that is sometimes treated as an "assumption": that the zero-order (vanilla) ...
0
votes
1answer
20 views

How can I get the optimal perturbation of a trained model?

I get stuck while reading Goodfellow's paper on adversarial networks. In the explanation of the Figure 2 he stated that: b) The sign of the weights of a logistic regression model trained on ...
80
votes
8answers
11k views

Line of best fit does not look like a good fit. Why?

Have a look at this Excel graph: The 'common sense' line-of-best-fit would appear be an almost vertical line straight through the center of the points (edited by hand in red). However the linear ...
9
votes
2answers
1k views

What is the intuitive meaning behind plugging a random variable into its own pdf or cdf?

A pdf is usually written as $f(x|\theta)$, where the lowercase $x$ is treated as a realization or outcome of the random variable $X$ which has that pdf. Similarly, a cdf is written as $F_X(x)$, which ...
2
votes
0answers
106 views

GLM: link function and error distribution relationship, intuitive explanation

The very concise explanation of GLMs in my course describes the link function without saying much about the error distribution. From what I read in other places such distribution seems much more ...
1
vote
1answer
287 views

How is PCA applied to new data?

I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these ...
3
votes
3answers
404 views

Intuitive meaning of vector multiplication with covariance matrix

I often see multiplications with covariance matrices in literature. However I never really understood what is achieved by multiplication with the covariance matrix. Given $\Sigma * r = s$ with $\...
29
votes
12answers
4k views

What is the intuition behind the formula for conditional probability?

The formula for the conditional probability of $\text{A}$ happening given that $\text{B}$ has happened is:$$ P\left(\text{A}~\middle|~\text{B}\right)=\frac{P\left(\text{A} \cap \text{B}\right)}{P\left(...
13
votes
2answers
375 views

Intuition about parameter estimation in mixed models (variance parameters vs. conditional modes)

I have read many times that random effects (BLUPs/conditional modes for, say, subjects) are not parameters of a linear mixed effects model but instead can be derived from the estimated variance/...
5
votes
0answers
111 views

Is there any geometric intuition on least absolute deviation regression?

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.) Is there similar geometric intuition for least ...
1
vote
1answer
622 views

How does a one-class SVM model work?

I am working on a problem involving outliers detection and I found that it was possible to perform this using one-class SVM. I have been googling it and reading some blogs and papers, but I have a ...
0
votes
0answers
318 views

difference between correlation and significance?

What is the difference between correlation and significance? Why a correlation of two variables have a r value of .23 is significant while the correction of another set (different variables) has a r ...
2
votes
1answer
216 views

what is the intuition of why randomforest often works well?

What is the intuition behind why randomForest works well with prediction on lots of applied datasets? No fancy math required for an answer-but if you have some that helps intuition great.
1
vote
1answer
49 views

Correlation for different slopes intuition

I just plotted some very simple R-Code to illustrate how two lines x1 and x2 can correlate perfectly but estimate very different values. I then got curious and ...
2
votes
1answer
32 views

Why do type III sums of squares require orthogonal contrasts?

I have read many times that one has to set orthogonal contrast to get correct type III sums of square. E.g. John Fox says To compute Type-III tests using incremental F-tests, one needs contrasts ...
0
votes
0answers
61 views

How to explain differences between values of Kendall's $\tau$ to a layperson?

Suppose I have two data variables $x^{(1)}_t$, $x^{(2)}_t$, and a response variable $y_t$ of interest, all indexed by time in months, and I calculate Kendall's $\tau$ based on $x_t^{(1)}$ and $y_t$, ...
0
votes
1answer
22 views

Why is parameter identification defined on the distribution of observables, rather than just the sample?

I'm struggling to intuitively understand parameter estimation. Specifically, why do we say that a parameter is identified if it is determined by the probability distributions of observables? e.g. we ...