Greatest Hits

1338 votes
27 answers
931k views

Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...
claws's user avatar
  • 13.7k
74 votes
10 answers
1.1m views

What is the difference between discrete data and continuous data?

What is the difference between discrete data and continuous data?
Albort's user avatar
  • 891
402 votes
7 answers
1.7m views

How to normalize data to 0-1 range?

I am lost in normalizing, could anyone guide me please. I have a minimum and maximum values, say -23.89 and 7.54990767, respectively. If I get a value of 5.6878 how can I scale this value on a scale ...
Angelo's user avatar
  • 4,475
814 votes
11 answers
1.1m views

How to choose the number of hidden layers and nodes in a feedforward neural network?

Is there a standard and accepted method for selecting the number of layers, and the number of nodes in each layer, in a feed-forward neural network? I'm interested in automated ways of building neural ...
Rob Hyndman's user avatar
  • 56.4k
427 votes
9 answers
870k views

What is the difference between fixed effect, random effect in mixed effect models?

In simple terms, how would you explain (perhaps with simple examples) the difference between fixed effect, random effect in mixed effect models?
Andrew's user avatar
  • 6,158
180 votes
7 answers
888k views

What's the difference between variance and standard deviation?

I was wondering what the difference between the variance and the standard deviation is. If you calculate the two values, it is clear that you get the standard deviation out of the variance, but what ...
Le Max's user avatar
  • 3,719
561 votes
11 answers
657k views

What is the difference between test set and validation set?

I found this confusing when I use the neural network toolbox in Matlab. It divided the raw data set into three parts: training set validation set test set I notice in many training or learning ...
xiaohan2012's user avatar
  • 7,159
301 votes
6 answers
703k views

What is batch size in neural network?

I'm using Python Keras package for neural network. This is the link. Is batch_size equals to number of test samples? From ...
user2991243's user avatar
  • 4,241
600 votes
5 answers
486k views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
amoeba's user avatar
  • 104k
665 votes
12 answers
490k views

What is the difference between "likelihood" and "probability"?

The wikipedia page claims that likelihood and probability are distinct concepts. In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a ...
Douglas S. Stones's user avatar
567 votes
15 answers
242k views

What is the intuition behind beta distribution?

Disclaimer: I'm not a statistician but a software engineer. Most of my knowledge in statistics comes from self-education, thus I still have many gaps in understanding concepts that may seem trivial ...
ffriend's user avatar
  • 9,960
47 votes
5 answers
428k views

What is the difference between a population and a sample?

What is the difference between a population and a sample? What common variables and statistics are used for each one, and how do those relate to each other?
Baltimark's user avatar
  • 2,248
288 votes
11 answers
221k views

What exactly are keys, queries, and values in attention mechanisms?

How should one understand the keys, queries, and values that are often mentioned in attention mechanisms? I've tried searching online, but all the resources I find only speak of them as if the reader ...
Sean's user avatar
  • 3,787
388 votes
12 answers
384k views

Difference between logit and probit models

What is the difference between Logit and Probit model? I'm more interested here in knowing when to use logistic regression, and when to use Probit. If there is any literature which defines it using ...
Beta's user avatar
  • 6,304
241 votes
20 answers
272k views

Intuitive explanation for dividing by $n-1$ when calculating standard deviation?

I was asked today in class why you divide the sum of square error by $n-1$ instead of with $n$, when calculating the standard deviation. I said I am not going to answer it in class (since I didn't ...
Tal Galili's user avatar
  • 21.4k
360 votes
9 answers
352k views

What should I do when my neural network doesn't learn?

I'm training a neural network but the training loss doesn't decrease. How can I fix this? I'm not asking about overfitting or regularization. I'm asking about how to solve the problem where my network'...
Sycorax's user avatar
  • 90.5k
83 votes
28 answers
397k views

Examples for teaching: Correlation does not mean causation

There is an old saying: "Correlation does not mean causation". When I teach, I tend to use the following standard examples to illustrate this point: number of storks and birth rate in Denmark; number ...
243 votes
4 answers
494k views

How to interpret a QQ plot?

I am working with a small dataset (21 observations) and have the following normal QQ plot in R: Seeing that the plot does not support normality, what could I infer about the underlying distribution? ...
JohnK's user avatar
  • 20.2k
292 votes
15 answers
552k views

What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It ...
Sharpie's user avatar
  • 4,344
219 votes
8 answers
529k views

In linear regression, when is it appropriate to use the log of an independent variable instead of the actual values?

Am I looking for a better behaved distribution for the independent variable in question, or to reduce the effect of outliers, or something else?
d_2's user avatar
  • 2,351
159 votes
5 answers
616k views

How to choose between Pearson and Spearman correlation?

How do I know when to choose between Spearman's $\rho$ and Pearson's $r$? My variable includes satisfaction and the scores were interpreted using the sum of the scores. However, these scores could ...
user avatar
193 votes
8 answers
399k views

What is the influence of C in SVMs with linear kernel?

I am currently using an SVM with a linear kernel to classify my data. There is no error on the training set. I tried several values for the parameter $C$ ($10^{-5}, \dots, 10^2$). This did not ...
alfa's user avatar
  • 2,685
291 votes
8 answers
214k views

How to choose a predictive model after k-fold cross-validation?

I am wondering how to choose a predictive model after doing K-fold cross-validation. This may be awkwardly phrased, so let me explain in more detail: whenever I run K-fold cross-validation, I use K ...
Berk U.'s user avatar
  • 5,015
191 votes
80 answers
258k views

Statistics Jokes

Well, we've got favourite statistics quotes. What about statistics jokes?
289 votes
7 answers
457k views

What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
josh's user avatar
  • 3,249
382 votes
84 answers
182k views

What is your favorite "data analysis" cartoon?

Data analysis cartoons can be useful for many reasons: they help communicate; they show that quantitative people have a sense of humor too; they can instigate good teaching moments; and they can help ...
282 votes
153 answers
148k views

Famous statistical quotations

What is your favorite statistical quote? This is community wiki, so please one quote per answer.
385 votes
5 answers
452k views

What is the trade-off between batch size and number of iterations to train a neural network?

When training a neural network, what difference does it make to set: batch size to $a$ and number of iterations to $b$ vs. batch size to $c$ and number of iterations to $d$ where $ ab = cd $? To ...
Franck Dernoncourt's user avatar
560 votes
23 answers
322k views

Why square the difference instead of taking the absolute value in standard deviation?

In the definition of standard deviation, why do we have to square the difference from the mean to get the mean (E) and take the square root back at the end? Can't we just simply take the absolute ...
c4il's user avatar
  • 5,855
211 votes
4 answers
201k views

What does the hidden layer in a neural network compute?

I'm sure many people will respond with links to 'let me google that for you', so I want to say that I've tried to figure this out so please forgive my lack of understanding here, but I cannot figure ...
FAtBalloon's user avatar
  • 2,247
3 votes
1 answer
67k views

Data analysis for "aviator" online casino game

This casino game called aviator gives random returns each round. For example 1st round : x2.1 2nd round : x1.43 3rd round : x56 4th round : x1 etc. The game continues 24/7 and you essentially bet what ...
Dots's user avatar
  • 31
45 votes
3 answers
553k views

How to interpret F- and p-value in ANOVA?

I am new to statistics and I currently deal with ANOVA. I carry out an ANOVA test in R using aov(dependendVar ~ IndependendVar) I get – among others – an F-value ...
JanD's user avatar
  • 561
241 votes
4 answers
367k views

When (and why) should you take the log of a distribution (of numbers)?

Say I have some historical data e.g., past stock prices, airline ticket price fluctuations, past financial data of the company... Now someone (or some formula) comes along and says "let's take/use ...
PhD's user avatar
  • 14.6k
118 votes
10 answers
210k views

Validation Error less than training error?

I found two questions here and here about this issue but there is no obvious answer or explanation yet.I enforce the same problem where the validation error is less than training error in my ...
Bido's user avatar
  • 1,283
179 votes
11 answers
204k views

What is the difference between off-policy and on-policy learning?

Artificial intelligence website defines off-policy and on-policy learning as follows: "An off-policy learner learns the value of the optimal policy independently of the agent's actions. Q-learning ...
cgo's user avatar
  • 9,047
282 votes
11 answers
166k views

How would you explain covariance to someone who understands only the mean?

...assuming that I'm able to augment their knowledge about variance in an intuitive fashion ( Understanding "variance" intuitively ) or by saying: It's the average distance of the data ...
PhD's user avatar
  • 14.6k
405 votes
7 answers
412k views

When conducting multiple regression, when should you center your predictor variables & when should you standardize them?

In some literature, I have read that a regression with multiple explanatory variables, if in different units, needed to be standardized. (Standardizing consists in subtracting the mean and dividing ...
mathieu_r's user avatar
  • 4,461
205 votes
4 answers
295k views

How to determine which distribution fits my data best?

I have a dataset and would like to figure out which distribution fits my data best. I used the fitdistr() function to estimate the necessary parameters to ...
tobibo's user avatar
  • 2,155
443 votes
5 answers
174k views

How to understand the drawbacks of K-means

K-means is a widely used method in cluster analysis. In my understanding, this method does NOT require ANY assumptions, i.e., give me a dataset and a pre-specified number of clusters, k, and I just ...
KevinKim's user avatar
  • 6,869
168 votes
4 answers
204k views

Percentile vs quantile vs quartile

What is the difference between the three terms below? percentile quantile quartile
luciano's user avatar
  • 14.2k
130 votes
4 answers
255k views

Softmax vs Sigmoid function in Logistic classifier?

What decides the choice of function ( Softmax vs Sigmoid ) in a Logistic classifier ? Suppose there are 4 output classes . Each of the above function gives the probabilities of each class being the ...
mach's user avatar
  • 1,795
407 votes
19 answers
169k views

What happens if the explanatory and response variables are sorted independently before regression?

Suppose we have data set $(X_i,Y_i)$ with $n$ points. We want to perform a linear regression, but first we sort the $X_i$ values and the $Y_i$ values independently of each other, forming data set $(...
arbitrary user's user avatar
154 votes
3 answers
245k views

Help me understand Bayesian prior and posterior distributions

In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
Bob's user avatar
  • 1,541
434 votes
14 answers
281k views

Bayesian and frequentist reasoning in plain English

How would you describe in plain English the characteristics that distinguish Bayesian from Frequentist reasoning?
Daniel Vassallo's user avatar
277 votes
12 answers
187k views

How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?

Maybe the concept, why it's used, and an example.
Neil McGuigan's user avatar
195 votes
3 answers
243k views

What is the difference between a consistent estimator and an unbiased estimator?

What is the difference between a consistent estimator and an unbiased estimator? The precise technical definitions of these terms are fairly complicated, and it's difficult to get an intuitive feel ...
MathematicalOrchid's user avatar
223 votes
13 answers
202k views

What is the difference between data mining, statistics, machine learning and AI?

What is the difference between data mining, statistics, machine learning and AI? Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
55 votes
5 answers
228k views

What is difference between “in-sample” and “out-of-sample” forecasts?

I don't understand what exactly is the difference between "in-sample" and "out of sample" prediction? An in-sample forecast utilizes a subset of the available data to forecast ...
Engin YILMAZ's user avatar
232 votes
8 answers
282k views

What are the advantages of ReLU over sigmoid function in deep neural networks?

The state of the art of non-linearity is to use rectified linear units (ReLU) instead of sigmoid function in deep neural network. What are the advantages? I know that training a network when ReLU is ...
RockTheStar's user avatar
  • 12.9k
103 votes
9 answers
286k views

What's the difference between correlation and simple linear regression?

In particular, I am referring to the Pearson product-moment correlation coefficient.
Neil McGuigan's user avatar