Greatest Hits

1011
votes
28answers
604k views

Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...
67
votes
10answers
1.0m views

What is the difference between discrete data and continuous data?

What is the difference between discrete data and continuous data?
276
votes
6answers
1.1m views

How to normalize data to 0-1 range?

I am lost in normalizing, could anyone guide me please. I have a minimum and maximum values, say -23.89 and 7.54990767, respectively. If I get a value of 5.6878 how can I scale this value on a scale ...
134
votes
6answers
734k views

What's the difference between variance and standard deviation?

I was wondering what the difference between the variance and the standard deviation is. If you calculate the two values, it is clear that you get the standard deviation out of the variance, but what ...
442
votes
11answers
497k views

What is the difference between test set and validation set?

I found this confusing when I use the neural network toolbox in Matlab. It divided the raw data set into three parts: training set validation set test set I notice in many training or learning ...
562
votes
11answers
621k views

How to choose the number of hidden layers and nodes in a feedforward neural network?

Is there a standard and accepted method for selecting the number of layers, and the number of nodes in each layer, in a feed-forward neural network? I'm interested in automated ways of building neural ...
276
votes
9answers
522k views

What is the difference between fixed effect, random effect and mixed effect models?

In simple terms, how would you explain (perhaps with simple examples) the difference between fixed effect, random effect and mixed effect models?
455
votes
12answers
178k views

What is the intuition behind beta distribution?

Disclaimer: I'm not a statistician but a software engineer. Most of my knowledge in statistics comes from self-education, thus I still have many gaps in understanding concepts that may seem trivial ...
39
votes
5answers
376k views

What is the difference between a population and a sample?

What is the difference between a population and a sample? What common variables and statistics are used for each one, and how do those relate to each other?
498
votes
11answers
313k views

What is the difference between “likelihood” and “probability”?

The wikipedia page claims that likelihood and probability are distinct concepts. In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a ...
185
votes
4answers
280k views

What is batch size in neural network?

I'm using Python Keras package for neural network. This is the link. Is batch_size equals to number of test samples? From ...
75
votes
29answers
342k views

Examples for teaching: Correlation does not mean causation

There is an old saying: "Correlation does not mean causation". When I teach, I tend to use the following standard examples to illustrate this point: number of storks and birth rate in Denmark; number ...
304
votes
10answers
293k views

Difference between logit and probit models

What is the difference between Logit and Probit model? I'm more interested here in knowing when to use logistic regression, and when to use Probit. If there is any literature which defines it using ...
374
votes
3answers
238k views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
253
votes
16answers
473k views

What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It ...
238
votes
5answers
358k views

What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
346
votes
79answers
162k views

What is your favorite “data analysis” cartoon?

This is one of my favorites: One entry per answer. (This is in the vein of the Stack Overflow question What’s your favorite “programmer” cartoon?.) P.S. Do not hotlink the cartoon without the site's ...
915
167
votes
76answers
213k views

Statistics Jokes

Well, we've got favourite statistics quotes. What about statistics jokes?
251
votes
150answers
131k views

Famous statistical quotations

What is your favorite statistical quote? This is community wiki, so please one quote per answer.
124
votes
5answers
475k views

How to choose between Pearson and Spearman correlation?

How do I know when to choose between Spearman's $\rho$ and Pearson's $r$? My variable includes satisfaction and the scores were interpreted using the sum of the scores. However, these scores could ...
189
votes
4answers
171k views

What does the hidden layer in a neural network compute?

I'm sure many people will respond with links to 'let me google that for you', so I want to say that I've tried to figure this out so please forgive my lack of understanding here, but I cannot figure ...
42
votes
3answers
484k views

How to interpret F- and p-value in ANOVA?

I am new to statistics and I currently deal with ANOVA. I carry out an ANOVA test in R using aov(dependendVar ~ IndependendVar) I get – among others – an F-value ...
168
votes
8answers
328k views

In linear regression, when is it appropriate to use the log of an independent variable instead of the actual values?

Am I looking for a better behaved distribution for the independent variable in question, or to reduce the effect of outliers, or something else?
178
votes
4answers
271k views

How to interpret a QQ plot

I am working with a small dataset (21 observations) and have the following normal QQ plot in R: Seeing that the plot does not support normality, what could I infer about the underlying distribution? ...
210
votes
13answers
196k views

What is the difference between data mining, statistics, machine learning and AI?

What is the difference between data mining, statistics, machine learning and AI? Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
213
votes
10answers
125k views

How would you explain covariance to someone who understands only the mean?

...assuming that I'm able to augment their knowledge about variance in an intuitive fashion ( Understanding "variance" intuitively ) or by saying: It's the average distance of the data ...
179
votes
2answers
254k views

When (and why) should you take the log of a distribution (of numbers)?

Say I have some historical data e.g., past stock prices, airline ticket price fluctuations, past financial data of the company... Now someone (or some formula) comes along and says "let's take/use ...
373
votes
5answers
120k views

How to understand the drawbacks of K-means

K-means is a widely used method in cluster analysis. In my understanding, this method does NOT require ANY assumptions, i.e., give me a dataset and a pre-specified number of clusters, k, and I just ...
242
votes
11answers
168k views

How would you explain Markov Chain Monte Carlo (MCMC) to a layperson?

Maybe the concept, why it's used, and an example.
143
votes
7answers
213k views

What is the influence of C in SVMs with linear kernel?

I am currently using an SVM with a linear kernel to classify my data. There is no error on the training set. I tried several values for the parameter $C$ ($10^{-5}, \dots, 10^2$). This did not ...
99
votes
10answers
267k views

What's the difference between correlation and simple linear regression?

In particular, I am referring to the Pearson product-moment correlation coefficient.
123
votes
3answers
262k views

What is the difference between linear regression and logistic regression?

What is the difference between linear regression and logistic regression? When would you use each?
421
votes
22answers
210k views

Why square the difference instead of taking the absolute value in standard deviation?

In the definition of standard deviation, why do we have to square the difference from the mean to get the mean (E) and take the square root back at the end? Can't we just simply take the absolute ...
169
votes
5answers
92k views

What should I do when my neural network doesn't learn?

I'm training a neural network but the training loss doesn't decrease. How can I fix this? I'm not asking about overfitting or regularization. I'm asking about how to solve the problem where my ...
126
votes
3answers
174k views

Help me understand Bayesian prior and posterior distributions

In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
112
votes
6answers
213k views

How would you explain the difference between correlation and covariance?

Following up on this question, How would you explain covariance to someone who understands only the mean?, which addresses the issue of explaining covariance to a lay person, brought up a similar ...
141
votes
2answers
188k views

How to determine which distribution fits my data best?

I have a dataset and would like to figure out which distribution fits my data best. I used the fitdistr() function to estimate the necessary parameters to ...
295
votes
7answers
266k views

When conducting multiple regression, when should you center your predictor variables & when should you standardize them?

In some literature, I have read that a regression with multiple explanatory variables, if in different units, needed to be standardized. (Standardizing consists in subtracting the mean and dividing ...
163
votes
6answers
104k views

How to choose a predictive model after k-fold cross-validation?

I am wondering how to choose a predictive model after doing K-fold cross-validation. This may be awkwardly phrased, so let me explain in more detail: whenever I run K-fold cross-validation, I use K ...
306
votes
18answers
110k views

What happens if the explanatory and response variables are sorted independently before regression?

Suppose we have data set $(X_i,Y_i)$ with $n$ points. We want to perform a linear regression, but first we sort the $X_i$ values and the $Y_i$ values independently of each other, forming data set $(...
355
votes
15answers
210k views

Bayesian and frequentist reasoning in plain English

How would you describe in plain English the characteristics that distinguish Bayesian from Frequentist reasoning?
21
votes
3answers
239k views

How to check for normal distribution using Excel for performing a t-test?

I want to know how to check a data set for normality in Excel, just to verify that the requirements for using a t-test are being met. For the right tail, is it appropriate to just calculate a mean ...
63
votes
10answers
101k views

Validation Error less than training error?

I found two questions here and here about this issue but there is no obvious answer or explanation yet.I enforce the same problem where the validation error is less than training error in my ...
39
votes
1answer
346k views

How to interpret and report eta squared / partial eta squared in statistically significant and non-significant analyses?

I have data that has eta squared values and partial eta squared values calculated as a measure of effect size for group mean differences. What is the difference between eta squared and partial eta ...
141
votes
15answers
70k views

Amazon interview question—probability of 2nd interview

I got this question during an interview with Amazon: 50% of all people who receive a first interview receive a second interview 95% of your friends that got a second interview felt they had a good ...
237
votes
5answers
174k views

Tradeoff batch size vs. number of iterations to train a neural network

When training a neural network, what difference does it make to set: batch size to $a$ and number of iterations to $b$ vs. batch size to $c$ and number of iterations to $d$ where $ ab = cd $? To ...
77
votes
10answers
303k views

Is there a minimum sample size required for the t-test to be valid?

I'm currently working on a quasi-experimental research paper. I only have a sample size of 15 due to low population within the chosen area and that only 15 fit my criteria. Is 15 the minimum sample ...
71
votes
4answers
206k views

How does the correlation coefficient differ from regression slope?

I would have expected the correlation coefficient to be the same as a regression slope (beta), however having just compared the two, they are different. How do they differ - what different information ...
136
votes
4answers
121k views

Cohen's kappa in plain English

I am reading a data mining book and it mentioned the Kappa statistic as a means for evaluating the prediction performance of classifiers. However, I just can't understand this. I also checked ...
251
votes
7answers
180k views

Bagging, boosting and stacking in machine learning

What's the similarities and differences between these 3 methods: Bagging, Boosting, Stacking? Which is the best one? And why? Can you give me an example for each?