All Questions

Filter by
Sorted by
Tagged with
243 votes
11 answers

Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
Tim's user avatar
  • 140k
255 votes
8 answers

Algorithms for automatic model selection

I would like to implement an algorithm for automatic model selection. I am thinking of doing stepwise regression but anything will do (it has to be based on linear regressions though). My problem ...
S4M's user avatar
  • 2,716
378 votes
15 answers

Is normality testing 'essentially useless'?

A former colleague once argued to me as follows: We usually apply normality tests to the results of processes that, under the null, generate random variables that are only asymptotically or ...
shabbychef's user avatar
  • 14.9k
132 votes
5 answers

Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?

TL;DR See title. Motivation I am hoping for a canonical answer along the lines of "(1) No, (2) Not applicable, because (1)", which we can use to close many wrong questions about unbalanced ...
Stephan Kolassa's user avatar
365 votes
9 answers

What should I do when my neural network doesn't learn?

I'm training a neural network but the training loss doesn't decrease. How can I fix this? I'm not asking about overfitting or regularization. I'm asking about how to solve the problem where my network'...
Sycorax's user avatar
  • 92.6k
299 votes
15 answers

What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It ...
Sharpie's user avatar
  • 4,414
88 votes
4 answers

Reduce Classification Probability Threshold

I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
sdgaw erzswer's user avatar
185 votes
6 answers

Can a probability distribution value exceeding 1 be OK?

On the Wikipedia page about naive Bayes classifiers, there is this line: $p(\mathrm{height}|\mathrm{male}) = 1.5789$ (A probability distribution over 1 is OK. It is the area under the bell curve ...
babelproofreader's user avatar
292 votes
3 answers

How to know that your machine learning problem is hopeless?

Imagine a standard machine-learning scenario: You are confronted with a large multivariate dataset and you have a pretty blurry understanding of it. What you need to do is to make predictions ...
Tim's user avatar
  • 140k
211 votes
9 answers

How to deal with perfect separation in logistic regression?

If you have a variable which perfectly separates zeroes and ones in target variable, R will yield the following "perfect or quasi perfect separation" warning message: ...
user333's user avatar
  • 7,271
119 votes
8 answers

What is the benefit of breaking up a continuous predictor variable?

I'm wondering what the value is in taking a continuous predictor variable and breaking it up (e.g., into quintiles), before using it in a model. It seems to me that by binning the variable we lose ...
Tom's user avatar
  • 1,801
1351 votes
27 answers

Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...
claws's user avatar
  • 13.8k
342 votes
13 answers

How to understand degrees of freedom?

From Wikipedia, there are three interpretations of the degrees of freedom of a statistic: In statistics, the number of degrees of freedom is the number of values in the final calculation of a ...
Tim's user avatar
  • 19.6k
395 votes
12 answers

Difference between logit and probit models

What is the difference between Logit and Probit model? I'm more interested here in knowing when to use logistic regression, and when to use Probit. If there is any literature which defines it using ...
Beta's user avatar
  • 6,386
103 votes
1 answer

What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?

The Mean Absolute Percentage Error (mape) is a common accuracy or error measure for time series or other predictions, $$ \text{MAPE} = \frac{100}{n}\sum_{t=1}^n\frac{|A_t-F_t|}{A_t}\%,$$ where $A_t$ ...
Stephan Kolassa's user avatar
110 votes
10 answers

What is meant by a "random variable"?

What do they mean when they say "random variable"?
Baltimark's user avatar
  • 2,288
183 votes
11 answers

When is it ok to remove the intercept in a linear regression model?

I am running linear regression models and wondering what the conditions are for removing the intercept term. In comparing results from two different regressions where one has the intercept and the ...
analyticsPierce's user avatar
29 votes
2 answers

Is accuracy an improper scoring rule in a binary classification setting?

I have recently been learning about proper scoring rules for probabilistic classifiers. Several threads on this website have made a point of emphasizing that accuracy is an improper scoring rule and ...
Zyzzva's user avatar
  • 291
307 votes
16 answers

Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

It seems that through various related questions here, there is consensus that the "95%" part of what we call a "95% confidence interval" refers to the fact that if we were to exactly replicate our ...
Mike Lawrence's user avatar
108 votes
6 answers

Principled way of collapsing categorical variables with many levels?

What techniques are available for collapsing (or pooling) many categories to a few, for the purpose of using them as an input (predictor) in a statistical model? Consider a variable like college ...
shadowtalker's user avatar
  • 12.8k
67 votes
5 answers

Best practice when analysing pre-post treatment-control designs

Imagine the following common design: 100 participants are randomly allocated to either a treatment or a control group the dependent variable is numeric and measured pre- and post- treatment Three ...
Jeromy Anglim's user avatar
288 votes
6 answers

Is $R^2$ useful or dangerous?

I was skimming through some lecture notes by Cosma Shalizi (in particular, section 2.1.1 of the second lecture), and was reminded that you can get very low $R^2$ even when you have a completely linear ...
raegtin's user avatar
  • 10.1k
412 votes
7 answers

When conducting multiple regression, when should you center your predictor variables & when should you standardize them?

In some literature, I have read that a regression with multiple explanatory variables, if in different units, needed to be standardized. (Standardizing consists in subtracting the mean and dividing ...
mathieu_r's user avatar
  • 4,531
436 votes
9 answers

What is the difference between fixed effect, random effect in mixed effect models?

In simple terms, how would you explain (perhaps with simple examples) the difference between fixed effect, random effect in mixed effect models?
Andrew's user avatar
  • 6,278
129 votes
18 answers

Including the interaction but not the main effects in a model

Is it ever valid to include a two-way interaction in a model without including the main effects? What if your hypothesis is only about the interaction, do you still need to include the main effects?
Glen's user avatar
  • 7,450
103 votes
8 answers

When is unbalanced data really a problem in Machine Learning?

We already had multiple questions about unbalanced data when using logistic regression, SVM, decision trees, bagging and a number of other similar questions, what makes it a very popular topic! ...
Tim's user avatar
  • 140k
29 votes
2 answers

Proper scoring rule when there is a decision to make (e.g. spam vs ham email)

Among others on here, Frank Harrell is adamant about using proper scoring rules to assess classifiers. This makes sense. If we have 500 $0$s with $P(1)\in[0.45, 0.49]$ and 500 $1$s with $P(1)\in[0.51, ...
Dave's user avatar
  • 65.1k
218 votes
5 answers

How exactly does one “control for other variables”?

Here is the article that motivated this question: Does impatience make us fat? I liked this article, and it nicely demonstrates the concept of “controlling for other variables” (IQ, career, income, ...
JackOfAll's user avatar
  • 2,997
125 votes
3 answers

Does an unbalanced sample matter when doing logistic regression?

Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables. My problem is the following: ...
Michiel's user avatar
  • 1,353
619 votes
5 answers

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
amoeba's user avatar
  • 106k
140 votes
3 answers

What if residuals are normally distributed, but y is not?

I've got a weird question. Assume that you have a small sample where the dependent variable that you're going to analyze with a simple linear model is highly left skewed. Thus you assume that $u$ is ...
MarkDollar's user avatar
  • 5,993
133 votes
14 answers

Maximum Likelihood Estimation (MLE) in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
StatsUser's user avatar
  • 1,819
196 votes
1 answer

Crossed vs nested random effects: how do they differ and how are they specified correctly in lme4?

Here is how I have understood nested vs. crossed random effects: Nested random effects occur when a lower level factor appears only within a particular level of an upper level factor. For ...
Joe King's user avatar
  • 3,873
146 votes
4 answers

Is it possible to have a pair of Gaussian random variables for which the joint distribution is not Gaussian?

Somebody asked me this question in a job interview and I replied that their joint distribution is always Gaussian. I thought that I can always write a bivariate Gaussian with their means and variance ...
MarkSAlen's user avatar
  • 2,967
106 votes
10 answers

What is a complete list of the usual assumptions for linear regression?

What are the usual assumptions for linear regression? Do they include: a linear relationship between the independent and dependent variable independent errors normal distribution of errors ...
tony's user avatar
  • 1,069
243 votes
4 answers

When (and why) should you take the log of a distribution (of numbers)?

Say I have some historical data e.g., past stock prices, airline ticket price fluctuations, past financial data of the company... Now someone (or some formula) comes along and says "let's take/use ...
PhD's user avatar
  • 14.8k
226 votes
8 answers

In linear regression, when is it appropriate to use the log of an independent variable instead of the actual values?

Am I looking for a better behaved distribution for the independent variable in question, or to reduce the effect of outliers, or something else?
d_2's user avatar
  • 2,421
27 votes
1 answer

Goodness of fit and which model to choose linear regression or Poisson

I need some advice regarding two main dilemmas in my research, which is a case study of 3 big pharmaceuticals and innovation. Number of patents per year is the dependent variable. My questions are ...
Nitzan's user avatar
  • 371
230 votes
3 answers

R's lmer cheat sheet

There's a lot of discussion going on on this forum about the proper way to specify various hierarchical models using lmer. I thought it would be great to have all ...
72 votes
2 answers

How to interpret type I, type II, and type III ANOVA and MANOVA?

My primary question is how to interpret the output (coefficients, F, P) when conducting a Type I (sequential) ANOVA? My specific research problem is a bit more complex, so I will break my example ...
djhocking's user avatar
  • 1,941
60 votes
5 answers

Generic sum of Gamma random variables

I have read that the sum of Gamma random variables with the same scale parameter is another Gamma random variable. I've also seen the paper by Moschopoulos describing a method for the summation of a ...
OSE's user avatar
  • 1,227
213 votes
7 answers

PCA on correlation or covariance?

What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
Random's user avatar
  • 2,310
31 votes
1 answer

Omitted variable bias in logistic regression vs. omitted variable bias in ordinary least squares regression

I have a question about omitted variable bias in logistic and linear regression. Say I omit some variables from a linear regression model. Pretend that those omitted variables are uncorrelated with ...
ConfusedEconometricsUndergrad's user avatar
63 votes
3 answers

What is the intuition behind conditional Gaussian distributions?

Suppose that $\mathbf{X} \sim N_{2}(\mathbf{\mu}, \mathbf{\Sigma})$. Then the conditional distribution of $X_1$ given that $X_2 = x_2$ is multivariate normally distributed with mean: $$ E[P(X_1 | ...
eroeijr's user avatar
  • 631
44 votes
3 answers

Relation between confidence interval and testing statistical hypothesis for t-test

It is well known that confidence intervals and testing statistical hypothesis are strongly related. My questions is focused on comparison of means for two groups based on a numerical variable. Let's ...
Lan's user avatar
  • 1,419
562 votes
23 answers

Why square the difference instead of taking the absolute value in standard deviation?

In the definition of standard deviation, why do we have to square the difference from the mean to get the mean (E) and take the square root back at the end? Can't we just simply take the absolute ...
c4il's user avatar
  • 5,875
137 votes
7 answers

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical ...
Alec's user avatar
  • 2,395
678 votes
12 answers

What is the difference between "likelihood" and "probability"?

The wikipedia page claims that likelihood and probability are distinct concepts. In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a ...
Douglas S. Stones's user avatar
140 votes
8 answers

How to choose between t-test or non-parametric test e.g. Wilcoxon in small samples

Certain hypotheses can be tested using Student's t-test (maybe using Welch's correction for unequal variances in the two-sample case), or by a non-parametric test like the Wilcoxon paired signed rank ...
Silverfish's user avatar
  • 23.8k
61 votes
4 answers

Why do statisticians say a non-significant result means "you can't reject the null" as opposed to accepting the null hypothesis?

Traditional statistical tests, like the two sample t-test, focus on trying to eliminate the hypothesis that there is no difference between a function of two independent samples. Then, we choose a ...
ryu576's user avatar
  • 2,600

15 30 50 per page
2 3 4 5