All Questions
26,259
questions
43
votes
3
answers
16k
views
Relation between confidence interval and testing statistical hypothesis for t-test
It is well known that confidence intervals and testing statistical hypothesis are strongly related. My questions is focused on comparison of means for two groups based on a numerical variable. Let's ...
- 1,371
236
votes
4
answers
457k
views
How to interpret a QQ plot
I am working with a small dataset (21 observations) and have the following normal QQ plot in R:
Seeing that the plot does not support normality, what could I infer about the underlying distribution? ...
- 19.4k
200
votes
8
answers
49k
views
What intuitive explanation is there for the central limit theorem?
In several different contexts we invoke the central limit theorem to justify whatever statistical method we want to adopt (e.g., approximate the binomial distribution by a normal distribution). I ...
user28
146
votes
9
answers
304k
views
What is the difference between linear regression on y with x and x with y?
The Pearson correlation coefficient of x and y is the same, whether you compute pearson(x, y) or pearson(y, x). This suggests that doing a linear regression of y given x or x given y should be the ...
- 3,203
133
votes
7
answers
112k
views
How to choose between t-test or non-parametric test e.g. Wilcoxon in small samples
Certain hypotheses can be tested using Student's t-test (maybe using Welch's correction for unequal variances in the two-sample case), or by a non-parametric test like the Wilcoxon paired signed rank ...
- 22.1k
117
votes
4
answers
37k
views
Assessing approximate distribution of data based on a histogram
Suppose I want to see whether my data is exponential based on a histogram (i.e. skewed to the right).
Depending on how I group or bin the data, I can get wildly different histograms.
One set of ...
- 1,171
77
votes
5
answers
39k
views
How can adding a 2nd IV make the 1st IV significant?
I have what is probably a simple question, but it is baffling me right now, so I am hoping you can help me out.
I have a least squares regression model, with one independent variable and one ...
- 1,150
63
votes
3
answers
75k
views
Interpretation of log transformed predictor and/or response
I'm wondering if it makes a difference in interpretation whether only the dependent, both the dependent and independent, or only the independent variables are log transformed.
Consider the case of
<...
- 2,807
176
votes
10
answers
114k
views
Bottom to top explanation of the Mahalanobis distance?
I'm studying pattern recognition and statistics and almost every book I open on the subject I bump into the concept of Mahalanobis distance. The books give sort of intuitive explanations, but still ...
- 5,617
79
votes
6
answers
17k
views
Optimization when Cost Function Slow to Evaluate
Gradient descent and many other methods are useful for finding local minima in cost functions. They can be efficient when the cost function can be evaluated quickly at each point, whether numerically ...
- 1,013
64
votes
5
answers
53k
views
What should I do when my neural network doesn't generalize well?
I'm training a neural network and the training loss decreases, but the validation loss doesn't, or it decreases much less than what I would expect, based on references or experiments with very similar ...
- 16.9k
56
votes
3
answers
15k
views
Consider the sum of $n$ uniform distributions on $[0,1]$, or $Z_n$. Why does the cusp in the PDF of $Z_n$ disappear for $n \geq 3$?
I've been wondering about this one for a while; I find it a little weird how abruptly it happens. Basically, why do we need just three uniforms for $Z_n$ to smooth out like it does? And why does the ...
- 1,436
277
votes
11
answers
160k
views
How would you explain covariance to someone who understands only the mean?
...assuming that I'm able to augment their knowledge about variance in an intuitive fashion ( Understanding "variance" intuitively ) or by saying: It's the average distance of the data ...
- 14.2k
32
votes
1
answer
13k
views
How do you deal with "nested" variables in a regression model?
Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory ...
- 112k
553
votes
15
answers
234k
views
What is the intuition behind beta distribution?
Disclaimer: I'm not a statistician but a software engineer. Most of my knowledge in statistics comes from self-education, thus I still have many gaps in understanding concepts that may seem trivial ...
- 9,720
261
votes
15
answers
288k
views
What are the differences between Factor Analysis and Principal Component Analysis?
It seems that a number of the statistical packages that I use wrap these two concepts together. However, I'm wondering if there are different assumptions or data 'formalities' that must be true to use ...
- 6,892
30
votes
3
answers
11k
views
Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value?
I've fitted my model and am trying to understand whether it's any good. I've calculated the recommended metrics to assess it ($R^2$/ AUC / accuracy / prediction error / etc) but do not know how to ...
- 15.9k
557
votes
11
answers
645k
views
What is the difference between test set and validation set?
I found this confusing when I use the neural network toolbox in Matlab.
It divided the raw data set into three parts:
training set
validation set
test set
I notice in many training or learning ...
- 7,099
95
votes
4
answers
40k
views
What is an "uninformative prior"? Can we ever have one with truly no information?
Inspired by a comment from this question:
What do we consider "uninformative" in a prior - and what information is still contained in a supposedly uninformative prior?
I generally see the prior in ...
- 21.9k
73
votes
5
answers
27k
views
How small a quantity should be added to x to avoid taking the log of zero?
I have analysed my data as they are. Now I want to look at my analyses after taking the log of all variables. Many variables contain many zeros. Therefore I add a small quantity to avoid taking the ...
- 3,584
57
votes
2
answers
17k
views
Is it unusual for the MEAN to outperform ARIMA?
I recently applied a range of forecasting methods (MEAN, RWF, ETS, ARIMA and MLPs) and found that MEAN did surprisingly well. (MEAN: where all future predictions are predicted as been equal to the ...
- 1,114
19
votes
1
answer
14k
views
Question on how to normalize regression coefficient
Not sure if normalize is the correct word to use here, but I will try my best to illustrate what I am trying to ask. The estimator used here is least squares.
Suppose you have $y=\beta_0+\beta_1x_1$, ...
- 799
60
votes
4
answers
37k
views
Box-Cox like transformation for independent variables?
Is there a Box-Cox like transformation for independent variables? That is, a transformation that optimizes the $x$ variable so that the y~f(x) will make a more ...
- 20.8k
53
votes
4
answers
50k
views
Does the sign of scores or of loadings in PCA or FA have a meaning? May I reverse the sign?
I performed principal component analysis (PCA) with R using two different functions (prcomp and princomp) and observed that the ...
- 987
36
votes
3
answers
26k
views
How to tell the difference between linear and non-linear regression models?
I was reading the following link on non linear regression SAS Non Linear. My understanding from reading the first section "Nonlinear Regression vs. Linear Regression" was that the equation below is ...
- 657
81
votes
0
answers
64k
views
How can a regression be significant yet all predictors be non-significant? [duplicate]
My multiple regression analysis model has a statistically significant F value however all beta values are statistically non-significant.
All the regression assumptions are met. No multicollinearity ...
- 811
36
votes
2
answers
34k
views
Interpretation of simple predictions to odds ratios in logistic regression
I'm somewhat new to using logistic regression, and a bit confused by a discrepancy between my interpretations of the following values which I thought would be the same:
exponentiated beta values
...
- 847
95
votes
1
answer
99k
views
When to use an offset in a Poisson regression? [duplicate]
Does anybody know why offset in a Poisson regression is used? What do you achieve by this?
- 5,785
59
votes
3
answers
40k
views
How to select a clustering method? How to validate a cluster solution (to warrant the method choice)?
One of the biggest issue with cluster analysis is that we may happen to have to derive different conclusion when base on different clustering methods used (including different linkage methods in ...
- 909
142
votes
3
answers
81k
views
Removal of statistically significant intercept term increases $R^2$ in linear model
In a simple linear model with a single explanatory variable,
$\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$
I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
- 2,252
193
votes
5
answers
70k
views
Training on the full dataset after cross-validation?
TL:DR: Is it ever a good idea to train an ML model on all the data available before shipping it to production? Put another way, is it ever ok to train on all data available and not check if the model ...
- 18.6k
88
votes
9
answers
83k
views
Why is it possible to get significant F statistic (p<.001) but non-significant regressor t-tests?
In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests?
In my model, there are 10 ...
- 1,529
440
votes
5
answers
171k
views
How to understand the drawbacks of K-means
K-means is a widely used method in cluster analysis. In my understanding, this method does NOT require ANY assumptions, i.e., give me a dataset and a pre-specified number of clusters, k, and I just ...
- 6,709
278
votes
2
answers
227k
views
Interpretation of R's lm() output
The help pages in R assume I know what those numbers mean, but I don't.
I'm trying to really intuitively understand every number here. I will just post the output and comment on what I found out. ...
- 4,261
135
votes
4
answers
68k
views
Nested cross validation for model selection
How can one use nested cross validation for model selection?
From what I read online, nested CV works as follows:
There is the inner CV loop, where we may conduct a grid search (e.g. running K-fold ...
- 18.6k
75
votes
4
answers
79k
views
Why is sample standard deviation a biased estimator of $\sigma$?
According to the Wikipedia article on unbiased estimation of standard deviation the sample SD
$$s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}$$
is a biased estimator of the SD of the ...
- 759
313
votes
10
answers
184k
views
What's the difference between a confidence interval and a credible interval?
Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the difference between confidence intervals and credible intervals were the correct ones. How you would ...
- 5,837
236
votes
14
answers
197k
views
How should I transform non-negative data including zeros?
If I have highly skewed positive data I often take logs. But what should I do with highly skewed non-negative data that include zeros? I have seen two transformations used:
$\log(x+1)$ which has the ...
- 54.5k
85
votes
5
answers
34k
views
Central limit theorem for sample medians
If I calculate the median of a sufficiently large number of observations drawn from the same distribution, does the central limit theorem state that the distribution of medians will approximate a ...
- 1,047
60
votes
4
answers
41k
views
Confidence interval for Bernoulli sampling
I have a random sample of Bernoulli random variables $X_1 ... X_N$, where $X_i$ are i.i.d. r.v. and $P(X_i = 1) = p$, and $p$ is an unknown parameter.
Obviously, one can find an estimate for $p$: $\...
Oleg
108
votes
5
answers
208k
views
Loadings vs eigenvectors in PCA: when to use one or another?
In principal component analysis (PCA), we get eigenvectors (unit vectors) and eigenvalues. Now, let us define loadings as $$\text{Loadings} = \text{Eigenvectors} \cdot \sqrt{\text{Eigenvalues}}.$$
I ...
- 1,379
72
votes
9
answers
86k
views
Which pseudo-$R^2$ measure is the one to report for logistic regression (Cox & Snell or Nagelkerke)?
I have SPSS output for a logistic regression model. The output reports two measures for the model fit, Cox & Snell and ...
- 13.9k
176
votes
2
answers
186k
views
Deriving the conditional distributions of a multivariate normal distribution
We have a multivariate normal vector ${\boldsymbol Y} \sim \mathcal{N}(\boldsymbol\mu, \Sigma)$. Consider partitioning $\boldsymbol\mu$ and ${\boldsymbol Y}$ into
$$\boldsymbol\mu
=
\begin{bmatrix}
\...
- 6,079
104
votes
12
answers
18k
views
What, precisely, is a confidence interval?
I know roughly and informally what a confidence interval is. However, I can't seem to wrap my head around one rather important detail: According to Wikipedia:
A confidence interval does not ...
- 8,449
147
votes
6
answers
289k
views
Correlations with unordered categorical variables
I have a dataframe with many observations and many variables. Some of them are categorical (unordered) and the others are numerical.
I'm looking for associations between these variables. I've been ...
- 1,767
84
votes
5
answers
28k
views
Intuition on the Kullback–Leibler (KL) Divergence
I have learned about the intuition behind the KL Divergence as how much a model distribution function differs from the theoretical/true distribution of the data. The source I am reading goes on to say ...
- 8,547
64
votes
5
answers
240k
views
Correlations between continuous and categorical (nominal) variables
I would like to find the correlation between a continuous (dependent variable) and a categorical (nominal: gender, independent variable) variable. Continuous data is not normally distributed. Before, ...
229
votes
18
answers
247k
views
Intuitive explanation for dividing by $n-1$ when calculating standard deviation?
I was asked today in class why you divide the sum of square error by $n-1$ instead of with $n$, when calculating the standard deviation.
I said I am not going to answer it in class (since I didn't ...
- 20.8k
52
votes
1
answer
50k
views
Alternatives to one-way ANOVA for heteroskedastic data
I have data from 3 groups of algae biomass ($A$, $B$, $C$) which contain unequal sample sizes ($n_A=15$, $n_B=13$, $n_C=12$) and I would like compare if these groups are from the same population.
One-...
- 521
32
votes
3
answers
3k
views
I've heard that ratios or inverses of random variables often are problematic, in not having expectations. Why is that?
The title is the question. I am told that ratios and inverses of random variables often are problematic. What is meant is that expectation often do not exist. Is there a simple, general explication of ...
- 71.6k