# Tagged Questions

Hypothesis testing assesses whether data support a given hypothesis rather than being an effect of random fluctuations or some other process described by an alternative hypothesis.

13k views

### Is normality testing 'essentially useless'?

A former colleague once argued to me as follows: "we usually apply normality tests to the results of processes that, under the null, generate random variables that are only asymptotically or ...
135k views

### What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It ...
2k views

### What tests do I use to confirm that residuals are normally distributed?

I have some data which looks from plotting a graph of residuals vs time almost normal but I want to be sure. How can I test for normality of error residuals?
2k views

### How to interpret type I (sequential) ANOVA and MANOVA?

My primary question is how to interpret the output (coefficients, F, P) when conducting a Type I (sequential) ANOVA? My specific research problem is a bit more complex, so I will break my example into ...
10k views

### Significance of coefficients in linear regression: significant t-test vs non-significant F-statistic [duplicate]

I'm fitting a multiple linear regression model between 4 categorical variables (with 4 levels each) and a numerical output. My dataset has 43 observations. R gives me the following p-values from the ...
3k views

### Statistical inference when the sample “is” the population

Imagine you have to do reporting on the numbers of candidates who yearly take a given test. It seems rather difficult to infer the observed % of success, for instance, on a wider population due to the ...
8k views

### Are large data sets inappropriate for hypothesis testing?

In a recent article of Amstat News, the authors (Mark van der Laan and Sherri Rose) stated that "We know that for large enough sample sizes, every study—including ones in which the null hypothesis of ...
2k views

### How to test hypothesis of no group differences?

Imagine you have a study with two groups (e.g., males and females) looking at a numeric dependent variable (e.g., intelligence test scores) and you have the hypothesis that there are no group ...
6k views

### Why Levene test of equality of variances rather than F ratio?

SPSS uses the Levene test to evaluate homogeneity of variances in the independent group t-test procedure. Why is the Levene test better than a simple F ratio of the ratio of the variances of the ...
2k views

### What are examples where a “naive bootstrap” fails?

Suppose I have a set of sample data from an unknown or complex distribution, and I want to perform some inference on a statistic $T$ of the data. My default inclination is to just generate a bunch of ...
567 views

### Testing random variate generation algorithms

Which methods are used for testing random variate generation algorithms?
4k views

### How can I calculate margin of error in a NPS (Net Promoter Score) result?

I'll let Wikipedia explain how NPS is calculated: The Net Promoter Score is obtained by asking customers a single question on a 0 to 10 rating scale, where 10 is "extremely likely" and 0 is ...
747 views

### Testing for moderation with continuous vs. categorical moderators

I am testing an interaction effect where X and Y are continuous variable and M (Moderator) is a categorical variable (effects coding +1, -1). I have no clue about how to do a post-hoc probing of ...
716 views

### Effect size as the hypothesis for significance testing

Today, at the Cross Validated Journal Club (why weren't you there?), @mbq asked: Do you think we (modern data scientists) know what significance means? And how it relates to our confidence in our ...
2k views

### F and t statistics in a regression

In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests? In my model, there are 10 ...
999 views

### Wald test in regression (OLS and GLMs): t- vs. z-distribution

I understand that the Wald test for regression coefficients is based on the following property that holds asymptotically (e.g. Wasserman (2006): All of Statistics, pages 153, 214-215):  ...
1k views

### Understanding p-value

I know that there are lots of materials explaining p-value. However the concept is not easy to grasp firmly without further clarification. Here is the definition of p-value from Wikipedia: The ...
5k views

### Appropriate normality tests for small samples

Title says enough... so far, I've been using Shapiro-Wilk's statistic in order to test normality assumptions in small samples. Could you, please, recommend another technique?
2k views

### When to use Fisher and Neyman-Pearson framework?

I've been reading a lot lately about the differences between Fisher's method of hypothesis testing and the Neyman-Pearson school of thought. My question is, ignoring philosophical objections for a ...
1k views

### Interpretation of p-value in hypothesis testing

I recently came across the paper "The Insignificance of Null Hypothesis Significance Testing", Jeff Gill (1999). The author raised a few common misconceptions regarding hypothesis testing and ...
461 views

### Is there a statistical test to compare two samples of size 1 and 3?

For an ecology project, my lab group added vinegar to 4 tanks containing equal volumes of pond water, 1 control with no elodea (an aquatic plant) and 3 treatments with the same amount of elodea in ...
2k views

### What test can I use to compare slopes from two or more regression models?

I would like to test the difference in response of two variables to one predictor. Here is a minimal reproducible example. ...
2k views

### Regarding p-values, why 1% and 5%? Why not 6% or 10%?

Regarding p-values, I am wondering why 1% and 5% seem to be the gold standard for "statistical significance". Why not other values, like 6% or 10%? Is there a fundamental mathematical reason for ...
511 views

### Testing data against a known distribution

I asked a question similar to this a while ago, and the general answer was "your question is too vague". So let me try again with a little more detail... I have written a program which generates ...
4k views

### Can chi square be used to compare proportions?

I've read that the chi square test is useful to see if a sample is significantly different from a set of expected values. For example, here is a table of results of a survey regarding people's ...
1k views

### How can I test the fairness of a d20?

How can I test the fairness of a twenty sided die (d20)? Obviously I would be comparing the distribution of values against a uniform distribution. I vaguely remember using a Chi-square test in ...
15k views

### Chi-Squared always a one-sided test?

A published article contains these 2 sentences: "Moreover, misreporting may be caused by the application of incorrect rules or by a lack of knowledge of the statistical test. For example, the total ...
2k views

### Relation between confidence interval and testing statistical hypothesis for t-test

It is well known that confidence intervals and testing statistical hypothesis are strongly related. My questions is focused on comparison of means for two groups based on a numerical variable. Let's ...
1k views

### What to do when the means of two samples are significantly different but the difference seems too small to matter

I have two samples ($n \approx 70$ in both cases). The means differ by about twice the pooled std. dev. The resulting $T$ value is approximately 10. Whilst it's great to know that I have conclusively ...
371 views

### Desired effect size vs. expected effect size

When estimating the effect size for determining optimal sample size for a statistical test, the textbook approach seems to do this from pilot studies or the like. Is it also legal to define effect ...
876 views

### Test equivalence of non-nested models

Let's say $y$ is a linear function of $x$ and a dummy $d$. My hypothesis is that $d$ itself is like a hedonistic index of a vector of other variables, $Z$. I have support for this in a $MANOVA$ of $Z$ ...
4k views

### How to test whether a sample of data fits the family of Gamma distribution?

I have a sample of data which was generated from a continuous random variable X. And from the histogram I draw using R, I guess that maybe the distribution of X obeys a certain Gamma distribution. But ...
2k views

### How do I compare correlation coefficients of the same variables across different groups?

I am testing the correlation between Variable A and Variable B by gender and marital status. So I have the coefficients for males and females and single, married and other. I want to compare the ...
1k views

### What are good references containing arguments against null hypothesis significance testing?

In the last few years I've read a number of papers arguing against the use of null hypothesis significance testing in science, but didn't think to keep a persistent list. A colleague recently asked me ...
3k views

### Is it possible to prove a null hypothesis?

As the question states - Is it possible to prove the null hypothesis? From my (limited) understanding of hypothesis, the answer is no but I can't come up with a rigorous explanation for it. Does the ...
784 views

### Justification of one-tailed hypothesis testing

I understand two-tailed hypothesis testing. You have $H_0 : \theta = \theta_0$ (vs. $H_1 = \neg H_0 : \theta \ne \theta_0$). The p-value is the probability that $\theta$ generates data at least as ...
2k views

### Testing significance of peaks in spectral density

We sometimes use spectral density plot to analyze periodicity in time series. Normally we analyze the plot by visual inspection and then try to draw a conclusion about the periodicity. But has the ...
357 views

### Choosing a statistical test based on the outcome of another (e.g. normality)

So I have heard it said that it is not a good idea to choose one statistical test based on the outcome of another. This seems strange to me though. For example, people often choose to use a non ...
841 views

### Test for finite variance?

Is it possible to test for finiteness (or existence) of the variance of a random variable given a sample? As a null, either {the variance exists and is finite} or {the variance does not exist/is ...
2k views

### What is the difference between “testing of hypothesis” and “test of significance”?

Is there a difference between the phrases "testing of hypothesis" and "test of significance" or are they the same? After a detailed answer from @Micheal Lew, I have one confusion that nowadays ...
10k views

### What is the difference between confidence intervals and hypothesis testing?

I have read about controversies regarding hypothesis testing with some commentators suggesting that hypothesis testing should not be used. Some commentators suggest that confidence intervals should ...
3k views

### On univariate outlier tests (or: Dixon Q versus Grubbs)

In (most of) the analytical chemistry literature, the standard test for detecting outliers in univariate data (e.g. a sequence of measurements of some parameter) is Dixon's Q test. Invariably, all the ...
381 views

### How to test for simultaneous equality of choosen coefficients in logit or probit model?

How to test for simultaneous equality of choosen coefficients in logit or probit model ? What is the standard approach and what is the state of art approach ?
3k views

### Is Kolmogorov-Smirnov test valid with discrete distributions?

I'm comparing a sample and checking whether it distributes as some, discrete, distribution. However, I'm not enterily sure that Kolmogorov-Smirnov applies. Wikipedia seems to imply it does not. If it ...
9k views

### Difference between one-tailed and two-tailed testing?

While studying for my stats course, I was trying to understand the difference between one-tailed and two-tailed hypothesis tests. Specifically, why does the one-tailed test reject the null while the ...
94 views

### Is it possible to use a two sample $t$ test here?

I want to do statistical analysis to compare the results of the different specimen sizes (which I am comparing) with each other. Seeing as I have at least 12 specimens for each specimen size, I ...
1k views

### Hypothesis testing and total variation distance vs. Kullback-Leibler divergence

In my research I have run into the following general problem: I have two distributions $P$ and $Q$ over the same domain, and a large (but finite) number of samples from those distributions. Samples ...
2k views

### How can you have a non-significant multiple regression model w/ significant predictors? [duplicate]

Possible Duplicate: Not-significant F but a significant coefficient in multiple linear regression How can a regression be significant yet all predictors be non-significant? Significance of ...