All Questions
Tagged with hypothesis-testing inference
453 questions
119
votes
7
answers
214k
views
T-test for non normal when N>50?
Long ago I learnt that normal distribution was necessary to use a two sample T-test. Today a colleague told me that she learnt that for N>50 normal distribution was not necessary. Is that true?
If ...
- 2,397
76
votes
4
answers
140k
views
Testing equality of coefficients from two different regressions
This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this?
More formally, ...
- 2,046
69
votes
8
answers
8k
views
What is a good, convincing example in which p-values are useful?
My question in the title is self explanatory, but I would like to give it some context.
The ASA released a statement earlier this week “on p-values: context, process, and purpose”, outlining various ...
- 21.9k
64
votes
13
answers
11k
views
Two-tailed tests... I'm just not convinced. What's the point?
The following excerpt is from the entry, What are the differences between one-tailed and two-tailed tests?, on UCLA's statistics help site.
... consider the consequences of missing an effect in the ...
- 773
36
votes
2
answers
2k
views
Performing a statistical test after visualizing data - data dredging?
I'll propose this question by means of an example.
Suppose I have a data set, such as the boston housing price data set, in which I have continuous and categorical variables. Here, we have a "quality"...
- 1,410
32
votes
3
answers
3k
views
Why does basic hypothesis testing focus on the mean and not on the median?
In basic under-grad statistics courses, students are (usually?) taught hypothesis testing for the mean of a population.
Why is it that the focus is on the mean and not on the median? My guess is that ...
- 675
22
votes
7
answers
2k
views
Isn't it problematic to look at the data to decide to use a parametric vs. non-parametric test?
I've seen in some instances of people mentioning that using a parametric vs. non-parametric approach may be decided by looking at the data. For example this question: nonparametric vs. parametric
Isn'...
- 243
22
votes
3
answers
10k
views
Neyman-Pearson lemma
I have read the Neyman–Pearson lemma from the book Introduction to the Theory of Statistics by Mood, Graybill and Boes. But I have not understood the lemma.
Can anyone please explain the lemma to ...
- 1,705
20
votes
2
answers
4k
views
Elastic/ridge/lasso analysis, what then?
I'm getting really interested in the elastic net procedure for predictor shrinkage/selection. It seems very powerful.
But from the scientific point of view I don't know well what to do once I got the ...
- 2,939
18
votes
4
answers
23k
views
Do descriptive statistics have p-values?
I'm being asked to find the p-values for descriptive statistics. However, it's my understanding that p-values are for test statistics. If I'm not mistaken, a p-value is the probability of observing a ...
- 313
17
votes
3
answers
2k
views
Should "City" be a fixed or a random effect variable?
I am analyzing data on "BloodSugar" level (dependent variable) and trying to find its relation with "age", "gender" and "weight" (independent variables) of ...
- 10.2k
15
votes
6
answers
4k
views
Can you multiply p-values if you perform the same test multiple times?
I believe the interpretation of the p-value is that it is the probability of seeing your sample's test statistic under the null hypothesis.
But what happens if you perform the same exact test multiple ...
- 313
13
votes
8
answers
2k
views
Is descriptive statistics enough to compare test scores of students in a class?
I am reviewing the theory on hypothesis testing and the book I am reading ("Hypothesis Testing" by Jim Frost) stresses the fact that we do hypothesis testing and inferential statistic when ...
- 566
13
votes
2
answers
1k
views
How to define a Rejection Region when there's no UMP?
Consider the linear regression model
$\mathbf{y}=\mathbf{X\beta}+\mathbf{u}$,
$\mathbf{u}\sim N(\mathbf{0},\sigma^2\mathbf{I})$,
$E(\mathbf{u}\mid\mathbf{X})=\mathbf{0}$.
Let $H_0: \sigma_0^2=\...
- 5,882
12
votes
2
answers
4k
views
Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?
I have been taught that we can produce a parameter estimate in the form of a confidence interval after sampling from a population. For example, 95% confidence intervals, with no violated assumptions, ...
- 321
11
votes
2
answers
3k
views
What is a pivotal statistic?
I'm currently reading "Computer Age Statistical Inference" by Efron and Hastie.
In section 2.1, they talk about some of the mechanisms that frequentist inference uses to circumvent the defect of ...
- 309
11
votes
1
answer
1k
views
Hypothesis testing on tossing the coin n times
You toss the coin n times, and you have observed 60% of times, it is heads.
How large does n need to be in order to achieve 95% confidence that it is not a fair coin?
=======
Attempt: Basically use ...
- 317
11
votes
1
answer
2k
views
Hypothesis Testing and the Scientific Method
Reading the answers to this thread, I started wondering about how Hypothesis Testing relates to the Scientific Method. While I have a good understanding of both, I am having a hard time drawing the ...
- 19.7k
10
votes
4
answers
1k
views
Implications of current debate on statistical significance
In the past few years, various scholars have raised a detrimental problem of scientific hypothesis testing, dubbed "researcher degree of freedom," meaning that scientists have numerous choices to make ...
10
votes
2
answers
438
views
Reference request: Classical statistics for working data scientists
I'm a working data scientist with solid experience in regression, other machine learning type algorithms, and programming (both for data analysis and general software development). Most of my working ...
9
votes
2
answers
5k
views
How do the t-distribution and standard normal distribution differ, and why is t-distribution used more?
For statistical inference (e.g., hypothesis testing or computing confidence intervals), why do we use the t-distribution instead of the standard normal distribution? My class started with the standard ...
- 1,030
9
votes
1
answer
195
views
Question on Inference - Catching Cheating Students
In their paper "Catching cheating students", Levitt and Lin propose a simple reduced-form method to identify cheating of students in exams.
The strategy works as follows: For each possible pair of ...
- 101
8
votes
3
answers
855
views
Practical implication of failing to reject a null hypothesis
Consider a scenario where you are trying to measure a dosage of a medicine. The machine is calibrated to fill a mean dosage of 50mg. But for a reason you believe that machine's calibration is off. For ...
- 141
8
votes
3
answers
2k
views
Why does $\mu > 0$ (or even $\mu > \epsilon$) "seem easier” to substantiate than $\mu \neq 0$?
Consider a random variable $X$ following a normal distribution $N(\mu,\sigma^2)$. Suppose that we have drawn iid samples of $X$, obtaining a data set with a sample mean $\bar{x}>0$.
We want to test ...
- 138
8
votes
5
answers
6k
views
Likelihood Ratio for the Bivariate Normal distribution
For a random sample from a Bivariate Normal distribution with $\rho=\frac{1}{2}$ and equal variances, i.e. $\sigma^2_x=\sigma^2_y=\sigma^2$, I would like to derive the Likelihood Ratio Test for the ...
- 21.1k
8
votes
4
answers
2k
views
$\chi^2$ tabulated value
I noticed that the critical $\chi^2$ value increases as the degrees of freedom increase in a $\chi^2$ table. Why is that?
- 81
8
votes
1
answer
14k
views
what does it mean by more "efficient" estimator
When comparing two estimators, say $T_1$ and $T_2$, what does it mean by saying $T_1$ is more efficient than $T_2$?
Could someone give an easy but very concrete example?
Also I have another ...
- 649
8
votes
2
answers
6k
views
Is there a test/technique/method for comparing principal components decompositions between samples?
Is there any methodical way to compare the directions, magnitudes, etc of PCA results for different samples drawn from the same population?
I'm leaving the nature of the test deliberately vague ...
- 12.8k
8
votes
1
answer
2k
views
Understanding how to find more "extreme" values when calculating p values in two sided hypothesis tests
In hypothesis testing, the definition of p value is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is ...
user279822
7
votes
3
answers
467
views
If we disbelieve $H_0$, why quote a p value calculated assuming $H_0$ was true?
Hypothesis testing seeks to reject a null hypothesis ($H_0$) on the basis of an assumption made about the sample following a certain distribution. This assumption is conditional on $H_0$ being true. ...
- 1,427
6
votes
3
answers
1k
views
What to conclude when most results are statistically significant to fail to reject null hypothesis but not all?
I have sampled 8 bags of a certain brand of candy to compare the color distributions of the candies. I have 4 bags for each size of bag, 8 oz and 1.9 lb. The bags were paired randomly. Here are my ...
6
votes
3
answers
427
views
The rationale for when significance or null hypothesis testing is needed
Why do people sometimes claim that an effect is so huge and "obvious" that it does not warrant any inferential statistics calculation, even though the sample size is not large?
This is ...
- 175
6
votes
2
answers
366
views
Frequentist inference with a null hypothesis that reflects theory a good-enough belt around it
TL;DR:
With frequentist statistics, does it make sense to 1) no longer use significance testing, 2) set the point null hypothesis to reflect theory and decide a priori when to refute it, and 3) use a ...
- 319
6
votes
2
answers
793
views
Is it true that we can always increase statistical power/estimator precision by increasing sample size?
Suppose a test has ~$16.67\%$ power to detect some arbitrary but fixed effect size when sample size is $3$, and as we increase size by adding IID random observations to the sample ${4, 5, 6, 7,...}$ ...
- 642
6
votes
1
answer
8k
views
Why no degrees of freedom for Z test
I am reading about hypothesis testing and have encountered Z test and T test. I have understood both the tests and their usage. One part that is not still clear is why does Z test does not depend on ...
- 275
6
votes
1
answer
4k
views
Ways to find a UMP test
I'm studying for my final exams and the subject of proof will basically test hypotheses, I will try to summarize here my doubts.
For found the UMP test the ways are
1) Use Neyman–Pearson lemma ...
user72621
6
votes
2
answers
484
views
Relevance of the homoscedasticity assumption
I understand that the homoscedasticity assumption is one of the Gauss Markov assumptions to get a BLUE estimator.
Why is homoscedasticity crucial for justifying the usual t and F statistics?
- 285
6
votes
1
answer
544
views
Likelihood ratio test for $H_0:(\mu_1,\mu_2)=(0,0)$ vs $H_1:(\mu_1,\mu_2) \neq (0,0)$
There are $X_1, X_2$ where $X_i \sim N(\mu_i,1), i=1,2$. They are independent. The question is
Find the likelihood ratio test with $H_0:(\mu_1,\mu_2)=(0,0), H_1:(\mu_1,\mu_2) \neq (0,0)$. The ...
- 478
5
votes
4
answers
356
views
Testing Hypothesis with different alternatives
I want to test whether $\mu=\mu_0$ where $\mu_0$ is some fixed number.
Consider the following two different testings.
Hypothesis Testing 1: $H_0:\mu=\mu_0,H_1:\mu<\mu_0$
Hypothesis Testing 2: $H_0:\...
- 1,465
5
votes
1
answer
1k
views
We flip a coin 20 times and observe 12 heads. What is the probability that the coin is fair?
im having some trouble getting around this. A little explanation would be really helpful.
- 51
5
votes
2
answers
257
views
Term for "extent to which a test throws away information"?
A statistical test $T$ is a mapping from the space $\Delta$ of possible data $D$ to $\{R,A \}$, (meaning: Reject, Accept).
If we have a null hypothesis $H_0:\theta \in \Theta_0$ for which $T$ is a ...
- 2,987
5
votes
1
answer
653
views
Chi square test
I am reading about chi-square test but came across Chi square test of independence, Chi square test for goodness of fit and Chi square test of variance. I am confused and unable to find out what is ...
- 275
5
votes
2
answers
1k
views
Wald statistic with known mean and unknown variance in a Normal Distribution
I'm working from the Casella-Berger book and I've run across this problem,
I've managed to answer part (a) and most of part (b); however, upon looking in the solutions manual, I'm not able to get the ...
- 569
5
votes
1
answer
473
views
Hosmer-Lemeshow recommendations
During lectures I came across following statement:
If you want Hosmer-Lemeshow test to be valid, number of expected events ($E_1g$) should be >5 in most of $g$ groups
Then after few lectures, ...
- 273
5
votes
2
answers
815
views
Unknown process outputs binary results, how to prove that this process is (or not) a Bernoulli trial
I have an unknown process that produces binary results. I am trying to determine if this process is a Bernoulli trial.
From wikipedia:
In the theory of probability and statistics, a Bernoulli ...
- 86
5
votes
2
answers
337
views
Should the Wilcoxon Rank sum test be used for testing the mean difference significance?
I was looking replicate the results of the paper DOI:10.3905/jpm.2014.40.3.087 (Exploring Macroeconomic Sensitivities: How Investments Respond to Different Economic Environments, Ilmanen Maloney Ross ...
- 77
5
votes
2
answers
929
views
Is the Likelihood Ratio test using cluster robust standard errors fixable by Bootstrap (or someting else)?
There is a common agreement about the invalidity of using likelihood ratio tests when computing Maximum Likelihood Estimates (MLE) using clustered corrected standard errors. The main argument is that ...
- 219
5
votes
1
answer
305
views
Approximate the critical region such that the size of the test tends to $\alpha$
Consider this question,
Suppose $X_1, X_2, . . . , X_n$ is a random sample from an exponential distribution with mean $\lambda$. Assume that the observed data is available on $[X_1], . . . , [X_n]$,...
- 353
5
votes
1
answer
899
views
Can an independent t-test be used on paired data when the pairing is unknown?
Suppose the effectiveness of a training course is examined, and performance of each individual in a group is taken both before and after, and the differences are compared in a paired $t$-test.
Would ...
4
votes
5
answers
3k
views
Is there a statistical test for one participant measured many times?
Pretty much throughout my undergrad and postgrad, I have always learned statistical models predicated upon things like large subject size. I also know that a lot of repeated measures designs typically ...
- 18.4k