Linked Questions

246
votes
16answers
463k views

What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It ...
4
votes
1answer
79 views

Am I taking crazy pills here or this A/B testing tool's documentation dead wrong?

In the documentation here for a popular A/B testing they say, For example, if you run an A/B testing experiment with a significance level of 95%, this means that if you determine a winner, you ...
1
vote
1answer
51 views

Interpretation of the D-statistic from 2-sample KS-tests

I've been using the two sample Kolmogorov-Smirnov test to compare observations from 2018 vs observations from 2019. I heard from my co-worker that you can interpret the D-statistic as the magnitude ...
2
votes
2answers
7k views

interpreting confidence intervals in t.test

I'm trying to run a t-test using t.test() but am confused about the p-value and confidence interval. Looking at the output I see a p-value of ~0.03. That leads me ...
5
votes
1answer
15k views

Can p-value be greater than 1?

Say I perform Student's t-test: can the p-value be greater than 1? The reason I ask this is that, some say that the p-value is the probability that we will have a Type I error and I would like to ...
8
votes
5answers
2k views

What is the relationship between $p$ values and Type I errors [duplicate]

In hypothesis testing we set an accepted level of Type I error probability $\alpha$ and observe whether a sample statistic is equally likely or less likely to be observed if the null hypothesis was ...
4
votes
3answers
2k views

Why do we use the chi-squared test to test independence of 2 categorical variables?

I sometimes use the chi-squared test to test the independence of categorical variables. I know how to do the test but don't know the underlying concepts and details. So, I was wondering about the ...
14
votes
4answers
4k views

Probability that Null Hypothesis is True

So, this may be a common question, but I’ve never found a satisfactory answer. How do you determine the probability that the null hypothesis is true (or false)? Let’s say you give students two ...
33
votes
2answers
3k views

Degrees of freedom of $\chi^2$ in Hosmer-Lemeshow test

The test statistic for the Hosmer-Lemeshow test (HLT) for goodness of fit (GOF) of a logistic regression model is defined as follows: The sample is then split into $d=10$ deciles, $D_1, D_2, \dots ...
1
vote
2answers
466 views

Why is it necessary to give a-priori a threshhold for p-Values?

Let's say the results for an experiment give a p-Value of 0.0354234. Why is it necessary to fix a threshhold before doing the experiment and than report significance? Why do I not simply report the p-...
2
votes
1answer
153 views

Is my interpretation of the p-value and confidence interval correct?

Background: let's say I have a website with a big buy button on it and I preformed an A/B testing with the following result: Version A (control): conversion rate 5% Version B (variation): ...
14
votes
3answers
4k views

What follows if we fail to reject the null hypothesis? [duplicate]

What conclusions can we draw if $p>\alpha$? Does not rejecting the $H_0$ mean anything?
9
votes
6answers
2k views

Using p-value to compute the probability of hypothesis being true; what else is needed?

Question: One common misunderstanding of p-values is that they represent the probability of the null hypothesis being true. I know that's not correct and I know that p-values only represent the ...
3
votes
1answer
195 views

Suppose I have a Cramér-von Mises $p = 0.99$ and Chi-squared $p = 0.88$ of a distribution being Student's-$t$. What can I say?

Suppose I have a Cramér-von Mises $p = 0.99$ and Chi-squared $p = 0.88$ of a distribution being Student's-$t$. What can I say? What can I say about the underlying distribution's being a Student's-$t$...
0
votes
2answers
129 views

Do probabilities for selected type I error **rates** (alpha's) have confidence intervals?

In a journal article, there can be dozens of statistical tests that are unrelated, and, the more tests there are, the greater the likelihood that the probabilities for one of those tests for a given ...

15 30 50 per page