0
$\begingroup$

I've been reading about statistical significance and I am confused about one thing.

Say I have 100 control group and 100 experimental group(drug received). I've gathered result of test and want to conclude that there is difference btwn two groups => drug is effective.

We must start by creating null hypothesis
Ho = two groups are the same.
Ha = two groups are NOT the same.

our goal is to reject the null hypothesis to conclude that Ha is true.

To reject null hypothesis p < alpha = 0.05. To calculate p value we calculate z-score of Xi (each person in exp group) and then find their p-values.

Here is what I'm confused about --> so we get p-value for each individual and if its p-value < 0.05 then we say that one person's result is statistically significant? and do the same for rest 99 people and if more than 95 people's results are statistically significant we say from our experiment we can conclude that drug is effective with 95% confidence?

Another Q: Why not simply try to prove that Ha is true? instead of rejecting Ho?

Thanks in advance!

Improve this question
$\endgroup$
1
1
$\begingroup$

To reject null hypothesis p < alpha = 0.05. To calculate p value we calculate z-score of Xi (each person in exp group) and then find their p-values.

This is incorrect. We don't find the p-value for each subject, rather the p-value is computed from the test statistic, which is a function of the data observed from each patient.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks, by test statistic you are talking about mean, median or others right? so if we use mean than we would find p-value for the mean then if it is less than alpha we conclude our drug is effective with 95% confidence? $\endgroup$ – haneulkim May 14 '20 at 1:47
  • 1
    $\begingroup$ Test statistics may involve things like the mean or the median, but they are not themselves test statistics. Also, the 95% in 95% confidence interval is not what you think. Before clarifying that, I think it would be wise to revisit some introductory material on statistics to re-familiarize yourself with hypothesis testing. $\endgroup$ – Demetri Pananos May 14 '20 at 2:06
  • $\begingroup$ So you are saying we can use one of the test statistics to calculate its p-value to see statistical significance right? $\endgroup$ – haneulkim May 14 '20 at 2:12
  • 1
    $\begingroup$ Yes, that is correct. $\endgroup$ – Demetri Pananos May 14 '20 at 3:23
0
$\begingroup$

This is not a full answer but, nevertheless :

IF your "null" hypothesis is true, i.e. if the treatment has no effect, i.e. if the two samples are really from the same population, THEN what is the probability of getting an absolute DIFFERENCE, between the two groups' means, as high as the one you observe in your CURRENT experiment, imagining you could repeat the entire experiment an infinite number of time to estimate that probability ?

IF that probability, calculated over an infinite number of replicated experiment, is INFERIOR to a threshold that YOU determined to be the THRESHOLD OF SIGNIFICANT EFFECT or THRESHOLD OF "HEY, THAT IS STRANGE" or THRESHOLD OF "HEY, THAT DIFFERENCE IS TOO MUCH TO BE POSSIBLE", and that threshold is YOUR p-value. THEN you will tell the story of a possible effect. Else you will tell the story of how you did your experiment, and you did not observe a signify effect at p-value 0.05 (natural sciences for example) or maybe 0.01 or 0.00001 (physics).

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.