2
$\begingroup$

Please note Edit Below:

I have to establish the statistical relevance of a hypothesis which is of the form

H0 = The event lead to an increase in performance
Ha = The event caused a negative or no effect in performance

The data set that i have has achievements before and after the event.
The question is how do I run the hypothesis testing?
More precisely-> for a given time i have a number of records measuring performance both before and after the event like below

enter image description here

I was wondering what parameter to take as mu in the 2-tailed T test. Should i create another metric combining both the achievements which will be a measure of the performance? If yes, any suggestion on how i can make such a metric?

Any suggestions beyond the above will also be helpful.

Edit 1:

I figured that the 2 sample T test would fit the case where my hypothesis would be of the form
Ho : mu1 - mu2 = hypothesised mean difference (in this case 0)
Ha : mu1 - mu2 < / > / not equal the hypothesised mean difference

Assuming independent populations and normally distributed data

But my data is like the following as mentioned above.

Is it right to compare the means of the achievements through the above test as the data is not at the same level .. as in it is at a person product level.
Will ANOVA makes more sense for the above dataset to see the difference between the performances before and after the event?

Is it possible to do the analysis at this level or should i club the products and have the data like the following and then go ahead with the 2 sample T test?

enter image description here

Improve this question
$\endgroup$
2
  • $\begingroup$ Isn't the null, H0 generally that the event had no effect? $\endgroup$
    – Matthew Gunn
    Commented Dec 12, 2015 at 8:24
  • $\begingroup$ Sounds like a start is to take the difference between after and before for each person, for each product, and then for each product, compute the average difference? Test if the average difference is statistically different from zero. $\endgroup$
    – Matthew Gunn
    Commented Dec 12, 2015 at 8:26

1 Answer 1

Reset to default
2
$\begingroup$

There are few misconceptions in your question. First, null hypothesis generally refers to "no effect" hypothesis that is verified by your research. Looking at your problem definition, it seems that you are interested in one-tailed hypothesis test about $\mu_\text{after} - \mu_\text{before} > 0$ (i.e. you are interested if your treatment improves achievements; see also this thread). Another misconception is about the data you have and the test you need. You say that you have

a number of records measuring performance both before and after the event

but in fact you have records measuring performance both before and after the event for different products, so you have two independent variables and you probably need rather ANOVA than $t$-test.

Referring to your edit, if you ignore in your analysis the fact that you had different products, that you can use $t$-test for such data. However in many cases this may not be the best way to go. As an example, you can search for Simpsons paradox, i.e. situation where your data shows something different if you look at it at local level, than on global level (see also this example). So such analysis may in some cases lead to different results than when you included information about products. This all depends on what is your data and what you want to learn from it (maybe products are something you can ignore, maybe not -- this is something you should consider).

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks. I think ANOVA will make sense for the kind of data that i am dealing with. However as per my last edit. if i club the products (as per the second dataset given in the question) then I should be able to use a 2 sample T test .. is it? $\endgroup$
    – plv
    Commented Dec 12, 2015 at 20:52
  • $\begingroup$ I was just pondering over this further and realized that wouldn't MANOVA make more sense for the original level of data with the products? Since i have 3 independent variables (Month ,Person and Product) and multiple levels in that and two dependent variables (Achievement before and Achievement After). ANOVA is for one dependent variable only right? $\endgroup$
    – plv
    Commented Dec 12, 2015 at 22:04
  • $\begingroup$ @user3568002 I used here ANOVA as an overall name for a group of methods, not only univariate case. $\endgroup$
    – Tim
    Commented Dec 12, 2015 at 22:10
  • $\begingroup$ I have one more query. I noticed that i am testing for only one dependent variable i.e. improvement in one field due to the event.so ANOVA holds good and not MANOVA since i wrongly assumed both the achievements as two dependent variables . However how do i create a metric indicating performance using the two achievements? i mean is there a proper way of quantifying such difference in performances? $\endgroup$
    – plv
    Commented Dec 13, 2015 at 10:30
  • $\begingroup$ @plv I do not fully understand the question in your last comment, however it seems that this is a question about conducting some different analysis than the one described in your initial question. If so, then you should probably start new question and describe this problem in detail in such question. $\endgroup$
    – Tim
    Commented Dec 13, 2015 at 10:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.