I have determined that a one-way ANOVA test of variances will work best for my data. However, the nature of my data makes me very concerned about making any claims based on significance or not.
My research project is examining survey data (25 questions) on a likert-scale taken over a 3 year period, split among 5 subdomains (however, one domain has as little as one corresponding question, another has 8, etc.). I am testing whether or not there is a difference between when respondents took the test at baseline and throughout (7 total time categories, or every 6 months) for EACH SUBDOMAIN.
I am having some trouble saying exactly what the problem is, but simply put I do not know whether I should (a) put every individual response into a bin,count the total responses, and average them at each time slot for each subdomain, or (b) average each individual response, count the number of tests taken during that time slow as an "n", and then take the means and Std.Devs of those for the ANOVA test.
Here are my two tables that I've come up with:
The main difference is that the individual responses has much larger n values. This has direct implications on if my study is significant or not. Here is an example:
and here is why this is important... because it directly matters on whether my data is significant or not:
Results for Grouped Responses:
My intuition is telling me I should use option (B) because option A just seems to be artificially increasing the sample size, but again that is the raw data from my responses and I don't want to make any mistakes with this...
If anyone has any good ideas about how to treat this, that would be wonderful news!!! I also think I messed up anyway since I'm using Likert data and I do not know what test is appropriate to use for comparing means of likert data.