Which test-methods are right for my data?

Question

I'm writing my final thesis in Chinese Studies right now and planned on doing a qualitative test but as I have so much data I would like to analyse it quantitatively. I did so but am not sure about the test-methods - can you help me?

Here is what I did: I'm analysing the design differences between a) English and Chinese and b) German and Chinese versions of websites. I have 50 Chinese, 47 English and 37 German versions. I am trying to test if the differences found are significant. Am I doing it right?

Here are some screenshots from Excel.

1. Yes/No questions: Some criteria just ask if a site does have something or not (like Internet Explorer 6 (IE6) compatibility). What I get is:

   30 x Yes; 20 x No - for Chinese version
   15 x Yes; 22 x No - for German version
   17 x Yes; 30 x No - for English version 

   Can I use the Chi-square-test for that kind of data? As I might split the data into genres I might end up with quite small datasets. Is the exact Fisher's test better? (see screenshots in link).

2. Values 0, 1, 2, 3: I'm also analysing things like visibility of social media buttons (0=not visible, 3=very visible). In order to find out significant differences in the average can I take the t-Test? As only one side is important (does Chinese version get a higher number?) do I take the one-sided test or two-sided test? Excel also asks me if the data is paired, if I have the same variances or different variances - what does that mean? (see screenshots in link above for my results).

3. And finally I got data for the width and height of websites. Can I also use a t-test to analyse the data? I did a Box plot/box and Whiskers Chart but am not sure what it does for me besides giving me a quick overview - or can I also see significant differences with it? (see screenshots in link for the charts - the one does seem to have differences the other one not.)

Please help me by telling me if the used test-methods are right and confirm that I can use them or not. By the way I analyse all the data in Excel as I have no experience with any other tools (and for my major something like this isn't required at all but I would like to do it anyway).

Answer 1

These sound like paired data, since you've accessed different language versions of the same sites. Paired sites mean that the presence or absence of some item in one language version of one site is more closely related to presence or absence of that same item in the same version of the site but for a different language. To account for data of that structure, you need to think about paired categorical analysis.

The McNemar's test extends the direct analogues of Pearson's Chi Square and Fishers Exact Tests of 2x2 contingency table data. The basic null hypothesis remains the same: is frequency of site features equal between language versions? However, you can show mathematically that the McNemar's test conditions upon unmeasured sources of variation in which types of sites may be more or less likely to host certain services, such as public interest vs. social network vs. broadcast media.

Similarly, for pseudocontinuously valued data such as in your question 2, you can use the paired t-test as an extension of the regular t-test. To address your question about variance specification, I always specify my variances as unequal at the cost of a modest loss of power, but with the reward of a great deal of validity. However, the paired t-test makes this assumption unnecessary because we treat the differences between groups as a single random variable and avoid modeling between cluster effects.

Again, testing for differences in dimensions involves quantifying some continuous value and testing its difference using paired analyses. Your box-whiskers do not indicate what value you've measured, but let me suggest you consider the diagonal, as screens are measured, given in pixels as an outcome.