I’m looking for some help on testing whether there are statistical differences between proportions when the data is dependent. By dependent, I mean that all the data is from the same participants though I’m not talking about a repeated measures experimental design but instead just different questions from the same participants. Usually, I’d go straight for the McNemar test.
However, the problem I’m facing is that... although the data is all from the same group (1000 people), there are slight differences in the numbers of people answering the questions that I’m using to calculate the proportions. I asked all 1000 people whether they regularly used Windows (all of them said yes; i.e. 1000 said yes), Mac OSX (500 said yes), a Linux distribution (100 said yes) and a variety of other operating systems (10 said yes). Participants could combine their answers - they may have used both Windows, Mac OSX… or just Windows…. or Windows and Linux, or all three, etc.
For each operating system that a person indicated using, I asked them whether they used certain software on those operating systems (for instance, whether they used a Word processor). So if a person used OSX and Windows, they’d be asked whether they used a word processor for WIndows and separately asked the same question for OSX. Another participant might use Windows and Linux, and would have been asked whether he/she used a word processor for WIndows and separately asked the same question for Linux. Another person might have said he/she used only Windows and so was only asked about his/her word processor usage on Windows.
That is where the problem arises. If I work out that 85% of 1000 Windows users regularly access a Word processors, how do I compare this to 65% of 500 OSX users regularly accessing a Word processor. I had thought about using the McNemar test but I’m not sure how to deal with the discordant pairs - there is no concordant/discordant pair for somebody who uses Windows but doesn’t use OSX, for instance.
I had thought about using a Z-test for a proportions but that is for comparing proportions that are strictly independent - I’m fairly sure my data wouldn’t satisfy that condition (but stand to be corrected).
Any ideas?
Thanks everybody! I hope somebody can help me!
Simon :)
Addition of extra information as requested
I've just mocked up some raw data. So this is a representation of the data that I have for each participant (-1 = participants who did not answer that question because they don't use that operating system).
I'm looking to see whether the proportion of people using a word processors in Windows is similar to the proportion of people using a word processor in OSX (where the data is from the same sample 'somewhat' - 'somewhat' given the issue above about participants only answering the questions relevant to them meaning I'll be comparing slightly different carvings of the same sample). So yes, I am trying to compare the proportion of people who use word processors given their OS. However, I'm trying to break it down to compare between individuals operating systems. So in the summary table below... I'm trying to compare whether 85% (Windows) is significantly different from 65% (OSX). The reason why I've been avoiding a standard Chi-square (Pearson's) is that that data is somewhat dependent - the questions do not involve different participants.
Does that make it slightly clearer? If not then let me know and I can expand :)
