1
$\begingroup$

I asked survey participants which people influenced their idea of what college will be like, instructing them to "check all that apply." I am trying to see if there is a difference in the type of influences between two populations, so calculating the mean does not make sense. Any advice on what kind of test I can do? Or can I just represent the responses in bar graphs, showing the differences between the two populations?

$\endgroup$
4
  • 1
    $\begingroup$ Welcome to our site! You seem to be conflating two questions here - whether the bar chart is a suitable representation of your data, and what statistical test you can perform to detect a difference between your populations. I suggest you edit this question (perhaps to focus on the bar chart issue) and ask a second question to focus on the statistical test. $\endgroup$
    – Silverfish
    Commented Sep 13, 2016 at 13:39
  • $\begingroup$ @silver It seems like one unified question to me in that it asks for a suitable way to compare this kind of "all that apply" response between two groups. $\endgroup$
    – whuber
    Commented Sep 13, 2016 at 15:20
  • 1
    $\begingroup$ A bar graph is not a test! If, however, you have decided that the bar graph is the appropriate way to visualize these data, and you are interested in the hypothesis of whether or not any category differs in proportion between the two populations, a $\chi^2$ test of heterogeneity is an appropriate choice. $\endgroup$
    – AdamO
    Commented Sep 13, 2016 at 16:08
  • $\begingroup$ @whuber That's fair enough and I see where you're coming from. I do think that the graphical question and the statistical testing issues are somewhat distinct - to the extent that someone may feel qualified to give an answer to one of the two issues, but not the others. In that case splitting the issues into separate questions may allow each to be answered and explored more comprehensively. (I can see there is an underlying unity here, but it doesn't hurt to ask a pair of questions and put a link in each to the other.) $\endgroup$
    – Silverfish
    Commented Sep 13, 2016 at 17:01

1 Answer 1

1
$\begingroup$

A bar graph should be fine, as long as you are clear about what it is showing. In this case I would make clear that the number of survey respondents will be lower than the total number of influences reported, due to the "check all that apply" survey design. Depending on what you would like to investigate with your data, it might be worth doing additional analyses based on subpopulations sharing the total number of items checked (so, among people who marked three influences, influence X was the most common whereas among those who only marked one influence the most common was influence Y).

What tests you can do is a large question. What tests you can do that address what you would like to learn from your data is more manageable question, but one that requires more detail from you about what sorts of options your respondents had on the survey as well as what you would like to learn from the responses.

But based on what is written at present I see two straightforward types of comparison that may be of interest. First, you could compare the number of influences marked per respondent between the two populations. That would be a comparison of means, and something like a t-test would probably be appropriate. Second, you could compare the odds of a given respondent marking a particular influence for one population relative to another. The binary nature of the response (an influence is marked or left blank) suggests a logistic regression, and you could include a regressor for the number of influences a respondent marked to address the "check all that apply" setup.

$\endgroup$
2
  • $\begingroup$ I'm not quite sure what you're suggesting with a logistic model, but anything you might be suggesting is prone to ignore some key factors. Such as if a participant endorses the one response level, it is impossible for them to endorse any other response level (dependence) and that population membership is not an outcome of interest, but a predictor. Furthermore, the statistical definition of influence does not seem to coincide with your explanation. $\endgroup$
    – AdamO
    Commented Sep 13, 2016 at 16:12
  • $\begingroup$ My usage of "influence" was only to match the OP's terminology in describing the survey. A logistic regression can produce an odds ratio, indicating that a member of population A exhibited (for example) 2.1 times greater odds of reporting influence by Person Q than a member of population B. That would require using Person Q as a the outcome, of course. Including the response level (as the number of survey items marked) as a categorical term (which is mutually exclusive and from an exhaustive set) in the regression would seem to address the dependence, but perhaps I am misunderstanding you. $\endgroup$
    – Upper_Case
    Commented Sep 13, 2016 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.