Help find a statistical test for proportion of a binary outcome with repeated measures

Question

Here is an illustration of my problem:

I have two stimuli (A and B) given to a number of subjects. Each subject is sequentially subjected to each stimulus 5 times (AAAAABBBBB or BBBBBAAAAA, but not both). I have randomized the presentation order of these stimuli to each subject. The data might look like:

My question is how can I statistically show:

1. Did stimulus presentation order affect the "Responded" outcome?
2. Did one stimulus induce response more than the other?

First question addresses this comparison:

Second question addresses this comparison:

I don't remember learning how to deal with data like this. THANK YOU!

DATA: Here's some data for this question for replication purposes

Answer 1

Assuming that you are interested in the mean responses by subject and since you treat them as two separate question I will suggest two distinct and simple ways of dealing with this which I am sure are not the only ones.

For your first question you could create a new variable and code as 0 and 1 the type of presentations (e.g. 0=AAABBB, 1=BBBAAA), and perform an independent sample t-test (or the non-parametric equivalent as you are not giving out any information about the distribution of the responses) using the sum of the responses as the dependent variable.

For the second one you could create two variables (you 'd also have to change your dataset from long to wide format after computing the sum of the responses), one representing the responses for subjects when they were presented with stimulus A and a second one for subjects when they were presented with stimulus B and run a related-samples t-test (or non-parametric equivalent) using as the dependent variable the sum of responses.

Note: If your interest is in modelling the binary outcome though, then a GLM or GLMM would be suitable as Glen_b has already pointed out

Answer 2

Unfortunately, this answer is more of a comment, but I think that your question is too complex to be answered properly here without actually seeing your data. I would like to offer pointers that I consider important for you:

1. Decide what your outcome should be. Your example and your data suggest that a person responds to a stimuli for some time and then stops and doesn`t respond again. In this case your outcome might be whether and when an individual stops responding. If the subjects can switch back and forth between responding and not, I would also go with some sort of binomial model.
2. Consider whether your goal is to construct a model (explorative) or to do significance testing (confirmatory). When doing hypothesis testing you do not want to bias your p-values, so you should stick with a pre-decided model to answer these questions. If you want to construct a model, you probably would not go for hypothesis testing, but instead look at how well your model can simulate data that looks like your input. I think in Bayesian Data Analysis by Andrew Gelman there is a model for data very much like this. Unfortunately, I do not have my copy at hand.