In order to obtain confidence intervals for proportions I'm trying to calculate the standard error, but I'm having difficulty working out what N should be in a case such as mine.
My data is such that each observation is the number of occurrences of a particular outcome in a number of trials. Here is a vastly simplified representation of my data:
Participant Item Condition X N
1 1 A 10 50
1 2 A 15 50
1 1 B 5 50
1 2 B 20 50
2 1 A 15 50
2 2 A 30 50
2 1 B 5 50
2 2 B 25 50
Where N is the number of trials, X is the number of a specific outcome, condition is a fixed effect variable and participant and item are two random effect variables.
To get the mean proportion for each condition, the common approach in my field would be to take the mean of each participant's mean proportion, e.g. the mean of participant 1's mean for condition A and participant 2's mean for condition A.
What I'm not sure of is what N should be when I calculate the standard error of that proportion using $\sqrt{\frac{p(1-p)}{N}}$. This isn't discussed much in the field, but what little consensus I can find suggests that N should be the number of participants (in the above case, 2), which was the level to which the data was aggregated in order to calculate the mean. When I do this for any real data the confidence intervals are so large that the experienced researchers I've spoken to doubt they're accurate (the real data typically come from around 25 participants).
An alternative suggestion has been to use total number of observations that are being aggregated over (in the above case, this would be 4). This seems more plausible, and makes sense when each observation is binomial. However, I'm still a little worried about whether the number of trials that make up each observation ought to be factored in somehow. If each trial is treated as a single observation then you end up with a large N (200 in this example, 30k+ in real data) which leads to implausibly small confidence intervals.