What is the right test to measure *variance* across questions I have two subsamples=treatments where I ask individuals to state their binary preferences ('Do you like this: Yes/No'). In two treatments choices are the same, but the amount of information varies, so the H1 is that with more information the variation across items will be larger than in a control group. For instance, let's say baseline is that:

and treatment looks like that

In this purely imaginary example we may assume that people who can see nutrition information will be more eager to be more 'diverse' in their preferences, meaning that an amount of 'All YES' or 'All NO' will be smaller in treatment. Thus we may expect that distribution of 'Yes' answers across treatments may look like:

But what is the proper test to test this hypothesis? I've been thinking of constructing a standard deviation variable that would get two possible values: 0 for the cases of all yes (1,1,1) and for all 'no' (0,0,0), and 0.57 for 1 yes, 2 no's (1,0,0) or 1 no's and two yes (0,0,1). And since this new variable has only two values, I can use either Chi-square of Fisher test to see if this varies across treatments. But I don't know whether it's legit and I would appreciate any references and/or hints. I also wonder what's the strategy if there are 4, not 3 questions as an example above.
 A: It's actually straightforward to test your hypothesis directly as a nonlinear test on the coefficients of a model for the probability of Yes vs. No for each response. First, structure your data so that each response is a row with columns for which question is being asked (Q), whether the treatment is in effect (T) and whether the participant chose "Yes" (Y). You can then build a linear probability model for the outcome as
$$Y = \beta_1 Q_A + \beta_2 Q_O + \beta_3 Q_B + \beta_4 TQ_A + \beta_5 TQ_O + \beta_6 TQ_B + \varepsilon$$
$\beta_1$, $\beta_2$, and $\beta_3$ are the probabilities of "Yes" for the un-treated responses, and $\beta_1+\beta_4$, $\beta_2+\beta_5$, and $\beta_3+\beta_6$ are the probabilities for the treated responses. You can explicitly test whether the variance of the probabilities of "Yes" is different between the treated and untreated responses. Ideally, you would want to test whether
$$
\left( \beta_1 - \frac{\beta_1 + \beta_2 + \beta_3}{3}\right)^2 +
\left( \beta_2 - \frac{\beta_1 + \beta_2 + \beta_3}{3}\right)^2 +
\left( \beta_3 - \frac{\beta_1 + \beta_2 + \beta_3}{3}\right)^2
$$
is different from
$$
\left( \beta_1+\beta_4 - \frac{\beta_1+\beta_4 + \beta_2+\beta_5 + \beta_3+\beta_6}{3}\right)^2 +
\left( \beta_2+\beta_5 - \frac{\beta_1+\beta_4 + \beta_2+\beta_5 + \beta_3+\beta_6}{3}\right)^2 +
\left( \beta_3+\beta_6 - \frac{\beta_1+\beta_4 + \beta_2+\beta_5 + \beta_3+\beta_6}{3}\right)^2
$$
This can be programmed as a nonlinear hypothesis test of the above function of coefficients. In Stata, you can run this using nltest after running regress with vce(robust). In R, you can use the nlWaldTest::nlWaldtest() after a call to lm(), making sure to supply a robust covariance matrix using sandwich::vcvoHC(). That is, your code will look like this:
fit <- lm(Y ~ 0 + Q + Q:T, data = data)
nlWaldTest::nlWaldtest(fit, Vcov = sandwich::vcovHC(fit), df2 = TRUE,
                       texts = test_string)

where test_string is a string containing the hypothesis test of interest.
