Chi Square independence test for continuous measure (not counts) - updated

Question

Suppose I do A/B testing and I collect many observations. An observation is the visit of a client into the store. I collect if the client bought something, the value of the eventual purchase, if the client was exposed to the treatment or control (treatment or control could be anything like how I arrange products into the store, or whatever). Additionally I can segment my clients into various nominal groups, and suppose right know I segment my clients into age groups like: child, young, adult, old.

I know I can run an independence test for control/treatment vs age group, by counting them, having the ability to answer if the age group distribution on control is different than the age distribution on treatment.

Noting with $I_{buy}$ an indicator function which has value $1$ if the client bought something or not, I think I can also study if the effect of the treatment is different for different age groups of the clients. This can be translated as: "the intention of the client to buy something".

My question is if I use the purchase value instead of $I_{buy}$, can I do a chi square independence test to study if the effect of the treatment is the same for all age groups for "how much clients have spent"? I did not seen something like that in literature, but I think that because chi square test uses the assumption that the value from each cell can be approximated with a normal distribution (at least this is what I think, because it is considered a multinomial which is approximated with a normal distribution) I might use the same test if I can safely assume the values from each cells are normally distributed. The sum of purchases usually is approximated considered a log normal so I could take a log. Another approach would be to use the mean values considering that the mean values of the purchases in each cell is normally distributed due to central limit theorem.

[Later edit]

I knew about the proof of chi-square test provided here. However I disagree that we need the counts for my case. In the standard count case the counts are used to prove that the counts are used to compute the covariance structure of the variable.

After that it is found another random variable in the form of $g-<g,p>p$, where $g$ is a sequence of i.i.d. standard normal sequence and $p = (\sqrt(p_1), \sqrt(p_2), .., \sqrt(p_r))$ with $||p||^2_2=1$ (a unit vector). It is proved that the new vector has the same covariance structure as our vector derived from counts (technically it is the vector of terms from chi-square test which asymptotically is a sequence of normal i.i.d. variables ).

Later on, based on some other things it was shown that the 2nd norm of that vector is $\chi^2_{r-1}$ distributed.

Now my point is that I don't need counts for my case. The point is that counts are useful for the standard count test to find the covariance structure. However, instead a vector of counts I have a sequence of i.i.d. normal variables. I consider each or $r$ normal variable as the sum of all the random variables which happens to stay in the r-th category. As a side note, in the original case, for each r-th term the r.v. is a sum of Bernoulli i.i.d trials which is distributed as binomial (to the limit as normal), and in my case for each r-th component is a sum of Gaussian i.i.d. values, which is also Gaussian. We note also that the expected proportions from each category remains the same for both cases.

Now what I tried to find if anybody found a way to prove that this sequence of i.i.d. normal variables with given expected proportions has the same covariance structure. Intuitively I would expect it to happen, but I am not yet able to prove that myself. Now I see that it looks there's no answer in the literature yet. I will try to prove that myself. But before that I will try a lot of simulations (since basically I am a programmer).

Answer 1

I would suggest a linear modelling with an interaction term. More precisely, it seems one could reframe your question as testing this model :

$I_{buy}$ depends on 2 categorical factors $Age$ (4 levels) and $Treatement$ (2 levels). More importantly, you suspect (and want to highlight) an interaction between $Age$ and $Treatement$.

Then your linear model is $I_{buy}$ ~ $Age$ + $Treatement$ + $Age$*$Treatement$.

You can run your model in r with :
> lm(Ibuy~ Age + Treatement + Age*Treatement)

You might also want to plot your interaction by :
> interaction.plot(Treatement, Age, Ibuy)

Answer 2

Counts are required for the proof of the chi-squared test which uses the probability of an observation being in a specific cell. There is a central limit theorem argument where $$ \begin{align} \frac{n_j - np_j}{\sqrt{np_j(1-p_j)}} &\rightarrow N(0, 1)\\ \\ \frac{n_j - np_j}{\sqrt{np_j}} &\rightarrow N(0, 1-p_j) \end{align} $$

It is not the value of the cell ($n_j$), but this term that is normally distributed. As a side note, since the variance depends on individual $p_j$s, the residuals as a group is only approximately normally distributed.

For the remainder of the proof, since the cell probabilities are not independent, the covariance matrix is calculated and eventually it is shown that the chi-squared statistic converges to a chi-squared distribution as $n\rightarrow\infty$ $$ X^2=\sum_{j=1}^{k}\frac{(n_j - np_j)^2}{np_j} \rightarrow \chi^2_{k-1} $$