frequency table chi square test "model"

Question

I just have a question about if chi square test for the association say between two variables (I think both are discrete in this case) in a 2 by 2 table says anything about what "model" is used in the analysis.

I know that for example if you have a continuous response variable (say BMI index), and say another continuous variable (say age). And if we assume the relationship is linear, then we could fit a linear regression to the data between these two variables right? and the linear regression is called a "model", is that correct?

But say if we have a 2 by 2 table for two discrete variables, and we want to see if there is any association. I think we use a chi-square test right? in this case, do we say we have a model for these two variables? I was reading somewhere that this can be considered a logistic regression model, is it correct?

My question is, when we say a "model", is it correct that we are saying the observed data are being generated by the model? So in a linear regression, we are saying the response variable is being generated by a linear mechanism and it is linear related to another variable, is it correct?

So for a two by two tables, can we also say the same thing? Can we say we have a "model"? If I am a statistician, and someone approaches me and say "how do we find out if there is association between these two discrete variables in the data, what "model" should we try?" , is there actually a "model" in this case? is there a generating mechanism like a linear regression?

Thank you for your time

Answer 1

In your regression example, the regression itself isn't the model, but a means of estimating $\alpha$ and $\beta$ if you assume that $Y_i = \alpha + \beta X_i + \epsilon_i$ where the $\epsilon_i$ are i.i.d mean zero random variables.

More generally, we're talking about some idealized, probabilistic model that generates your data. That is, we imagine that in some ideal world there is a process that generates our output according to the model. In your regression example again, we could pick people of various ages as our $X_i$ and the model is that we assume that for people of a fixed age, say, 50, distribution of the random variable $\alpha + 50 \beta + \epsilon$ is the distribution of BMIs of all 50 year olds.

Let's get to the table now. Say you're measuring two quantities $A$ and $B$ that can each take on the values 0 and 1. Remember that for a 2 by 2 table, if you divide each cell by $n$, the total number of observations, what you have are estimates the join probabilities $P(A = 1, B = 1)$, $P(A = 1, B = 0)$, $P(A = 0, B = 1)$, and $P(A = 0, B = 0)$.

Now you can think of $A$ and $B$ as each having a binomial distribution. The question the $\chi^2$ test answers is if they're independent or not.

If you want to get a little more concrete, imagine I've written two numbers, both either 0 or 1 on pieces of paper and put them in an urn. Perhaps the first number could be if a person is under or over 65 and the second could be if their BMI categorizes them as obese. If you sampled these pieces of paper (or people) you could fill up your table. Notice that if we ignored one of the numbers, we're back to a binomial distribution. In the context of this example, the $\chi^2$ test is asking whether I'd get approximately the same table if I drew from two separate urns where the pieces of paper had one number each on them and one urn had $p_A$ 1's and the other had $p_B$ where $p_A$ and $p_B$ are the same as the proportion of first and second numbers in our dual-number urn that are 1 respectively.

As for a logistic model, we certainly could use that, but what it would be doing is estimating $P(A ~|~ B)$. The probability model for logistic regression is generally that you're treating one variable as something you can fix or control and your want a model for the conditional probability of the outcome. In the case of $\chi^2$ you're not making that distinction and we can just think of that table and the four joint probabilities in it.