General Linear Model applied to binary data

Question

So here I am studying regression analysis. As an assignment, I have been asked to obtain some binary data, simulate its behavior from some sample of it and apply the resulting general linear model to the whole data in order to know how accurate it is.

The problem is I am not used to such method when applied to categorical (dummy) variables. More precisely, given the model $\textbf{Y} = \textbf{X}\textbf{B} + \textbf{E}$, I am a little bit lost as to the precise procedure to obtain the matrices $\textbf{Y}$ and $\textbf{X}$.

For example, consider that $\text{man} = 0$ and $\text{woman} = 1$, from whence we obtain the emprical matrix from some target population we are interested in \begin{align*} \textbf{Y} = \left[ {\begin{array}{c} \text{man} \\ \text{woman} \\ \text{man} \\ \text{woman} \\ \text{man} \\ \text{man} \\ \end{array} } \right] = \left[ {\begin{array}{cc} 1 & 0 \\ 0 & 1 \\ 1 & 0 \\ 0 & 1 \\ 1 & 0 \\ 1 & 0 \\ \end{array} } \right] \end{align*}

My question is: how do we proceed from here? Moreover, how do we get $\textbf{X}$? Any help is appreciated.

Answer 1

Your model is really $\hat Y$=$XB+E$. Notice the hat on top of the y; y hat is your predicted values for Y. Y is the dependent variable; Y is going to be a continuous variable if you're doing regression in the general linear model.

You normally know Y, it's data you've collected. Your X matrix (the design matrix) is what you've called Y. X is just your interdependent variables put into a matrix, you know the values already, they are the data you've collected. Each column represents a different variable. If you dummy code a variable, you split it into separate columns of 1's and 0's, one column for each level, so that each level get a different beta estimate. If your model has an intercept, which most models will, then add a column of 1's to the front your X matrix. Otherwise you've correctly made the dummy variables for gender--just remember which column is for which level.

If you don't know what B (beta coefficients/weights) is, then beta is estimated using $(X^T X)^{-1} X^t Y$

If this is simulated data, and you've got some made up B vector, you can literally matrix multiply it with your X and add some simulated normally distributed error and get Y hat. Use that to compare to actual Y by subtracting Y and getting the residuals.

If your Y (dependent variable) is meant to be a binary variable, you should not fit a regression from the general linear model because that will be biased, but fit using a distribution from the the generalIZED linear model, i.e. logistic regression.