0
$\begingroup$

So here I am studying regression analysis. As an assignment, I have been asked to obtain some binary data, simulate its behavior from some sample of it and apply the resulting general linear model to the whole data in order to know how accurate it is.

The problem is I am not used to such method when applied to categorical (dummy) variables. More precisely, given the model $\textbf{Y} = \textbf{X}\textbf{B} + \textbf{E}$, I am a little bit lost as to the precise procedure to obtain the matrices $\textbf{Y}$ and $\textbf{X}$.

For example, consider that $\text{man} = 0$ and $\text{woman} = 1$, from whence we obtain the emprical matrix from some target population we are interested in \begin{align*} \textbf{Y} = \left[ {\begin{array}{c} \text{man} \\ \text{woman} \\ \text{man} \\ \text{woman} \\ \text{man} \\ \text{man} \\ \end{array} } \right] = \left[ {\begin{array}{cc} 1 & 0 \\ 0 & 1 \\ 1 & 0 \\ 0 & 1 \\ 1 & 0 \\ 1 & 0 \\ \end{array} } \right] \end{align*}

My question is: how do we proceed from here? Moreover, how do we get $\textbf{X}$? Any help is appreciated.

$\endgroup$

1 Answer 1

0
$\begingroup$

Your model is really $\hat Y$=$XB+E$. Notice the hat on top of the y; y hat is your predicted values for Y. Y is the dependent variable; Y is going to be a continuous variable if you're doing regression in the general linear model.

You normally know Y, it's data you've collected. Your X matrix (the design matrix) is what you've called Y. X is just your interdependent variables put into a matrix, you know the values already, they are the data you've collected. Each column represents a different variable. If you dummy code a variable, you split it into separate columns of 1's and 0's, one column for each level, so that each level get a different beta estimate. If your model has an intercept, which most models will, then add a column of 1's to the front your X matrix. Otherwise you've correctly made the dummy variables for gender--just remember which column is for which level.

If you don't know what B (beta coefficients/weights) is, then beta is estimated using $(X^T X)^{-1} X^t Y$

If this is simulated data, and you've got some made up B vector, you can literally matrix multiply it with your X and add some simulated normally distributed error and get Y hat. Use that to compare to actual Y by subtracting Y and getting the residuals.

If your Y (dependent variable) is meant to be a binary variable, you should not fit a regression from the general linear model because that will be biased, but fit using a distribution from the the generalIZED linear model, i.e. logistic regression.

$\endgroup$
11
  • $\begingroup$ In the first place, thanks for the answer. But I still have some questions. Once I have the design matrix based on categorical data, how do I get the matrix $\textbf{Y}$ from empirical data? My question is based on the following dataset: users.stat.ufl.edu/~aa/cda/data.html I know how to get the design matrix, but I am a little bit lost as to how describe the matrix $\textbf{Y}$. $\endgroup$
    – user242554
    Commented Apr 7, 2019 at 19:30
  • $\begingroup$ which dataset from the webpage? $\endgroup$
    – Huy Pham
    Commented Apr 7, 2019 at 19:39
  • $\begingroup$ The least but one: "Election data set of Table 15.5 (1 = Dem, 0 = Rep)". But if you suggest another one, I would be equally grateful. $\endgroup$
    – user242554
    Commented Apr 7, 2019 at 19:40
  • $\begingroup$ yeah, that dataset isn't going to work for regression, I can't even tell what the dependent variable would be! And no matter which column you pick to be Y it's going to be binary which doesn't fit with regression. 14.15 might work. It's count data, so it might not be normally distributed. So try the other columns (except the first one) as the design matrix, and Y is the last column "total vote 2000". $\endgroup$
    – Huy Pham
    Commented Apr 7, 2019 at 19:49
  • $\begingroup$ Check a histogram of that Y column though, if it doesn't look normal then probably the assumptions of regression will be violated. (if you only care about the maths then it'll still add up--you are still fitting a regression, it's just that regression won't estimate the distribution correctly) $\endgroup$
    – Huy Pham
    Commented Apr 7, 2019 at 19:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.