3
$\begingroup$

I have spent a significant amount of time on this, but my knowledge in Matrix Algebra is still limited. So, I am not sure ...

I have a linear regression model based on $n$ observations. The dependent variable is $y$.

As predictors, there are 10 independent continuous variables + 9 dummy variables (based on a 10-category scale) + 5 dummy variables (based on a 6-category response). The model looks like: $$ y = \beta_0+\beta_1 \text{C}_1+\beta_2 \text{C}_2+ ...+\beta_{11}\text{dummy}_1+\beta_{12}\text{dummy}_2+ ...+\beta_{19}\text{dummy}_9+\beta_{20}\text{dummy}_{10}+...\beta_{24}\text{dummy}_{14}+\epsilon $$

Would this be an accurate matrix notation of the above model? $$ \textbf{y} = \textbf{C}{\beta}+\textbf{S} \gamma + \mathbf{R}\lambda + \textbf{e} $$

where $\textbf{C}$ is n $\times$ 10+1 design matrix with 1s in the first column and 10 other columns containing the continuous predictors , $\textbf{S}$ is the n $\times$ 9 matrix containing dummy variables for the 10-category scale, $\textbf{R}$ is the n $\times$ 5 matrix containing dummy variables for the 6-category response, ${\beta}$, ${\gamma}$ and ${\lambda}$ are the vectors of parameters, and $\textbf{e}$ is the error vector.

Is this notation sufficiently accurate? Any references on matrix notation for dummy variables are also very much appreciated.

Sincerely,

Jacob

$\endgroup$
1
  • $\begingroup$ Welcome to CV.SE. Yes, you are fine. Good job (+1) Please see my answer below I expand on this a bit more. $\endgroup$
    – usεr11852
    Dec 13, 2020 at 2:10

1 Answer 1

4
$\begingroup$

Yes, you are fine. Please note that all your categorical vectors will have their first level absorbed within the intercept $\beta_0$, therefore take note of the way the encoding of dummy variables is done. I would say that if anything your notation lends itself nicely to group lasso regularisation; standard lasso solution would only select individual dummy variables instead of whole factors. (check Meier et al.(2007) The group lasso for logistic regression for more information on that aspect.)

If you are starting and have some queries about notations, standard textbooks on the use of matrices in Statistics are: Matrix Algebra Useful for Statistics by Searle and Matrix Algebra From a Statistician's Perspective by Harvill. Both are consider classics on the matter and you can use them as well-accepted references. (I also like: Matrix Algebra: Theory, Computations, and Applications in Statistics by Gentle, I have used it a lot).

$\endgroup$
2
  • $\begingroup$ Remarkably useful. Thank you so much! $\endgroup$
    – Jacob
    Dec 13, 2020 at 2:24
  • $\begingroup$ I am glad I could help. If this answer is helpful please consider upvoting it and if it resolves your question marking it as the accepted answer. $\endgroup$
    – usεr11852
    Dec 13, 2020 at 2:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.