2
$\begingroup$

I am analyzing educational data (a PISA-like exam) and I have hundreds of variables (each student answers a 50 items questionnaire, so does the student's teacher, and the school principal). I am modeling the regression of the student's grade using these variables.

In my mind, the variables are grouped into latent concepts. For example, variables V30, V31, V32, V33, and V34 are answers to questions that have to do with the teacher's working condition (TWC) - variables V10 to V19 are questions related to the student's attitude towards learning (SATL), and so on.

There is also a variable X that I am interested in, and it is not part of any group (in this case, whether the school is public or private).

The focus of the research is about this variable X and I think I know what to do about it, but it would be a bonus if I can make claims about the groups/latent concepts, like "TWC is important on the student outcome" Or "SATL is not important on the outcome" and so on.

I think I have 4 alternatives on how to add these groups and I think I know how to do it (except for the 1st) but I do not know why I should do it, and how to justify the decision. In particular, I would appreciate references to the literature on the alternatives.

The alternatives:

1) keep the regression on all variables and maybe there are ways of adding the importance of each variable in the group. I was planning on using either eta squared or the partial eta squared to measure the importance of each variable. I am not sure I can add the eta squared or the partial eta squared of different variables to get the group importance!

2) add V30 to V34 and create a new variable TWC, and work with that new variable in the regression instead.

3) perform a PCA on V30 to V34 and keep one dimension. This is the TWC new variable.

4) compute the regression

grade ~ V30+V31+V32+V33+V34 

and use the resulting coefficients to compute the new TWC.

I have seen alternative 2 in papers, and I understand 3, but I do not know why 4 is not used more.

I am particularly worried on how each alternative (except 1) will impact the variable X. I am worried that they may increase the eta/partial eta of X and thus unfairly increase the importance of the type of school in explaining the grades. The reason I think they may increase the eta is that with the new variable the regression will fit less well the grades which would increase the importance of X in the regression - but I am not 100% sure about this.

I would appreciate if anyone can point me to literature or examples dealing with this issue of groups of variables.

I am restarting a bounty on this question because the answer I got a year ago was unsatisfactory to the purposes of the research. I understand that all alternatives are similar in the sense that all new variable TWC is a linear combination of the variables V30 to V34 - but the central point of the question is how to compute the importance of this group of variables on the student's outcome.

$\endgroup$
  • $\begingroup$ Have you considered item response theory? It's designed to deal with the exact kinds of problems you're worried about, such as identifying a single latent trait that influences multiple questions, and is very widely used to analyze questionnaires. $\endgroup$ – olooney Jun 25 '18 at 20:13
  • $\begingroup$ If you know the 'groups', why not just compute regression over all variables, and separately compute the regressions grade ~ group_xyz, for each group xyz; showing that your overall model is (very / not at all / somewhat) good, using variable X alone is __ good, and also if we look at this chart we can see that TWC alone is __ good but SATL is ___. I feel like there's no need to feature engineer things here, if the group to variable relation is well known. $\endgroup$ – Avik Mohan Jul 5 '18 at 20:22
  • $\begingroup$ What if you just fit a full linear regression and then test the hypothesis that all of the TWC coefficients are 0? $\endgroup$ – Akababa Jul 27 at 21:18
0
$\begingroup$

Just to point out , that , alternative 2) and 3) are similar as the Principal Component formed will be a linear combination of the variables , similar to 2) , where coefficients of the variables are equal to 1.

For alternative 4) , it is not used as , if you fit a model on, say ,just [V30, V31 , V32 , V33 , V34], then you may get small p values for these predictors, and incorrectly infer that these predictors are important for the model. When you fit these predictors with rest of the predictors in the model, you may get large p-values for them , and hence these predictors [V30, V31 , V32 , V33 , V34] may become insignificant. So using the alternative 4) , you will not be able to tell whether these predictors are important or not.

You can refer to Chapter 3 , Introduction to Statistical Learning , for detailed information.

$\endgroup$
  • $\begingroup$ all alternatives are a linear combination of the variables - the point of the question is what are the correct coefficients for this linear combination. Regarding your answer to alternative 4 - I do not want to analyze the predictor V30, or V31 and so on, I want to analyze the latent variable (TWC) that is a linear combination of V30, V31, to V34 (but which linear combination??? - that is the question). $\endgroup$ – Jacques Wainer Jun 30 '18 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.