I have a database containing records of several parameters: one is a quantitative parameter $D$ (e.g. average fuel consumption), others are qualtitative parameters $X_1, ..., X_n$ (e.g. make, color, engine type, ...) and quantitative parameters $Y_1, ... Y_m$ (e.g. average speed, driver's age, ...).

I am trying to assess whether there is any statistical relationship between $D$ and the other parameters $X_i, Y_j$. The ultimate goal would be to identify which of the $X_i, Y_j$ are relevant to "predict" $D$, and which are the ones that should be dismissed (because they are actually unrelated to $D$).

Please note that at this stage, I am just trying to identify what are the relevant parameters. The question of determining a quantitative relationship between $D$ and those parameters would come after this analysis.

I was thinking about performing chi-squared tests of independance between $D$ and the various $Y_j$, but I am rather confused about what to do with the qualtitative $X_i$. My questions are therefore:

  1. What tests/analysis can be used for the qualtitative parameters $X_i$ ?
  2. Is there any additional analysis that could be performed on top of the chi-squared tests for the $Y_i$?

I'd be inclined to use a multiple regression model with $D$ as dependent/target variable and the rest as predictors (all $X_i$ and $Y_i$). Note that you can use qualitative/categorical variables or continuous variables in this approach. And by including all predictors together in one model instead of doing separate chi-squared tests, you can account for the covariance between the predictors, and also model any interactions between them that you may expect.

Alternately, depending on how much data you have, I'd consider using a random forest (again $D$ as dependent/target variable and the rest as predictors). Random forests allow you to model complex interactions and nonlinear responses without specifying them a priori, which is why they may be of use. With this approach too you can include both categorical and continuous variables together.

In either case, you can get estimates of how important each predictor is, as well as quantitative descriptions of how they relate to $D$ (although this is easier with multiple regression than with random forests).

  • $\begingroup$ The problem I see with multiple regression is that I cannot assume a specific relationship (e.g. linear) between the parameters and D a priori. I will have a look at random forests as I am not familiar with this concept. For my specific case, I have around 3000 records with less than 7 parameters for each. $\endgroup$ – G. Vaurs Aug 2 '19 at 13:11
  • $\begingroup$ @G.Vaurs Random forests can handle complex nonlinear relationships without your having to specify them. I've edited my answer to reflect that. They should work fine with 3000 data points. $\endgroup$ – mkt - Reinstate Monica Aug 2 '19 at 14:05
  • $\begingroup$ I tried the random forest approach -which seems a very useful tool- on my dataset. What I feared happened: it seems that the 7 parameters (predictors) are not exhaustive enough to grasp an actual understanding of D, the OOB error estimate being around 75%. Should I conclude that modeling D based on my parameters is a vain attempt? $\endgroup$ – G. Vaurs Aug 7 '19 at 15:40
  • $\begingroup$ @G.Vaurs Hard to say, we don't know that much about your data or area. You could try the multiple regression approach, possibly with some regularization and see how well it does. $\endgroup$ – mkt - Reinstate Monica Aug 7 '19 at 16:56

If you want an alternative (simpler and to some extent parametric) to the solutions mentioned above, without making a clear hypothesis about the shape of the relationship between dependent variable and predictors and the interaction between predictors (as in a regression), just look at how each single predictor is associated to the dependent variable regardless the other candidate predictors or shape of the relationship (is each predictor significantly associated to the dependent variable?). This way you can start grabbing an idea of what predictors are significantly associated/correlated to the dependent variable without forcing too many assumptions on the shape of the relationship ( as well as the relationship between all predictors when crowded into a multivariate regression). Typically this goes under the label of univariate ANOVA (t-tests for numerical variables and and tests of association for categoriacal variables).

To some extent this is similar to checking the classification power of a predictor on the dependent variable without making the shape of the relationship explicit. And it has the advantage that it is parametric for a sufficiently large sample size (due to the central limit theorem) and easy to carry out and well documented.

Then you can use the reduced set of significant predictors found above to specify a multivariate model, by comparing different model structures (linear, non-linear alternatives, ecc and choosing the predictors for each one) based on goodness of fit (you can use iterative procedures as well to accomplish this)

  • $\begingroup$ The problem I am facing with ANOVA is that the residuals do not seem normally distributed, even after applying a transform, thus violating one one the key assumptions of ANOVA. $\endgroup$ – G. Vaurs Aug 7 '19 at 6:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.