I have the matrix of the independent variables X with dimensions n_samples x m_features and a vector Y with the dependent variable with n_samples rows.

I suspect that the features I collected so far are insufficient to describe completely the behavior of Y and that there is some feature I'm missing.

Thus, I'm looking for a statistical quantitative index to tell me at which grade the features I collected so far, contained in the X matrix, are sufficient to describe the behavior of the dependent variable vector Y.

  • $\begingroup$ There are good reasons nobody has answered: among others, this question is prohibitively vague. Could you perhaps specify what you mean by "describe completely"? The answer you posted suggests you are interested only in models that are linear in the columns of $X,$ but that's not made clear in the question. But if you're interested only in such models, then please explain (1) what you mean by "sufficient" and (2) why the residuals to the ordinary least squares regression of $Y$ on $X$ do not supply the answer. $\endgroup$
    – whuber
    Jul 30 '19 at 13:05
  • $\begingroup$ Indeed I'm not interested just in linear models $\endgroup$ Jul 30 '19 at 13:10
  • $\begingroup$ For "describe completely" I mean that Y does not depend on other variables but the ones on the columns of X $\endgroup$ Jul 30 '19 at 13:11
  • $\begingroup$ But depend in what way? For instance, when all the rows $x_i$ of $X$ are distinct, it's always possible to find a function $f$ for which $f(x_i)=y_i$ for every row $i.$ It's unlikely $f$ is linear, which makes the PCA solution you posted irrelevant. $\endgroup$
    – whuber
    Jul 30 '19 at 13:14
  • $\begingroup$ I know, indeed this 'solution', as I wrote, evaluate only the part of linear dependency which is a first approximation... $\endgroup$ Jul 30 '19 at 13:15

You can try fitting a linear regression model and examining $R^2$.

$R^2$, or the coefficient of determination, is (according to Wikipedia) "the proportion of the variance in the dependent variable that is predictable from the independent variable(s)."

So if the $R^2$ from your model is 95%, then this can be interpreted as your independent variables explain 95% of the behaviour of your response variables. On the other hand if your $R^2$ is 5% then it appears your independent variables do a poor job at explaining your response variable.

  • $\begingroup$ Thank you, this seems a better solution for taking into account the linear dependency than the one I posted, I'll give it a try $\endgroup$ Jul 30 '19 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.