0
$\begingroup$

So I'm using lasso logistic regression to classify my data.

My data matrix $X$ has dimension $n\times p$ for $p >> n$.

As $p$ is on the order of a billion, I expect to face some computational challenges when trying to run lasso logistic regression.

If I'm only interested in predictive performance, and I don't care about inference, is there any drawback to taking the SVD of

$$XX^T=U\Sigma^2U^T$$

and then simply applying lasso logistic regression to $U$?

As far as I can tell, all I've done is do a change of basis so that I'm only considering the $n-1$ dimensional affine space spanned by my data, and thus my classification algorithm should perform just as well.

Edit

It does appear that the above is correct, see the related question,

Using SVD on features before SVM classification, when p >> N

$\endgroup$
4
  • $\begingroup$ Is $p$ the number of features or the number of observations? As the first dimension of $X$ it would ordinarily be taken as the latter, but your reference to $n-1$ dimensions suggests the former. (But why $n-1$ and not $n$ or, if you throw in an intercept, $n+1$?) $\endgroup$ – whuber Jan 7 '20 at 17:51
  • $\begingroup$ @whuber, that was a typo, it should have been $n\times p$ (I'll correct). As for why $n-1$, that is the affine dimension of $n$ points, assuming no degeneracy. Although I guess I maybe wasn't clear that I was refering to that, rather than the $n$ dimensional space defined by $n$ vectors, I guess it's not a terribly important distinct to make in this context. $\endgroup$ – Thoth Jan 7 '20 at 18:36
  • $\begingroup$ Affine dimension doesn't seem relevant either to SVD or this kind of model. I agree the distinction between $n$ and $n-1$ looks unimportant when $n$ is huge, but the fact you were making it raised a flag suggesting the possibility of miscommunication. $\endgroup$ – whuber Jan 7 '20 at 18:42
  • 1
    $\begingroup$ @whuber, ya I meant that my reference to the $n-1$ affine dimension wasn't that important. But yah, $n$ isn't huge, $p$ is huge, and $n$ is (relatively) small. $\endgroup$ – Thoth Jan 7 '20 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.