9
$\begingroup$

Why I am getting errors in R for "'degree' must be less than number of unique points" in this case?

set.seed(0)
# this is fine
dim(poly(runif(1e3),20))
# this will not work
dim(poly(runif(1e3),30))

Where we have $1000$ unique data points from uniform distribution, and the degree is only $30$?

I found the answer to be "numerical overflow"

https://stackoverflow.com/questions/17998919/why-is-poly-complaining-about-degree-less-than-number-of-unique-points

But we are using orthogonal polynomial but not raw, would that solve the numerical problem (example kappa(poly(runif(100),20)))?

$\endgroup$
7
  • 1
    $\begingroup$ High degree polynomials can have big numerical stability problems as the condition number of the design matrix gets high enough that even double precision floating point numerical error becomes a problem. I'd ask if you really need a degree 20,degree 30 polynomial and/or fit a spline with lower degree polynomials. $\endgroup$ Commented Nov 3, 2016 at 20:09
  • 1
    $\begingroup$ @MatthewGunn but if we use orthogonal polynomial, should the conditional number always be $1.0$? check kappa(poly(runif(100),20)) $\endgroup$
    – Haitao Du
    Commented Nov 3, 2016 at 20:16
  • 1
    $\begingroup$ Have a read on my answer at Stack Overflow: High (or very high) order polynomial regression in R (or alternatives?) $\endgroup$
    – Zheyuan Li
    Commented Nov 4, 2016 at 0:47
  • 1
    $\begingroup$ @ZheyuanLi your SO answer perfectly answered all of my questions! $\endgroup$
    – Haitao Du
    Commented Nov 4, 2016 at 1:37
  • 1
    $\begingroup$ @ZheyuanLi +6 for your SO answer. I understand high degree polynomial almost never useful. The reason I had that is I want to do some simulations on overfitting experiments. Your GAM and other suggestions are really helpful.! $\endgroup$
    – Haitao Du
    Commented Nov 4, 2016 at 1:44

1 Answer 1

9
$\begingroup$

In R, the poly function generates a polynomial and then does a QR decomposition to make it orthogonal. For high degree polynomial input, you'll get a high condition number for the Vandermode matrix, and when poly tries to make the columns orthogonal, it's going to throw an error (because the input looks rank deficient within the limits of double precision floating point).

Consider:

x <- poly(runif(100),30,raw=TRUE)
QR <- qr(x)
QR$rank

This is basically what's in the source code of poly. QR$rank is going to be far lower than the degree.

$\endgroup$
3
  • 4
    $\begingroup$ Sounds like it's poor implementation on the package's part. $\endgroup$
    – Alex R.
    Commented Nov 3, 2016 at 20:44
  • 1
    $\begingroup$ @Alex Poor in what sense or by what standard? $\endgroup$
    – whuber
    Commented Nov 3, 2016 at 21:36
  • $\begingroup$ @whuber may be r should not use RAW expansion as intermediate step? $\endgroup$
    – Haitao Du
    Commented Nov 3, 2016 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.