I want to do some regression analysis that constrains the coefficients to vary smoothly as a function of their sequence. It is similar to the "Phoneme Recognition" example in the part 5 "Basis Expansions and Regularization" of The Elements of Statistical Learning (Hastie, Tibshirani & Friedman, 2008).

in the book it says:

The smooth red curve was obtained through a very simple use of natural cubic splines. We can represent the coefficient function as an expansion of splines $\beta(f)=\sum_{m=1}^M h_m(f)\theta_m$. In practice this means that $\beta=\mathbf{H}\theta$ where, $\mathbf{H}$ is a $p × M$ basis matrix of natural cubic splines, defined on the set of frequencies. Here we used $M = 12$ basis functions, with knots uniformly placed over the integers 1, 2, . . . , 256 representing the frequencies. Since $x^T\beta=x^T\mathbf{H}\theta$, we can simply replace the input features $x$ by their filtered versions $x^* = \mathbf{H}^T x$, and fit $\theta$ by linear logistic regression on the $x^*$. The red curve is thus $\hat\beta(f) = h(f)^T\hat \theta$.

But I am not sure about how to create the basis matrix $\mathbf{H}$. When using ns() function in R, how to set the knots?

Any hint will be appreciated.

  • $\begingroup$ still need your help. $\endgroup$ – Wei Si Jun 24 '13 at 12:46
  • $\begingroup$ Tried to using the method says in the book of Elements of Statistical Learning, but don't know how to create the basis matrix in this case. $\endgroup$ – Wei Si Jun 25 '13 at 1:22
knots <- 1:256

You should really take a look at splines before using them (wikipedia ?). Same goes for Basic R code.

  • $\begingroup$ Thanks for your answer, it is indeed something basic, however this is not a typical way of using splines. It simply works like this: ns(x<-1:256, df=12) $\endgroup$ – Wei Si Jun 27 '13 at 7:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.