In this R code:

# `data` is say 1000 x 50 data matrix
n = 1000  # no of observations
k = 5    # reduced dimensionality
x = seq(0, 1, length.out = n)
splinebasis_B = create.bspline.basis(c(0, 1), k)
base_B = eval.basis(x, splinebasis_B)
data_reduced = data %*% base_B

as I understand, the last operation does not project (or otherwise reducingly transform) data just yet, but rather it gives us data_reduced on which we can run some fitting algos (regressions, trees etc.).

For this reason, we cannot claim that data_reduced is compressed version of data.

Is this correct? Am I missing anything? What's the right way to think about data_reduced? It's clearly mapped to a lower-dimensional space, and there is loss as data's variance got collapsed during the multiplication with fewer columns... but I feel like somehow data_reduced is not "optimal". It's not the best representation of data in k-dimensional space... What is it then?

Appreciate your help.

For background, I'm talking specifically about using splines for dimensionality reduction, in particular as applied to functional data (screenshot)

.enter image description here


B-splines can be used to approximate a 1D curve.

Thus, if you have a set of points $(x,y)$, you can approximate the non-linear relation between $x$ and $y$ by first expanding x into a B-spline basis and then fitting the expanded basis to $y$ using a standard linear model such as linear regression.

So now if you wanted to reduce the dimensionality of several sets of $(x,y)$ pairs, you first do a basis expansion of all the $x$'s and then for each set, fit the B-spline expansion to the paired $y$'s. Then, the fitted coefficients from each $(x,y)$ set can be seen as a reduced rank approximation of the original $(x,y)$ set.

Note that it is vitally important that each $x$ be expanded at the same knot locations with the same degrees of freedom, otherwise comparing fitted coefficients becomes an apples to oranges comparison.

  • $\begingroup$ thank you for the response, I clarified the context. $\endgroup$ – Tim Jul 7 at 5:14
  • $\begingroup$ thanks a lot, Cliff. Any change you could elaborate. What exactly is the operation for 'basis expansion of all the $x$'s? Is data_reduced matrix from the snippet such expanded version of $\boldsymbol X$? $\endgroup$ – Tim Jul 7 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.