Error function of Functional Principal Component Regression is itself a curve. Why?

Question

I have data indexed from 0 to 54. Using 6 principal components that explains 99% variation of data I was not expecting the error function to exhibit such a clear shape? By error function from following formula that is: $$ y_t(x) = \hat{\mu}(x) + \sum_{k=1}^{10} \hat{\phi}_k(x) \hat{\beta}_{k,t} + \epsilon_t(x) $$

Where $$\hat{\phi}_k(x)$$ are principal component vectors and $$\hat{\beta}_{k,t}$$ are respective scores.

To my understanding, If I conduct PCA that can explain 99% of the data, then any functional curve in my data can be reconstructed using principal component vectors and scores. This is in fact not the case in my analysis and I believe it is coming from the fact that the error function itself is a curve or perhaps a principal component vector of higher K than 10? Why is this happening? Increasing K to 20 even can not get rid of this shape in the error.

Answer 1

With a little more clarity, you should probably redefine your model in the following format. Let $i$ represent the $i$th object $(i=1,2,\ldots,n)$ and $j$ represent the $j$th $(j=1,2,\ldots,m)$ principal component (PC) extracted from the correlation or covariance matrix, and let $\mathbf{F}$ be the $n \times m$ matrix of PC scores for the $m$ components.

Then the principal components regression model would be

$y_i = \beta_0 + \sum_{j=1}^m \beta_j f_{ij} + e_i$

where $\beta_j$ is the regression coefficient for the $j$th PC and $f_{ij}$ is the $i$th PC score from the $j$th PC.

In regression terminology, your $\phi_k(x)$ is $\beta_j$ and your $\beta_{k,t}$ is $f_{ij}$.

Your data must be sorted in such a way that you are getting the results shown, since PC scores don't follow a curve. "Indexed" also doesn't say a lot -- so you should state how many dimensions (attributes, features, variables) and how many objects (instances, records) are in your dataset. Also, did you run PCA on the correlation or the covariance matrix?