# Using eigenvalues of the covariance matrix to reduce noise in my data

I have an idea to help reduce the noise in my signal but am stuck with a significant problem.

I have a very noisy data set $$y_n[t]; n\in\{0, N_{\text{samples}}-1\}; t\in\{0, T-1\}$$ I am fitting this signal to a non-linear function $$f(t;\vec{\theta})$$ ($$\vec{\theta}$$ is my vector of parameters in my non-linear function), by minimizing the log-likelyhood function $$\text{log}(\mathcal{L}) = \big(y[t] - f(t,\vec{\theta})\big)\big(\Sigma^{-1}\big)_{t,t'}\big(y[t'] - f(t',\vec{\theta})\big)$$ where $$\Sigma_{t,t'} = \text{Cov}(y[t],y[t']).$$ I know that there exists unitary operators $$U$$ that can diagonalize the covariance matrix into components of least-to-most variance in the data-set $$\Sigma \rightarrow U\Sigma U^{\dagger} = \text{diag}\{\sigma'_1, \ldots, \sigma'_{T}\} = \Lambda\\ \sigma'_1 < \sigma'_2 < \ldots < \sigma'_T.$$ My idea is to project out the larger-variance vectors in the diagonalized covariance matrix. In effect, I would have the following: $$\Sigma_{T\times T} = U^{\dagger}_{T\times T}\Lambda_{T\times T}U_{T\times T} \rightarrow U^{\dagger}_{T\times T-d}\Lambda_{T-d\times T-d}U_{T-d\times T} = \tilde{\Sigma}_{T\times T}$$ where $$d$$ are the number of large-variance eigenvectors I would remove from the covariance matrix (a parameter that for the time being I can tweak once I figure out how to implement this).

There are two problems with this approach. First, my new covariance matrix $$\tilde{\Sigma}$$ is not full rank (null space is size $$d$$). I cannot invert this matrix and minimize my fit function. The second problem is, once I project out the large variance eigenvectors, how exactly should I transform my residual vector $$y[t] - f(t)$$ so that I don't bias my fit result?

This appears to be some kind of PCA application towards fitting curves. However I have not had much luck finding sources that use this kind of method for reducing noise for regression. Could you guys point me towards the right direction?