I have an idea to help reduce the noise in my signal but am stuck with a significant problem.

I have a very noisy data set $y_n[t]; n\in\{0, N_{\text{samples}}-1\}; t\in\{0, T-1\}$ I am fitting this signal to a non-linear function $f(t;\vec{\theta})$ ($\vec{\theta}$ is my vector of parameters in my non-linear function), by minimizing the log-likelyhood function $$ \text{log}(\mathcal{L}) = \big(y[t] - f(t,\vec{\theta})\big)\big(\Sigma^{-1}\big)_{t,t'}\big(y[t'] - f(t',\vec{\theta})\big) $$ where $$ \Sigma_{t,t'} = \text{Cov}(y[t],y[t']). $$ I know that there exists unitary operators $U$ that can diagonalize the covariance matrix into components of least-to-most variance in the data-set $$ \Sigma \rightarrow U\Sigma U^{\dagger} = \text{diag}\{\sigma'_1, \ldots, \sigma'_{T}\} = \Lambda\\ \sigma'_1 < \sigma'_2 < \ldots < \sigma'_T. $$ My idea is to project out the larger-variance vectors in the diagonalized covariance matrix. In effect, I would have the following: $$ \Sigma_{T\times T} = U^{\dagger}_{T\times T}\Lambda_{T\times T}U_{T\times T} \rightarrow U^{\dagger}_{T\times T-d}\Lambda_{T-d\times T-d}U_{T-d\times T} = \tilde{\Sigma}_{T\times T} $$ where $d$ are the number of large-variance eigenvectors I would remove from the covariance matrix (a parameter that for the time being I can tweak once I figure out how to implement this).

There are two problems with this approach. First, my new covariance matrix $\tilde{\Sigma}$ is not full rank (null space is size $d$). I cannot invert this matrix and minimize my fit function. The second problem is, once I project out the large variance eigenvectors, how exactly should I transform my residual vector $y[t] - f(t)$ so that I don't bias my fit result?

This appears to be some kind of PCA application towards fitting curves. However I have not had much luck finding sources that use this kind of method for reducing noise for regression. Could you guys point me towards the right direction?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.