0
$\begingroup$

I have an idea to help reduce the noise in my signal but am stuck with a significant problem.

I have a very noisy data set $y_n[t]; n\in\{0, N_{\text{samples}}-1\}; t\in\{0, T-1\}$ I am fitting this signal to a non-linear function $f(t;\vec{\theta})$ ($\vec{\theta}$ is my vector of parameters in my non-linear function), by minimizing the log-likelyhood function $$ \text{log}(\mathcal{L}) = \big(y[t] - f(t,\vec{\theta})\big)\big(\Sigma^{-1}\big)_{t,t'}\big(y[t'] - f(t',\vec{\theta})\big) $$ where $$ \Sigma_{t,t'} = \text{Cov}(y[t],y[t']). $$ I know that there exists unitary operators $U$ that can diagonalize the covariance matrix into components of least-to-most variance in the data-set $$ \Sigma \rightarrow U\Sigma U^{\dagger} = \text{diag}\{\sigma'_1, \ldots, \sigma'_{T}\} = \Lambda\\ \sigma'_1 < \sigma'_2 < \ldots < \sigma'_T. $$ My idea is to project out the larger-variance vectors in the diagonalized covariance matrix. In effect, I would have the following: $$ \Sigma_{T\times T} = U^{\dagger}_{T\times T}\Lambda_{T\times T}U_{T\times T} \rightarrow U^{\dagger}_{T\times T-d}\Lambda_{T-d\times T-d}U_{T-d\times T} = \tilde{\Sigma}_{T\times T} $$ where $d$ are the number of large-variance eigenvectors I would remove from the covariance matrix (a parameter that for the time being I can tweak once I figure out how to implement this).

There are two problems with this approach. First, my new covariance matrix $\tilde{\Sigma}$ is not full rank (null space is size $d$). I cannot invert this matrix and minimize my fit function. The second problem is, once I project out the large variance eigenvectors, how exactly should I transform my residual vector $y[t] - f(t)$ so that I don't bias my fit result?

This appears to be some kind of PCA application towards fitting curves. However I have not had much luck finding sources that use this kind of method for reducing noise for regression. Could you guys point me towards the right direction?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.