Is it possible to design a network to solve a multiple-output regression task with increasing number of outputs, so the difficulty of the task?

I am trying to solve the problem of frequencies estimation for sum of sine waves with NN. From

$$ x(t) = \sum_{k=0}^K a_k sin(2 \pi f_k t) + \text{noise}$$

predict $a_k$ and $f_k$. However there is an order in the data: $a_k > a_{k+1}$ and $f_k < f_{k+1}$.

I am familiar with the solution presented in the paper Neural networks for sinusoidal frequency estimation. I implemented it: the input is the signal with $K$ sine waves. The output are the $\{a_k, f_k\}_{k=0}^{K'}$ with $K' \leq K$.

For $K' = 2$ is working fine. For $K' > 2$ the estimation error is not as good, no metter the architetture (It tried with RNN and CNN or simple Feedforword NN). I noticed that also the prediction of $a_1, f_1$ for $K' > 2$ is worst then when $K'=2$.

The question: is there a DNN technique to start the learning with $K' = 2$ and then sequentially increase it?

I was thinking: is it right to use an NN with an output layer of dimension $2K$, and predict only 2 values, while $2K-2$ are forced to be zeros. So train for some epochs, then unfreeze the next 2 values, then repeat?

  • $\begingroup$ How exactly does your data look like? $\endgroup$
    – Tim
    Commented Jan 17, 2020 at 16:36
  • $\begingroup$ The input consists in a vector 1024 samples matching the observation of 64 milliseconds of waves sampled at 16kHz. The $f_k \in [100, 1000]$ Hz, $a_k \in [0.2, 1]$. $\endgroup$
    – Chutlhu
    Commented Jan 17, 2020 at 17:17
  • $\begingroup$ Have you tried recurrent neural networks? $\endgroup$
    – Tim
    Commented Jan 17, 2020 at 17:52
  • $\begingroup$ Yes and it is slightly better, but still I am more interested in knowing about some techniques for such increasing-difficulty-task. $\endgroup$
    – Chutlhu
    Commented Jan 17, 2020 at 17:59


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.