0
$\begingroup$

Suppose that we have $y = Ax$ where $x$ is a vector of size $m \times 1$, $A$ is a sparse matrix with size $n \times m$. Suppose that $n \times m$ is very large and we wish to find a non-linear approximation of $y$. In other words, find a set of parameters $\theta \in \mathbb{R}^d$ that

$\min_\theta L(Ax, f_\theta (x))$

where $L$ is an objective such as L2 norm $||Ax - f_\theta(x)||^2_2$.

I think this is a known problem in optimization but I couldn't find a reference on common/prominent approaches for solving this problem. In particular, I'm looking for choosing good non-linear model $f_\theta$ and the number of parameters $d$.

$\endgroup$
0
$\begingroup$

You can use the non-linear steepest descent algorithm. To calculate the step size, take the partial derivative with respect to the parameters and evaluate at the current point (Jacobian).

https://en.wikipedia.org/wiki/Gradient_descent#Solution_of_a_non-linear_system

$\endgroup$
1
  • $\begingroup$ Are there common non-linear functions $f_\theta(x)$ for approximating sparse linear transformation? $\endgroup$ – KRL Jul 7 '19 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.