basically I would like to solve this problem:
(1) say I have
N features that I want to transform with a generic
f(x, theta) where
theta is a continuous bounded variable
(2) I know that each variable has got an optimal different value of theta
(3) these features will be fed into a simple regression model, of which my objective is to maximize
Now I can set this up as an optimization problem where I minimize
-R2 as a bounded optimization problem. the question is: which is the "optimal" algorithm to use in this kind of cases? optimal = relatively good at finding "minima" with relatively few iteractions
I am familiar with the methods described here (https://docs.scipy.org/doc/scipy/reference/optimize.html) but perhaps there's something different to go about this kind of problem. maybe what I am looking for is something different altogether. I am especially troubled by getting stuck in local minima and by running a lot of iterations (which would be costly from a computational perspective.
p.s.: for the purpose of this, please know I have no "true"/"prior" thing that I can tether to, just some educated guesses as to the starting points.