Designing Asymmetric regression (assymettric loss for regression) I have a hybrid classification/regression problem.The predicted value can be assumed to be centred around 0. I want to penalize the predictor more, if the predicted value and actual value have opposite signs. The reason is some kind of demand prediction,for the same degree of offset from the actual demand, its better to predict the trend of demand correctly, as that reduces the loss.
One idea I have is this
$$
 \sum_{i=1}^n w_i  \exp{\bigl[-y_i\widehat{g}(x_i)/\alpha\bigr]} \cdot \left[y_i - \widehat{g}(x_i)\right]^2
$$
where 
$$
\widehat{g}(x_i) = \beta^Tx_i
$$
is just the linear predictor. We can think of $w_i$ as weights (e.g. for sampling), probably equalling $1/n$. My questions are:
1) Does someone know a similar problem in academic literature or actual use, (basically a hybrid mix of classification and regression, so that wrong sign error have higher weight).What kind of error function should I use, since the once I seemed to have chosen has bad convergence properties.
2) How do i optimize this kind of function. I havent checked but it probably is convex.
3) Does python or R have some library or package with this algorithm or, some kind of automatic minimzation algo.
4) How do i scale $\alpha$, as clearly its value needs to be determined
Finally I hope to get this done  on large dataset so probably some efficient way to evaluate the model will be great
As far as why asymmetric treatment is done, its quite simply that even if you are wrong $y_i\ne\widehat{g}(x_i)$ , the trend is in the correct direction. I'd think that $\alpha$ is s.t. $\exp{\bigl[-y_i\widehat{g}(x_i)/\alpha\bigr]} \approx 1/n$ should be used, so that some points are not overly weighted, but want some ideas for that.
Update1: (partly to @StasK)
I tried going by iterative LS, and seem to run into convergence issues even on a toy example. Is there some known things about ITLS convergence, or is it my issue. One thing which I do more is to snap some of the  $\beta$ coefficients to 0, or maybe there is some other thing which i did wrong
Update2:
On 3 different examples one quite realistic, I found that in the IRLS the β tends to get close to 2 different points and keeps jumping between them, likely somethign the weights keep changing drastically across those two. I tried to trim the exp number to a max value, and even then this seems to happen.So I am looking to redesign my loss now.
 A: There is no shortage of optimizers in R. I will let a more knowledgeable R user recommend the most appropriate one(s).
A cheap solution is to iteratively reweight your data, treating $r_i^{(k)}=\exp\bigl[-\alpha^{(k)} y_i\hat{g}^{(k-1)}(x_i)\bigr] $ as a fixed weight within iteration $k$ (I took a liberty to move $\alpha$ from the denominator to the numerator), where $\hat{g}^{(k-1)}=x^T\hat\beta^{(k-1)}$ is the result of the weighted least squares from the previous iteration. So


*

*Set iteration counter $k=0$, $\alpha^{(k)}=0$, run OLS to get $\hat\beta^{(0)} \equiv \hat\beta_{\rm WLS}$

*Update $k \leftarrow k+1$, $\alpha^{(k)}=\alpha$, $r_i^{(k)}=\exp\bigl[-\alpha^{(k)} y_i\hat{g}^{(k-1)}(x_i)\bigr] $

*Run weighted least squares $\sum_{i=1}^n w_i r_i (y_i - x'\beta)^2 \to \min_\beta$

*Iterate 2--4 a few times until the coefficients and/or the weights $r_i$ stabilize.


As an option to improve stability a little bit, you can devise a trajectory for $\alpha^{(k)}$, so that for the first say 10 iterations, $\alpha^{(k)}=k/10 \cdot \alpha$ for your selected ultimate $\alpha$, and then you stick $\alpha^{(k)}=\alpha$ for the remaining $k=11,12,\ldots$
Given how quick regression is, I won't be surprised to see that this converges faster than the full non-linear optimization. The standard errors, however, will be too small, as they don't account for uncertainty in $r_i$ (due to uncertainty in $\hat\beta$ that the weights $r_i$ utilize internally). You can get the right standard errors from the sandwich variance  formula.
Finding the "right" value of $\alpha$ must come from the same (substantive) considerations which dictate the need for asymmetric treatment of the regression error.  Given that you did not explain what these are, I am not sure how to advise on that.
