34
$\begingroup$

Problem

In regression one usually computes the mean squared error (MSE) for a sample: $$ \text{MSE} = \frac{1}{n} \sum_{i=1}^n\left(g(x_i) - \widehat{g}(x_i)\right)^2 $$ to measure the quality of a predictor.

Right now I'm working on a regression problem where the goal is to predict the price customers are willing to pay for a product given a number of numeric features. If the predicted price is too high no customer will buy the product, but the monetary loss is low because the price can simply be decremented. Of course it should not be too high as then the product may not be bought for a long time. On the other hand if the predicted price is too low, the product will be bought quickly without the chance to adjust the price.

In other words the learning algorithm should predict slightly higher prices which can be decremented if necessary rather than underestimating the true price which will result in an immediate monetary loss.

Question

How would you design an error metric incorporating this cost asymmetry?

Possible Solution

A way to define an asymmetric loss function would be to simply multiply by a weight: $$ \frac{1}{n} \sum_{i=1}^n \left| \alpha - \mathbb{1}_{(g(x_i) - \widehat{g}(x_i)) < 0} \right|\cdot \left(g(x_i) - \widehat{g}(x_i)\right)^2 $$ with $\alpha \in (0,1)$ being the parameter we can adjust to change the degree of asymmetry. I've found it here. This seems like the most straight forward thing to do, while maintaining the quadratic loss.

Improve this question
$\endgroup$
11
  • 1
    $\begingroup$ @MichaelChernick, FTR, I think this is a good question, which has been stated clearly & coherently, & acknowledge that I'm being a bit nit-picky. What I'm getting at is (as you know) fitting a regression (ie, solving for $\boldsymbol{\beta}$) is done (by default) by minimizing the OLS loss function, SSE. You're right that MSE could be used equivalently b/c dividing by a constant won't affect the ordering of candidate betas. $\endgroup$
    – gung - Reinstate Monica
    Sep 25, 2012 at 15:56
  • 1
    $\begingroup$ Another fact is that MSE (more often RMSE) is often used to assess the quality of a fitted model (although, again, SSE could be used equivalently). The thing is, this question seems (to me anyway) to be about how to think about / redesign the loss function, so that the fitted betas are different than they would have been by default, rather than about how to think differently about the quality of a model that has already been fit. $\endgroup$
    – gung - Reinstate Monica
    Sep 25, 2012 at 15:59
  • 1
    $\begingroup$ @Kiudee, if my interpretation of your Q is right, what would you think about editing it to add the loss-functions tag, & possibly revising the title to something like: "How to design & implement an asymmetric loss function for regression"? I won't make the edits myself in case you disagree w/ them. $\endgroup$
    – gung - Reinstate Monica
    Sep 25, 2012 at 16:05
  • 2
    $\begingroup$ For reference, I've seen quantile regression suggested when you want asymmetric loss functions, see Berk, 2011, PDF here. $\endgroup$
    – Andy W
    Sep 25, 2012 at 20:09
  • 1
    $\begingroup$ As I'm using a variety of learning algorithms to tackle this problem, the function should be differentiable at least once. $\endgroup$
    – Kiudee
    Sep 26, 2012 at 20:12

2 Answers 2

Reset to default
11
$\begingroup$

As mentioned in the comments above, quantile regression uses an asymmetric loss function ( linear but with different slopes for positive and negative errors). The quadratic (squared loss) analog of quantile regression is expectile regression.

You can google quantile regression for the references. For expectile regression see the R package expectreg and the references in the reference manual.

Improve this answer
$\endgroup$
2
$\begingroup$

This sort of unequal weighting is often done in classification problems with two classes. The Bayes rule can be modifed using a loss function that that weights the loss higher for one error than the other. This will lead to a rule that produces unequal error rates.

In regression it would certainly be possible to construct a weight function such as a weighted sum of squares that will give some weight to the negative errors and a higher weight to the positive ones. This would be similar to weighted least square but a little different because weighted least squares is intended for problems where the error variance is not constant over the space of possible values for the predictor variables. In that case the weights are higher for the points where the error variance is known to be small and higher where the error variance is known to be large. This of course will lead to values for the regression parameters that are different from what OLS would give you.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.