1
$\begingroup$

I'm working on a regression problem with imbalanced data, and I would like to know if I'm weighting the errors correctly. I'll try to illustrate the concept with a simple example.

Imagine I'm building a model to predict house prices in New York and Los Angeles. I have many more training examples in NY than in LA, but I want the algorithm to perform equally well in both cities. To further complicate the issue, house prices in NY have a greater variance than those in LA.

Here is an example training dataset:

City    N_rooms  House_Price
NY      4        400
NY      7        1000
NY      5        800
NY      3        300
NY      7        600
NY      2        100
NY      4        500
LA      3        400
LA      5        500
LA      4        500

I have 7 training examples for NY and 3 training examples for LA. If my cost function is MSE, namely sum((y_pred - y_true)^2)/10, to make sure that the algorithm performs equally well in both cities, I would need to give different weights on the prediction errors, namely

sum(w * (y_pred - y_true)^2)/10

I would like to know which one of the following would be the correct way to define w and/or rescale training data:

  1. Do not use weights (i.e., w=1)
  2. Define w as the inverse frequency of each class in the training set, namely w=1/3 for houses in LA and w=1/7 for houses in NY
  3. Standardize prices in NY and LA separately, namely subtract the average price in NY from the price of every house in NY, then divide the price of every house in NY by the standard deviation of house prices in NY. Similarly, subtract the average price in LA from the price of every house in LA, then divide the price of every house in LA by the standard deviation of house prices in LA. Now train the regression model on the scaled data. To predict actual prices, apply the inverse scaling to the model predictions.
  4. Apply both points 2 and 3.

Note: the goal is not only to minimize the overall error, but to build an algorithm that performs equally well in both cities.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.