# Regression over weighted datasets

In the context of machine learning, most examples of regression (linear, ridge, elasticnet) tend to split along the 50/50 probability line.

In the chart below, suppose that a red event is 10x more costly than a blue event. Because of that it might be desirable to incorrectly classify some more blue events (as red events) in the interest of correctly classifying more red events.

Is there an easy way to weight the regression in favor of one dataset over another? One might imagine the regression in that case to be closer to the red line than the dashed line.

... or is that the wrong approach entirely?

Two examples are linear discriminant analysis and logistic regression. In LDA, a hyperplane $\vec \beta \cdot \vec x = \beta_0$ is drawn through the parameter space -- corresponding to the red and dashed lines that you have shown here. The canonical LDA procedure produces a value for $\beta_0$ (based on certain assumptions that aren't important here). However, you are completely free to choose $\beta_0$ yourself. All this parameter sets is the location of the hyperplane along its normal, e.g. whether to the pick the red line or the dashed line in your graphic. Similarly, in logistic regression, the regression produces an estimate for the log-odds of membership in one of the two classes. Given a uniform prior, the line where the estimated log-odds is zero corresponds to the line where an event has a 50-50 chance of membership in either class, e.g. $\hat \pi = 0.5$. However, you are perfectly free to assume a different, non-uniform prior. In practice this ends up being the same as saying that you can pick a separation value other than $\hat \pi = 0.5$.