# Encoding quantitative outputs for regression

From Elements of Statistical Learning:

For a two-class G, one approach is to denote the binary coded target as Y , and then treat it as a quantitative output. The predictions Yˆ will typically lie in [0,1], and we can assign to Gˆ the class label according to whether ˆ y > 0.5. This approach generalizes to K-level qualitative outputs as well.

I am python programming the least square regression in section 2.3.1 of the book.

p_or set of orange points, p_bl set of blue points which i concatenate to X

#X the set of all points
X = p_or+p_bl
m = len(X)
#Add column of ones to account for the bias term
X = np.array([list(np.ones(m)),[x[0] for x in X],[x[1] for x in X]]).T
#y is coded as -1 for orange, 1 for blue
y = np.array( list(-1*np.ones(len(p_or))) + list(np.ones(len(p_bl))) )
#calculating βˆ = (X_T*X)^(−1)*X_T*y
beta = np.linalg.inv( X.T @ X )@( X.T @ y )

#Plot the boundary!
line_x = np.linspace(-5, 5)
line_y = -beta[0] / beta[2] - (beta[1] / beta[2]) * line_x

plt.plot(line_x, line_y)


Question is the following - I encoded categorical outputs with -1 and 1 y = np.array( list(-1*np.ones(len(p_or))) + list(np.ones(len(p_bl))) ). This way the regression looks great. If I encode with 0 and 1 regression doesn't look good : regression boundary is biased towards one of the two sets of points ( graphically on the plot ). More rigorously, how should be a categorical variable encoded ? is there a demonstrable rule for this ?

• Look into logistic regression. Search this site. – kjetil b halvorsen Apr 18 at 18:48
• Could you be more specific ? It seems to me logistic regression with encode with 0s and 1s. Furthermore Elements of Statistical Learning, the part I am trying to implement, is about least square estimation, not logit. – kiriloff Apr 19 at 19:26