# Why is $y=w^TX+b$?

In, say, a linear regression problem, why does the dot product of the weight vector (w) and feature vector (X) result in the value of the dependent variable (y) i.e., why does addition of products of weights and features result in y? Could we have chosen an operator other than addition, to act upon the product of the weights? I understand the intuition behind y=mx for a single variable, but why do we choose to addition as the operator in case of multiple features?

• it's just as you described it for the scalar case but now there is more than one predictor so $X$ now becomes an $n \times k$ matrix because there are $n$ responses and $k$ predictors. . So, you just stack the responses on top of each other but it's still linear relation on each of the responses. A more standard notation is $y = X \beta$ where $X$ is $n \times k$ and $\beta$ is $k \times 1$. The $X$ is assumed to have ones in its first column and $w_{0} = 1$ so no need for $b$. The linear model is viewed as a projection of $y$ onto the basis spanned by the columns of $X$. – mlofton Dec 16 '18 at 7:26
• The "linear" in "linear regression" means the expectation of the response is a linear combination of the explanatory variables: that is, a sum of "weighted" values. Your question thereby seems to be without any content and needs no answer. But perhaps you are trying to ask why people tend to work with linear regression models rather than nonlinear models (where $y$ is not a linear combination of $X$)? – whuber Dec 16 '18 at 21:22

Moreover, it can easily become equivalent to other algebraic operations: if weights have negative signs, it is the same as we subtracted parameters multiplied by positive weights $$a + (-w)b = a -wb$$, and if we log-transform the variables, then summation becomes multiplication $$\log(ab)=\log(a) + \log(b)$$.