I'm struggling with understanding the underlying vector multiplication that are used to define a decision boundary for a binary classification. According to my textbook, this line may be expressed as: $y(x) = w^T.x + w_0=0$

Here, $w$ represents the parameter vector, and $x$ represents the input vector. I'm confused about the dimensions of these $2$ vectors in a situation where for example, 'height' and 'weight' are used to class somebody as 'overweight' or 'underweight'. We now have $2$ input variables, and let's assume we have $5$ rows of data.

Would this mean that $x$ is a ($5\times 2$) vector, and that $w^T$ is a ($2\times ?$) vector? If so, how would the dot product translate into a line? Because we would end up with a ($5\times ?$) vector, not a scalar.

I understand how $y(x)$ is used to classify a point, because here that would be a ($1\times 2$) . ($2\times 1$) dot product, giving a scalar. But in terms of $y(x)=0$, I'm not able to wrap my head around it.

I apologise if this is something really basic or if I'm just being silly and missing something glaring, but I really do appreciate any guidance!



For a single point $x$ is $1 \times 2$ vector.

We have $$y(x)=w^Tx + w_0=x^Tw+w_0$$

Now consider $X$ is a $5$ by $2$ matrix, it means that each row refers to a single point, we let $e$ be the all-one vector.

We have $y=X^Tw +w_0e \in \mathbb{R}^{5 \times 1}$. Each entry tells us if the corresponding row should be classified as positive or negative.

  • $\begingroup$ Thanks so much for responding. If I'm understanding correctly, each row is individually multiplied by the parameter vector, which can then be fed into an activation function? How does this fit into the idea that the linear decision boundary is represented as $w_0 +w^T.x$? Also, are we using dot product or cross product in these multiplications? Sorry again if these are basic questions, but my textbook moves really quickly over this. Thanks again! $\endgroup$ – fastingfeasting Jul 28 '20 at 10:52
  • $\begingroup$ We are using dot product. $w_0+w^Tx$ gives u a scalar, of which you can pass it to an activation function, for example, the sigmoid function. If $w_0+w^Tx>0$, you can view it as the positive class, if $w_0+w^Tx <0$, you can view it as another class. That is, $w_0+w^Tx=0$ can then be viewed as a decision boundary. $\endgroup$ – Siong Thye Goh Jul 28 '20 at 13:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.