3
$\begingroup$

We have the following dataset:

$$ \begin{bmatrix} x_1 & x_2 & y\\ +1 & +1 & +1\\ -1 & +1 & +1\\ 0 & -1 & +1\\ 0 & 0 & -1 \end{bmatrix} $$

I was asked to find a mapping $\varphi$ to 3-dimenssion such that in the new dimension this data will become linear separable, and find this linear separator.

Here is the answer, I'd love if someone can explain me how to get it, and give me some intuition, because I didn't understand it:

$$\varphi(x)=(x_1~,~~x_2~,~~x_1^2+x_2^2)$$

By choosing the weights $(w_1,w_2,w_3,b)=(0,0,1,-0.5)$ resulting the predictor:

$$h(x_1,x_2)=sign(<w,~\varphi(x_1,x_2)>)=sign(<(0,0,1,−0.5), (𝑥_1 , 𝑥_2 , 𝑥_1^2+x_2^2, 1) >) = sign(x_1^2+x_2^2-0.5)$$

Thanks!

$\endgroup$

1 Answer 1

0
$\begingroup$

It's just mathematical intuition. The table says that if any of $x_i$ has absolute value $1$, the result is $1$, otherwise $0$. You can fit any other mapping to satisfy this condition. For example, $|x_1|+|x_2|$ would be another answer, so as $x_1^4+x_2^4$.

As for the weights, since we have found a mapping that defines the label, other features have no importance. Thus, their weights can be $0$, and weight corresponding to the mapping can be $1$.

The bias term is adjusted such that if the features are $0$, $\varphi(x)+b$ is negative (such that sign function can give $-1$). This could be $-0.6$ as well, but typically, the middle point is chosen, due to robustness reasons.

$\endgroup$
6
  • $\begingroup$ Thank you for your answer, I understand now that if any of the features has absolute values of $1$ then the classify is $1$ - but I do not understand what you mean by "you can fit mapping" to this - what is the intuition to fit a mapping should be? Do I need to find something that does the same in a higher dimenssion? $\endgroup$
    – user361992
    Commented Jun 30, 2022 at 8:07
  • $\begingroup$ I meant you can come up with a different formula satisfying this condition. I have given two alternatives to $x_1^2+x_2^2$. By saying "the table says", I meant, "the table says for me", and I realized it by intuition. Your intuition (or the author's intuition) could be something else, and they could have come with another interpretation. Given the size of the problem, coming up with a formula is just a brain exercise. $\endgroup$
    – gunes
    Commented Jun 30, 2022 at 8:13
  • $\begingroup$ Yes, I understand that there are multiple solutions, I just didn't understand how you came up with yours (but I understand its correctness) $\endgroup$
    – user361992
    Commented Jun 30, 2022 at 8:29
  • $\begingroup$ It is partly intuition. I came up with it the moment I looked at it. In such a small problem, this is nothing to be surprised of. $\endgroup$
    – gunes
    Commented Jun 30, 2022 at 9:00
  • $\begingroup$ If you can share more about it, that will be wonderful! $\endgroup$
    – user361992
    Commented Jun 30, 2022 at 9:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.