1
$\begingroup$

I could not understand the reason why PRML (Pattern Recognition and Machine Learning by Christopher Michael Bishop) says:

An interesting property of least-squares solutions with multiple target variables is that if every target vector in the training set satisfies some linear constraint $$ a^\top t_n+b=0 \tag{4.18} $$ for some constants a and b, then the model prediction for any value of x will satisfy the same constraint so that $$ a^\top y(x)+b=0. \tag{4.19} $$ Thus if we use a 1-of-K coding scheme for K classes, then the predictions made by the model will have the property that the elements of y(x) will sum to 1 for any value of x.

Why?
If the element of $y(x)$ sum to 1, it should be that $\mathbb{a} = \mathbb{1}$ and $b=-1$.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ although PRML book is famous, it is still nice to use the full name and provide the Amazon book link for more people to understand your question. $\endgroup$
    – Haitao Du
    Commented Sep 27, 2017 at 15:29
  • $\begingroup$ Some additional context for the formula would be helpful... What does it describe? What does the book say about it? $\endgroup$
    – Tim
    Commented Sep 27, 2017 at 15:34
  • $\begingroup$ @Tim thank you. I put the quote from the book, with some context. $\endgroup$
    – keisuke
    Commented Sep 27, 2017 at 15:40
  • $\begingroup$ Are you asking what a 1-of-K coding scheme is? Or are you asking about vector notation? $\endgroup$
    – whuber ♦
    Commented Sep 27, 2017 at 18:18
  • 2
    $\begingroup$ @whuber thank you. Now I could understand. Because $t_n$ is one-hot, it holds $a^\top t_n + b = 0$ --> every element of $a$ is $-b$. Then $a^\top y(x) + b = -b(1^\top y(x) - 1) = 0$, means the sum of elements of $y(x)$ is 1. (trivial solution $b = 0$ was ignored) $\endgroup$
    – keisuke
    Commented Sep 28, 2017 at 14:10

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.