My question pertains to section 2, called "Multi-class Logistic regression", of this pdf, especially the update rules. (The entire section is only a couple of paragraphs.)
Everything seems to make sense, but I don't see how one is supposed to implement the gradient ascent algorithm.
- What is $y_i^k$? What are its dimensions?
- What is the dimension of the weights?
- What is $x_i$? Say you have a matrix where each row is an observation and each column is a feature. What would $x_i$ be? Is it a row?
- Is $p(k|x_i)$ the same thing as $p(y = k|x)$ for a particular observation?
Side note: I'm trying to implement a simple multi-class logistic regression in MATLAB.