Deriving the cost function for a continuous bag of words model I've been reading the following link: http://cs224d.stanford.edu/lecture_notes/notes1.pdf, with the relevant content starting on page 6.
Some background on Continuous Bag of Words
The continuous bag of words (CBOW) model takes a context (e.g. a sentence with a word missing), and predicts a word that should fit into the context (e.g. the missing word). 
It starts by considering one-hot vector representations for each word in the input sentence: $x^{c-m},...,x^{c-1}, x^{c+1}, ..., x^{c+m}$, where $m$ is some integer context distance from the word to be found. 
Vector representations can then be found by doing $v_{c-m}=Vx^{c-m}$, and the same for every other one-hot vector, producing $2m$ results. These vectors are then averaged, to give $\hat{v}$. 
The score vector is $z=U\hat{v}$, and this is turned into a probability using softmax.
Cost function
The cost function to be minimised is: $$J=-\log{P(w_c|w_{c-m},...w_{c-1}, w_{c+1},...,w_{c+m}})=-\log{P(u_c|\hat{v})}.$$ Up to here makes sense to me. Obviously, the higher the probability of $w_c$, the lower the cost function is going to be, and $u_i$ is a vector of the $i$-th column in $U$. 
I'm struggling to understand the next part: $$J=-\log{\frac{\exp(u_c^T\hat{v})}{\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})}}=-u_c^T\hat{v}+\log{\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})}.$$
Could someone please break down this step to help me understand where it comes from?
Am I missing some fundamental understanding of linear algebra, or do I lack in understanding of the word vector representations?
 A: Understanding these steps requires a basic understanding of 


*

*The softmax function.

*Rules of logarithms.
Problem 1

$$-\log{P(u_c|\hat{v})}=-\log{\frac{\exp(u_c^T\hat{v})}{\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})}}$$

The softmax function is defined (as can be seen on https://en.wikipedia.org/wiki/Softmax_function) as $$P(y=j|x)=\frac{\exp(x^Tw_j)}{\sum_{k=1}^K\exp(x^Tw_k)},$$ where $x\longmapsto x^Tw_j \text{ }\forall j\in [1, K]$. 
Problem 2

$$-\log{\frac{\exp(u_c^T\hat{v})}{\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})}}=-u_c^T\hat{v}+\log\sum_{j=1}^{|V|}\exp(u_j^T\hat{v}).$$

Consider the following three log rules:


*

*$\log(\frac{a}{b})=\log(a)-\log(b)$

*$\log(a^n)=n\log(a)$

*$\ln(e)=1$
(In the case in the question, we are dealing with natural logarithms, so $\log = \ln$.)
Applying rule 1, we get $$-\log{\frac{\exp(u_c^T\hat{v})}{\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})}} = -\log(e^{u_c^T\hat{v}})+\log\Big(\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})\Big).$$
Applying rule 2 gives 
$$-\log(e^{u_c^T\hat{v}})+\log\Big(\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})\Big)=-u_c^T\hat{v}(\log(e))+\log\Big(\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})\Big).$$
Finally, applying rule 3 we get 
$$u_c^T\hat{v}(\log(e))+\log\Big(\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})\Big)=u_c^T\hat{v}+\log\Big(\sum_{j=1}^{|V|}\exp(u_j^T\hat{v})\Big),$$ which is the final answer.
