what is the difference between binary cross entropy and categorical cross entropy? So, I made a bidirectional LSTM model for sentiment classification. Model's job was to predict ratings of movies(1-5 stars) based on the movie review.
While training the model I first used categorical cross entropy loss function. I trained the model for 10+ hours on CPU for about 45 epochs. While training every epoch showed model accuracy to be 0.5098(same for every epoch). 
Then I changed the loss function to binary cross entropy and it seemed to be work fine while training. So, I want to know what exactly is the difference between these two?
 A: I would like to expand on ARMAN's answer: 
Not getting into formulas the biggest difference would be that categorical crossentropy is based on the assumption that only 1 class is correct out of all possible ones (so output should be something like [0,0,0,1,0] if the rating is 4) while binary_crossentropy works on each individual output separately implying that each case can belong to multiple classes (for instance if predicting what items a customer will get it is possible that they will buy multiple ones; i.e. output like [0,1,0,1,0] is a valid one if you are using binary_crossentropy). As ARMAN pointed out if you only have 2 classes a 2 output categorical_crossentropy is equivalent to 1 output binary_crossentropy one. 
In your specific case you should be using categorical_crossentropy since each review has exactly 1 rating. Binary_crossentropy gives you better scores but the outputs are not evaluated correctly. I would also recommend trying to use MSE loss since your data is ordinal (4 stars are closer to 5 than 1)
