Is there any direct relation in between accuracy and loss while performing text classification using neural network? I am trying to perform text classification using the deep recurrent neural network. My network is incurring a huge loss of 94%, 80% and sometimes 100% with certain accuracy. It is surprising that with 64% validation accuracy the incurred loss is 96%. I want to comprehend that whether the incurred loss has direct relation to accuracy or accuracy is being calculated on correctly acquired data. I am using the categorical crossentroy function to estimate the loss. I am aware that the loss is a distance from true values and loss function basically calculate the error rate. But how this can be related to accuracy. For instance both the accuracy and loss is high than what does this depicts?

 A: A high cross entropy and a high accuracy are not necessarily inconsistent results.
Here's a simple example. Suppose you have a problem with 10 classes, and a data set with 1 sample of each class. In 9 of the samples, the correct class has prediction $0.85$, but in one class it has prediction $0.001$. The cross-entropy loss in this case is almost $0.84$. This is because very incorrect predictions are penalized much more steeply than slightly incorrect predictions.
On the other hand, the accuracy of the model is 90%, because in 9 out of 10 cases, the correct class has the largest predicted probability.

First, you've written cross-entropy as a percent. This doesn't make much sense, because cross-entropy is not a part of a whole.
Also, you write

I am aware that the loss is a distance from true values and loss function basically calculate the error rate.

This is incorrect. The error rate of a model is the proportion of samples classified incorrectly. We can write $1 = \text{accuracy} + \text{error rate}$ because these names describe complementary concepts. Cross entropy measures "how wrong" a prediction is (larger values mean the prediction was more wrong), but this measurement of wrongness doesn't involve classification at all. Instead, cross-entropy penalizes predictions which are far from their correct values.
