Evaluate correlation between two datasets I created an algorithm that evaluates the user experience concerning web applications depending on response times and errors. That is, one of the following five states is calcualted: Very Happy, Happy, Ok, Unhappy and Angry. Furthermore, I ask the users to enter one of the five states manually. 
Now I have the following table with about 120 entries showing the automatic calculated user experience and the corresponding user experience the particular user has entered:
    ID      Automatically Generated              Entered By User

    1            Very Happy                        Very Happy

    2            OK                                Happy

    3            Angry                             Unhappy

    4            Happy                             Happy

    5            Very Happy                        OK

    6            Angry                             Unhappy
...

What I would like to know is how much my calculated results deviate from the actual values the users have entered. Is there any kind of correlational coefficient that can tell me if my algorithm is correct or not?
 A: I don't think that correlation in its core definition will help you here, correlation measures the association between two variables, and while it is applicable to discrete variables (such as your case), it is more efficient to be applied on continuous variables. Your goal is not to give better association (when X increases, Y increases). Your goal is to make them identical ($x_1 = y_1, x_2 = x_2... x_n=y_n$), or at least, as many points as possible to be identical to the actual ones selected by users.
I have made a simulation for your problem using R code:
actual <- ceiling(runif(120, 0, 5))
auto<- ceiling(runif(120, 0, 5))

To get some data to work on. I think you can use Mean Square Error:
MSE <- sum((actual-auto)^2)

Which as you may know the sum of squared differences between your auto-generated and actual responses, now you have a measure of similarity/difference and your job is to minimize this measure MSE. The less MSE you get, means your prediction is closer to the actual responses and an MSE of 0, means that your prediction is exactly equal to the actual responses. Notice that MSE always has a minimum value of 0 as it is a sum of squared differences.
A: You may want to explore Cohen's Kappa for two raters under the {irr} R package. Following your categories, and generating 100 observations (50 random; 50 with complete agreement)...
ID <-  c(1:100)
Autom_Gen <- c(sample(c("VH","OK","A","H"), 50, replace = T),rep(c("VH","OK","A","H","A"),10))
Enter_User <- c(sample(c("VH","OK","A","H"), 50, replace = T),rep(c("VH","OK","A","H","A"),10))
dat <- data.frame(ID,Autom_Gen,Enter_User)

dat[1:2, ]

  ID Autom_Gen Enter_User
1  1         A         OK
2  2         A          A

require(irr)
kappa2(dat[,c(2,3)], "unweighted") 

# Columns 2nd and 3rd = Automated rating vs. User
# All disagreements equally important.

 Cohen's Kappa for 2 Raters (Weights: unweighted)

 Subjects = 100 
   Raters = 2 
    Kappa = 0.491 

        z = 8.46 
  p-value = 0 

# There is agreement ~ 50% of the times as per our set up.
# And logically, we should conclude with this p value that this
# kappa level of agreement (1 being perfect) is statistically significant.

You may also want to look into the paper:
"Statistical Inferences for Interobserver Agreement Studies with Nominal Outcome Data" by Bartfay E and Donner A in The Statistician (2001) 50, pp. 135 - 146.
