Hypothesis testing that a sentiment appeared more often Im new to statistics and have a question that I cant answer. It's also difficult for me to tell if an online guide has the right solution for my question. So any help from you would be great.
What I did is following: For ten thousands of text documents, I predicted their sentiments based on a machine learning model. So the prediction is either 'negative', 'neutral' or 'positive'. My model does not always predict the correct sentiment and has an accuracy of about 70% (for all documents, about 70% of sentiments should be predicted correctly). For simplicity, I would ignore this 70% correctness issue and just change the confidence interval from 95% to 99% later, because my solution does not require to be  perfect. Probably I also would not understand a complicated statistical solution. Next, I counted how often each sentiment appeared. Results look like this: 15% negative, 14% neutral and 71% positive.
I would like to prove now that documents with a 'positive' sentiment appeared statistically significant more often than 'neutral' and 'negative'. Based on my statistical understanding, sentiments are categorical (nominal) values, thus Chi-square testing with goddness of fit. In general, could you please help me or post a link to set up the calculation? Also, is h0: 'All sentiments appeared in equal frequencies' and h1: 'Positive sentiments appeared more frequent than neutral and negative'?
Example data would look like this:
Document, Predicted Sentiment

*

*'I am so happy today', 'Positive'

*'I did not like the food!',
'Negative'

Thank you in advance
 A: In part, you are looking for a test of binomial proportions. In R, one way to do both of the tests you want is with the procedure prop.test.
Suppose you have a vector x with 100 responses 1 Negative, 2 Neutral, 3 Positive,
as generated in R below:
set.seed(909)
x = sample(1:3, 100, rep=T, p = c(.2,.2,.6))
table(x)
x
 1  2  3 
20 18 62 

Test that all three categories are equally likely. The null hypothesis
$H_0: p_1 = p_2 = p_3$ against the alternative that the three probabilties
are not equal is rejected with P-value near $0.$
prop.test(c(20,18,62), c(100,100,100))

       3-sample test for equality of proportions 
       without continuity correction

data:  c(20, 18, 62) out of c(100, 100, 100)
X-squared = 55.56, df = 2, p-value = 8.616e-13
alternative hypothesis: two.sided
sample estimates:
prop 1 prop 2 prop 3 
  0.20   0.18   0.62 

Test that Positive is less likely than Negative and Neutral together,
rejected with P-value $< 0.001 - 0.1\%.$
prop.test(c(38, 62), c(100,100), alt="less")

    2-sample test for equality of proportions 
    with continuity correction

data:  c(38, 62) out of c(100, 100)
X-squared = 10.58, df = 1, p-value = 0.0005716
alternative hypothesis: less
95 percent confidence interval:
 -1.0000000 -0.1170907
sample estimates:
prop 1 prop 2 
  0.38   0.62 

