$p$-value when standard deviation is zero What test can be used to indicate a $p$-value when both sets of data in a comparison have standard deviation of 0? For example, Set 1 has values of 1, 1, 1 and Set 2 has values of 5, 5, 5.
Or is it OK to just say that since the standard deviation are 0 for each set and no numbers are the same in each set, that the $p$-value is <0.05?
 A: *

*I think it may make sense to conduct a statistical test on this kind of data, but you haven't given much context to know what could be done.

*You definitely cannot just conclude that $p < 0.05$ just because there is no variance in the samples.  One problem is that to reach a p-value, you need to define a null hypothesis.  It's not clear from your question that you've defined a null hypothesis. (What kind of equivalence would be looking for?  Means, medians, stochastic equality?) A second problem is that you still need to take the sample size into account.  Imagine the edge case where you have one observation for each sample.  Can you jump to $p < 0.05$ in this case?

*One case you might get data like in your example would be if there are two candidates for a job, say, and you have three independent ratings for each, on a discrete 1 to 5 scale, like a Likert scale.  In this case, we can treat the responses as ordered categories and conduct a Cochran-Armitage test. The following does this in R, using functions from a couple of different packages.
Another option may be certain permutation tests.
if(!require(coin)){install.packages("coin")}
if(!require(multiCA)){install.packages("multiCA")}

Input =(
"Rating      1 2 3 4 5
Set
Set1         3 0 0 0 0
Set2         0 0 0 0 3
")

Table = as.table(read.ftable(textConnection(Input)))

library(coin)

chisq_test(Table,
           scores = list("Rating" = c(-2, -1, 0, 1, 2)))

   ### Asymptotic Linear-by-Linear Association Test
   ###
   ### data:  Rating (ordered) by Set (Set1, Set2)
   ### Z = -2.4495, p-value = 0.01431
   ### alternative hypothesis: two.sided

library(multiCA)

multiCA.test(Table)

   ### Multinomial Cochran-Armitage trend test
   ### 
   ### data:  Table
   ### W = 6, df.Set = 1, p-value = 0.01431
   ### alternative hypothesis: true slope for outcomes 1:nrow(x) is not equal to 0

A: The short answer is that you cannot do a statistical test on this type of data. The reason is because there is no way to measure chance variation.
I don't remember where I heard this analogy, but this may be a case where you need to ask yourself if a statistical inference is necessary. The analogy is ¿What is the $P$-value to assess that chickens have fewer legs than cows? Of course, barring any anomalies, your data sets will be {2,2,2,...,2} and {4,4,4,...4} (not even sure a sample size for either needs to be provided).
Hope this helps.
