Consider a competition with 10,000 entrants and 200 judges. Each entrant gets scored on a scale of 0-100 by 2 different judges for a total of 20,000 scores.
I want to remove any judge-to-judge variations in their means and standard deviations. To do this I'm using a Z-score for each judge's scores and converting that to a T-score to put in back on a 0-100 scale.
In R I'm doing
df$z_score <- ave(df$score, df$judge, FUN=scale)
df$t_score <- ave(df$score, df$judge, FUN = function(x) rescale(x, mean=50, sd=10, df=FALSE))
in Python the code would be
df['Z-Score'] = df.groupby('judge')['score'].transform(lambda x: stats.zscore(x, ddof=1))
df['T-Score'] = df['Z-Score'].transform(lambda x: x * 10 + 50)
However, for a variety of reasons, some judges only scored a handful of entrants. Let's say between 3 and 20.
Is it valid to calculate a Z-score/T-score for those particular judge's scores as the mean and standard deviations may be skewed due to the small sample or should I run a different test?