# Proper way to test proportions of data points output by a model

I have a dataset with a series $$1000$$ categoric observations. There are around 250 distinct categories, with $$20$$ distinct categories comprising around $$55 \%$$ of the dataset, thus most of them repeat many times.

My objective is to check whether a model I have, which simulates full datasets like the one I have, is able to accurately depict the proportion of observations for these $$20$$ observations.

With this model I have simulated $$200$$ series of $$1000$$ observations, with the intention to have robust mean and standard deviation estimators of the proportion.

Therefore, I have $$O_i$$, the observed frequency for observation of type $$i$$; $$\bar{M_i}$$, the mean frequency of observation of type $$i$$ in the modeled datasets and $$sd(M_i)$$, the standard deviation of the frequency across the $$200$$ modeled series, i.e something like this:

What would be the adequate approach here?

Further clarification

The observed dataset would be this way:

A, A, B, C, A, C, A, F, A, D, H, B, ...

And the models generate stochastic series (I have produced 200), e.g.:

A, B, A, A, E, C, A, C, D, D, K, A, ...

There series verify some particular characteristics, but for this particular question I have the objective of assessing whether the proportions of observations are being well captured by the model, es explained in the table above.

• Could you clarify what you mean by a "binary observations" and what it means to "comprise" part of a dataset? To me, a binary observation has exactly two possible outcomes--maybe three if you allow for missing values--and I just cannot make sense of the "20 distinct observations" from that perspective. – whuber Jan 14 at 18:33
• It's so unclear what you want. As I understood, you have 20 distinct data points which some of them are being repeated, and you want a model to produce those repeating points. If I'm correctly understood you, you should add a new feature as n_repeat and try to model your data. Otherwise, a model cannot have several distinct outputs. – Mehdi Jan 14 at 22:23
• @whuber Sorry, that was a typo, I meant categorical variables. I have added further clarification to the question, hopefully it is clear now. Thank you very much for your time. – D1X Jan 15 at 8:54
• @Mehdi I have edited the question to make it more clear. Thank you very much for your time. – D1X Jan 15 at 9:05
• Instead of "20 distinct observations" would you mean "20 distinct categories"? Why do you simulate series of 2000 observations in order to study a dataset of just 1000 observations? What exactly do you mean by "robust," given it doesn't seem to be used in the usual statistical sense (of a procedure that makes minimal distributional assumptions)? – whuber Jan 15 at 13:54