Quality Control Chart for non-normal distributed data

Question

I am interested in building quality control charts for time series data where every data point has different gradation within the range of 0 to 1. For example first data point X1 may be 0.1, 0.2,..,1 and second data point X2 may be 0, 1/3, 2/3, 1. Third data may have some another interval.

The data is not seasonal and there is no pattern in how the intervals look like (e.g., 0.1(X1), 1/3(X2) etc.). In other words I don't know what is the gradation of future data points but know that the range is between 0 and 1.

My questions are

1) Wether the limits that I will build based on some training data (UCL, LCL) will be effected as a result of having different gradation - the variance will be higher since the data is not continuous?

2) Is it a problem given the fact that this represents the nature of future data and that I have a training set of 30 samples?

3) The data points represent performance of student over different tests in consequent time, and were normalised to be between 0 and 1 (10 questions in test will lead to interval of 0.1, 3 questions will lead to interval of 1/3). Maybe there are some ways to normalise the data to avoid the problem of blowing up the variance?

Many thanks for your patience and help!

Answer 1

If I understand correctly, you will use the 30 samples to estimate the in-control (IC) mean, which in this case is a actually proportion on the interval (0, 1). When you start monitor, will you monitor each individual's scores or the average score of all the students? If it's the average of all students, you may be able to get away with using standard control charts that assumes Normality if the subgroups are big enough.

One place you might want to look is the 1998 Hawkins and Olwell text. In section 5.7 they describe weighted binomial CUSUM charts. These are charts designed to monitor the probability parameter of binomial distributed data and don't require the same number of trials (test questions in your example) at each observation.

It's fairly likely that there are more sophisticated methods available and I wouldn't be surprised if this was still an open research area. You may also want to see what Google Scholar says when you look for control charts for binomial distributed data.

Answer 2

The proper chart for plotting the percent defective over time would be a $p$ chart, however, the sample size should not be less than 50. You can monitor the class of 30 students this way because, with three questions and 30 students, the number of possible defects becomes 90.

You could track each of your questions using a binomial distribution, testing your hypothesis of how difficult your questions are (e.g. probability of successfully answering your question is 0.80) and you can use the sample size of 30 for each question.

A sample size of 3–10 for each quiz is too small, and without establishing your hypothesised pass rate at the beginning you would need too many exams to establish the $\overline{p}$ for the $p$ charts by student—and that hypothesis may only be meaningful if your questions pass their hypothesis test on their chart. Distinguishing between the hypotheses may be difficult. Another option would be to use a $u$ chart with the $\overline{u}$ set at the average required to pass the class (e.g. 0.70), but you are still going to struggle with far too small of a sample size for discrete data.