2
$\begingroup$

Hope you can help me with my 2 (or 3) questions here below.

I want to add/sum some variables having different units. I decide to standardize (Z-scores) the values and then, once transformed in Z-scores, I can sum them.

The problem is that all my variables distributions are not approximatively gaussian. I know that my distributions don't need to be gaussian to calculate Z-scores, however, if the distributions are not close to gaussian, I will give more or less weight to unusually high or low values. As I don't want to give more or less weight to some values, I decide, for the distributions that are skewed (and or have some outliers) to use the Sn robust estimator, in the robustbase package like this :

enter code hereA=Dataset$A enter code hereSn(A, constant=1.1926)

For the distributions that are not close enough to gaussian (skewed and/or outliers), if I use the MAD (median absolute deviation), I can combine Z-scores from regular and robust estimators, but to make the standard deviation of the robust estimator equivalent to the standard deviation of the regular standard deviation, we need to do the following : (Value – Median) / (1.486 x MAD) However, if my data is skewed using MAD may not be a good idea : MAD would overestimate the variability of my data. That is the reason why I use the Sn robust estimator.

1) If I decide to use the robust Sn estimator, can I calculate a kind of Z-score using this formula : (Value – Median) / Sn ? If not, what would be the formula to get Z-scores ?

2) Now, if my distribution is (slightly) skewed and have long-tailed, what would you use ? MAD ? Sn ? Other robust estimators ? And which formula would you use to calculate the Z-scores ?

I build a synthetic indicator of the quality of the educational systems.

I have 5 criteria : a) effectiveness of the education system b) efficiency of the education system c) equity of the educations system d) student engagement e) teacher engagement

The synthetic indicator of quality is composed of five criteria (here above) themselves consisting each of 6 statistics, a total of 30 statistics. My goal is to sum /add all these 30 statistics to get a score and then to be able to establish a ranking. As all 30 statistics are not of the same units, their aggregation is only possible by using the Z-scores method of removing from each observation the mean of all the observations and dividing the result by the standard deviation.

By suming these 6 statistics transformed in Z-scores (so summing 6 Z-scores) for each of the 5 criteria and then by dividing by 6, I will obtain a general score for each of the 5 criteria (I could say an averaged Z-score for effectiveness, for efficiency, for equity, for student engagement and for teacher engagement). Then, I sum these 5 general averaged Z-scores (the averaged Z-score for effectiveness, the averaged Z-score for efficiency, the average Z-score for equity, ...), I will obtain my final Z-scores.

So my question is the following : Out of the 30 statistics that are summed/added, some distributions are not so closed to gaussian (at least are not symmetrical enough) and may have outliers and some have long-tailed. I have all kind of weird distributions. Which robust estimators to use to calculate robust Z-scores ? MAD ? Sn ? AO ? Other ? And thus which formula to use to calculate thses Z-scores ?

$\endgroup$
  • $\begingroup$ I would use the AO score instead of the Z score. The AO score is described in equation 3 of this paper. You can start by setting the MC to 0 to simplify the computations --in which case you only need access to a function to a quantile() function --like quickselect-- $\endgroup$ – user603 Oct 19 '17 at 22:39
  • $\begingroup$ Many thanks for your suggestion. I did not know the AO. After having had a glance at equation (3) as suggested by you, let me know if I am wrong but the AO can not be negative ? $\endgroup$ – varin sacha Oct 20 '17 at 11:56
  • $\begingroup$ Yes you can! I guess, I mean you can always multiply $AO(x_i, X_n)$ by $\text{sign}(x_i-\text{med}(X_n))$. I'm not quiet so sure about the idea of summing these things though. What are you trying to accomplish is not all that clear to me quiet yet. See, the premise: ''I want to add/sum some variables having different units.'' and I wonder ''but why?'' $\endgroup$ – user603 Oct 20 '17 at 13:18
  • $\begingroup$ My comment was too long, so I decide to edit my previous question and at the end of my previous question, I precise my ultimate goal. Many thanks for your time. $\endgroup$ – varin sacha Oct 20 '17 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.