1
$\begingroup$

I have a sample of data with N values from which I calculate basic moments such as mean, standard deviation and skewness. I will then change these moments to different values according to my own rules, these new values are not based on any existing sample.

I would then like to transform the original sample into another with the same size (N) and that best approximates the new target moments. This new sample could be computed from scratch or it could (preferrably) be iteratively constructed based on the sample from which I calculated the original moments.

Is there any known algorithm or method that does this (either doing from scratch or changing an existing sample)?

Thank you very much.

$\endgroup$
2
  • $\begingroup$ I think you would have to optimise your own figure-of-merit. Clearly changing any of mean, SD, skewness affects the others. SD must be positive for skewness to be defined; that will be evident to you; but skewness is also bounded as a function of sample size $\endgroup$ – Nick Cox Feb 9 '15 at 19:42
  • 1
    $\begingroup$ See Boltzmann's Theorem for one example of such a recipe. Whether it is appropriate for you depends on your objective in making this transformation. $\endgroup$ – whuber Feb 9 '15 at 19:55
1
$\begingroup$

This is related to equating (e.g. here). In the case of equating you take two variables $X$ and $Y$ (e.g. two tests taken by a single group of students) and transform the distribution of $X$ in such a way that it resembles the distribution of $Y$. If you want to preserve only the first two moments you could use linear equating:

$$ \operatorname{Lin}_Y(x_i) = \frac{\sigma_Y}{\sigma_X}x_i + \left( \mu_Y - \frac{\sigma_Y}{\sigma_X}\mu_X \right) $$

However if you want to be more precise and match the quantiles, then a little bit more advanced method is needed, i.e. equipercentile equating, where you use cumulative distribution functions and their inverses of $X$ and $Y$:

$$ \operatorname{Equi}_Y(x_i) = F^{-1}_Y \left[ F_X(x_i) \right] $$

So the general idea is that you can find such values $u_i$ that range from $0$ to $1$ that correspond to certain values of $F_X(x_i)$ and then find values of $F_Y(y_i)$ that correspond to those values:

$$ F_X(x_i) = u_i = F_Y(y_i) $$

enter image description here

Of course this works only for continuous data, so with discrete data you need an additional step - "smoothing" of data. For this purpose kernel smoothing is often used.

Below you can find an example of R library that implements those methods (req4 from example(equate)). Empirical distribution x is transformed to match y, where yx is the outcome. As you can see, resulting mean, standard deviation, skewness and kurtosis of the outcome are close to the target.

Equipercentile Equating: rx to ry 

Design: equivalent groups 

Smoothing Method: loglinear presmoothing 

Summary Statistics:
    mean   sd skew kurt  min   max    n
x  19.85 8.21 0.38 2.30 1.00 40.00 4329
y  18.98 8.94 0.35 2.15 1.00 40.00 4152
yx 18.98 8.99 0.35 2.21 0.01 40.05 4329
$\endgroup$
3
  • $\begingroup$ How does this work with mean, SD, skewness? $\endgroup$ – Nick Cox Feb 9 '15 at 19:56
  • 1
    $\begingroup$ I see why the percentile-equating method might give a starting approximation to a solution, but I do not understand why it should match moments. $\endgroup$ – whuber Feb 9 '15 at 20:08
  • $\begingroup$ Thanks @Tim, I will study these examples. Skewness is of special importance to me, I might even be able to ignore changes to mean and sd and focus solely on achieving a sample with the target skewness. $\endgroup$ – Chicoscience Feb 9 '15 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.