Effect size between 2 groups on a record-by-record and variable-by-variable basis

Question

Assume a dataset containing 50 variables about customers of your business. Each record represents 1 customer during the month of May. Most variables are not normally distributed and outliers are common.

In this scenario, you are comparing two groups of customers. The first group, the "gainers," gained sales in May. Group two, the "decliners," declined sales in May.

For a given decliner, you want to identify which variables for that customer are significantly different from the typical gainer.

You build an algorithm similar to Cohen's d to identify these different variables.

An example: For decliner "Chevy", and the variable "complaints"...

1. Calculate median value for "complaints" for gainers (mG)
2. Calculate std. dev. for "complaints" for gainers (sdG)
3. Calculate | (Chevy_complaints - mG) / sdG
4. Use #3 to determine if Chevy's "complaints" are significantly different than gainer "complaints" by comparing the value of #3 to some threshold, and if #3 exceeds that threshold, then "complaints" for record Chevy is flagged.
5. Repeat this process for all records and variables for decliners, resulting, for each decliner record, a list of flagged variables

My question is, for #1,2 above..

• If variables were normally distributed, mean would be preferred to median and sd would be an appropriate denominator for #3
• However, since median is being used in #1 to defend against outliers, should the value used in #2 be something like Mean Absolute Deviation instead of standard deviation? OR alternately, what if you calculated standard deviation, substituting median for mean, e.g. sqrt(sum(Xi - median(X))/n-1)

Thanks!

Answer 1

It sounds like question pertains to finding an appropriate standardized variable given your use of median over mean in the numerator. Consider the use of Interquartile Range to solve this problem.