0
$\begingroup$

I am doing research in (multi-touch) attribution models. The goal is to assign credit to each channel. I have several models that assign normalized attribution to the channels.

I have simulated some data to illustrate it.

Model 1:
Channel 1: Attribution = 0.80, SD = 0.030
Channel 2: Attribution = 0.10, SD = 0.001
Channel 3: Attribution = 0.10, SD = 0.003

Model 2:
Channel 1: Attribution = 0.30, SD = 0.010
Channel 2: Attribution = 0.30, SD = 0.010
Channel 3: Attribution = 0.40, SD = 0.015

What is a "fair" way to say something about the stability of the model?

Options:

  1. Calculate the average SD.

Model 1 = 0.0113 ((0.030 + 0.001 + 0.003) / 3)

Model 2 = 0.0116

  1. Compute the average standard deviation as a percentage of the average attribution.

Model 1 = 2.5833 ((0.8/0.03*100) + (0.1/0.001*100) + (0.1/0.003*100) / 3)

Model 2 = 3.4722 ((0.3/0.01*100) + (0.3/0.01*100) + (0.4/0.015*100) / 3)

  1. Or are there other options to compare the SD between models?

Does anyone know a how to compare the standard deviation between models?

$\endgroup$
  • $\begingroup$ While it would be useful to know more about the metric(s) with which attribution is being evaluated it's not essential. Regardless of the metric, each attribution channel will have SDs unique to that channel thereby making the SD noncomparable between channels as well as models. Assuming the metrics are all nonnegatively realized, the coefficient of variation (CV=SD/mean*100) would be one comparable alternative. Frank Harrell has been an outspoken advocate for use of the GMD (gini mean deviation) as the single best metric of dispersion as testified to on his blog and in his books. $\endgroup$ – Mike Hunter May 24 '18 at 11:48
  • $\begingroup$ Thank you @DJohnson. Can you please elaborate on how one can determine the gini mean deviation (GMD) by means of my example? $\endgroup$ – Joep van der Plas May 24 '18 at 16:01
  • $\begingroup$ The formula for the GMD is in the cited literature. $\endgroup$ – Mike Hunter May 24 '18 at 16:54
  • $\begingroup$ There are several ways into calculating the Gini mean difference (which I think is a more common term). One is that it is twice the second L-moment, so search for L-moments code in your favourite software or one close to it. $\endgroup$ – Nick Cox May 26 '18 at 0:59
  • $\begingroup$ This article contains the most comprehensive discussion of the GMD that I've been able to find.... metron.sta.uniroma1.it/RePEc/articoli/2003-2-285-316.pdf $\endgroup$ – Mike Hunter Jun 4 '18 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.