2
$\begingroup$

I have a list of stock trades made by US senators and the exact amount of money they spend on a trade is not disclosed, they only give a range of values. Below is the frequency table for the 8600 trades.

enter image description here

Here is a visual distribution enter image description here

There is a clear trend in the histogram, which is why I am asking, what method I should use to approximate the mean value for each bucket? I would like to utilize this exponential trend instead of just using the midpoint of each bucket, as it is highly skewed.

My initial idea was to find a line of best fit for the histogram, and then integrate over each bucket and divide by the width of the bucket to find the mean of each bucket. Part of my goal for the project is to estimate how much money each trade makes.

Improve this question
$\endgroup$
5
  • $\begingroup$ Interesting question! What assumptions can we make about the data? Would normality be reasonable? Or are these data known to have a skew? $\endgroup$ – PAF Apr 29 '20 at 7:48
  • $\begingroup$ Data like this are, and are expected to be, highly skewed and far from normal. $\endgroup$ – Nick Cox Apr 29 '20 at 8:48
  • 1
    $\begingroup$ That single 50M+ trade contributes an appreciable proportion of the total. The trades above 5M contribute about half the total. Thus, any results about averages or totals you estimate will be exquisitely sensitive to the details of just nine trades--only about 0.1% of them all. Perhaps, then, the best thing you could do would be to investigate those 9 trades. If you can find out which stocks were traded, for instance, you might be able to find the exact amounts in the public records of those companies. $\endgroup$ – whuber♦ Apr 29 '20 at 12:46
  • $\begingroup$ I think based off of the histogram, I might be a little more precise with my mean values for each bucket than if I just assume normality for each bucket. For instance with the first bucket, 1,001-15,000, my guess would be that there is another similar logarithmic distribution, with many more trades near 1,001 than 15,000. $\endgroup$ – Paul M Apr 30 '20 at 2:14
  • $\begingroup$ Also that is an interesting comment by whuber, for instance Berkshire Hathaway is a ~300,000 dollar stock. However I think with the relatively new addition of buying fractions of shares, it is a little less clear, as I found someone buying Berkshire with <15,000$. $\endgroup$ – Paul M Apr 30 '20 at 2:18
1
$\begingroup$

The histogram doesn't help more than the table does already. The bins are roughly logarithmically spaced, but I would still focus on the table.

I am usually highly in favour of a graphical or geometric approach, but all the information is in the table any way. Graphical integration is in principle what you want but where does the integral stop?

I would estimate a weighted mean using the frequencies and the geometric mean of bin endpoints. For the one person in the top bin, try once with USD 50 million and once with say USD 100 million.

Assuming no measurement error (!) you can bracket estimates by assuming first that all people are at the lower end of their bin and then that they all are at the upper end. You still need some device for the uppermost bin.

I would note that even the median has to be guessed at with some ad hoc interpolation.

Whatever you do:

  1. Vary the recipe so that you get a good idea of how sensitive results are to your assumptions.

  2. Consider the very strong possibility that there isn't enough information to get out a good estimate of the mean. Why do you want or need it any way? The table and histogram are imperfect, but still informative. (Statistical advice that you really need the full data is natural but futile here.)

EDIT

The calculations are, or should be, mundane. Recall that the geometric mean of two values is the square root of their product. Working with geometric means is equivalent to working on logarithmic scale. Here lower, geometric mean and upper values for each bin are used in weighted means, first with the uppermost bin just USD 50,000,001 and then with its upper limit that plus another USD 50 million. The results do underline how fraught is this exercise. A one-sentence summary would just be that the mean necessarily depends on the total and the total must in turn depend on the distribution within bins and most of all on the very largest value. (Stata output, although that doesn't matter.)

More optimistic one-liner: Mean perhaps around USD 50000. 

 . su lower geomean upper [fw=freq]

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
       lower |      8,600    31852.74    676837.3          1   5.00e+07
     geomean |      8,600    48887.25    827200.4   122.4786   5.00e+07
       upper |      8,600    90684.14     1170382      15001   5.00e+07

. replace upper = 100e6 + 1 in L
(1 real change made)

. replace geomean = sqrt(lower * upper)
(1 real change made)

. su lower geomean upper [fw=freq]

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
       lower |      8,600    31852.74    676837.3          1   5.00e+07
     geomean |      8,600    51295.47    987294.6   122.4786   7.07e+07
       upper |      8,600    96498.09     1496963      15001   1.00e+08

.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the detailed response! The integral for the buckets <50,000,000 is from the min to the max of each bucket. For instance, integrating from 1,001 to 15,000, then dividing by the width (15,000-1,001) to get the mean value. My hope is that this will capture the skewed nature of graph, as I suspect that there are many more trades between 1,001-8,000 than the other half of the bucket, 8,001-15,000. Sorry if I misunderstood your explanation $\endgroup$ – Paul M Apr 30 '20 at 2:12
  • $\begingroup$ My use of the geometric mean is based on precisely the same idea of skewed distribution within each bin. There is clarity about bin limits -- except for the uppermost. $\endgroup$ – Nick Cox Apr 30 '20 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.