3
$\begingroup$

I have an array of 50,972 household incomes for a small metro area, and I want to measure how much the so-called top "one-percent" make (e.g., the 99th percentile). As you'd expect, it's a power curve:

enter image description here

To start, I just sorted the array and took the last 510 values, which begin at \$385,244 and go up to \$1,070,357.

First question: To be in the top 1%, do you have to make AT LEAST the cutoff or MORE than the cutoff? It makes very little difference in this case, but I'm curious whether the percentile values are considered inclusive.

R has a quantile function that can find the cutoffs with less effort:

distribution = quantile(income_array, prob = seq(0, 1, length = 101))

(The 101 is to account for the 0% up to the 100%, I believe.) If you look at the 99% value, it matches my manual calculation of \$385,244 exactly, while the 100%, naturally, matches \$1,070,357.

So we know how much you have to make to be in the vaunted 1%. But in terms of expressing how MUCH richer that 1% are than the median--which is an important policy consideration--I'm not certain if it makes sense to use the lower bound of \$385K or some median value within that top slice.

In other words, the top one-percent themselves are also weighted to the richest of the rich.

enter image description here

For example, would it be more accurate to take the median of the 1%? (\$584,438, which I realize is the same as cutoff to be in the top 0.5%).

I'm not sure if this is relevant, but the R percentile function optionally takes a type argument that specifies which of 9 different algos to use, documented here. This is beyond my level of stats prowess. The default is 7.

I should note that the data has many repeat values because it comes from a carefully weighted sample of households. Each income is repeated in the array a number of times equal to the household's weight, such that the outcome is equivalent to the number of households.

$\endgroup$
  • 2
    $\begingroup$ "This is beyond my level of stats prowess" - 2 observations: 1. Use logarithmic scaling to compress the range. 2. Are you concerned about "income", "wealth", or both? - "Wealth is a function of numerous elements: Beyond income, there is spending, savings, investments, stock option compensation, business ownership and more." Source: Graphing Income and Wealth - Look at the Graphic: some people earn various amounts (bottom of the chart) yet are worth nothing, some claim they earned nothing (left side) yet are worth from \$1K to \$100M. $\endgroup$ – Rob May 26 '18 at 6:37
  • $\begingroup$ Good question. Unfortunately, the American Community Survey doesn't account for net worth. However, in this case the more relevant variable is income (which includes all forms: wages, dividends, etc.). I'm interested in how income for the highest earners has grown compared to the middle class. I also have data on cost of living (including health care) to then layer over this. $\endgroup$ – Chris Wilson May 26 '18 at 17:28
  • $\begingroup$ Unfortunately I am deep into trying to answer another question, but after 5 min. of looking around: "... the American Community Survey doesn't account for net worth." It seems to. -- "I'm interested in how income for the highest earners has grown compared to the middle class." Answer: The highest earners probably get better at hiding their "earnings" and earn money where it's not reported to the ACS; they can afford to, and can't afford not to. $\endgroup$ – Rob May 26 '18 at 17:57
  • $\begingroup$ As best I can tell, table C25079 only goes up to a 2012 1-yr sample, sad to say. I'm using IPUMS, which has the microdata I need for a granular analysis, and I can't locate a single variable that represents net worth. The Census has this analysis more recently -- might be CPS, not ACS, and CPS has many methodological errors. $\endgroup$ – Chris Wilson May 27 '18 at 19:30
  • $\begingroup$ But I take your very astute point about the vagaries of high-income earners. I think this is a case where we have to make-do with the best-available information, fully caveated, which is total income (from any source--wages, farms, rental, etc) $\endgroup$ – Chris Wilson May 27 '18 at 19:32
2
+25
$\begingroup$

If you would like to get a clearer visualisation of the income distribution, you should create a density plot and a cumulative distribution plot of income on a logarithmic scale. Presently your top graph is an inverted cumulative distribution plot on a linear scale, and this leads to the "squished" results you see on the plot. You might also consider creating a plot of the Lorenz curve, which is a useful plot for measuring inequality in a variable, and using this to report the Gini coefficient, which is a common measure of inequality. Make sure all your plots are clearly labelled, with titles, axis labels (with units), and clear numbering.

  • Cumulative distribution plot on a logarithmic scale: The horizontal axis of this plot is Household Income ($) and the vertical axis is the Number of Households with income no greater than this amount. Household income should be displayed on a logarithmic scale, so that large values do not dominate the plot.

  • Kernel density plot on a logarithmic scale: The horizontal axis of this plot is Household Income ($) and the vertical axis is the Density of households with income at that amount. Household income should again be displayed on a logarithmic scale, so that large values do not dominate the plot.

  • Lorenz Plot: The horizontal axis of this plot is the cumulative Share of Households (%) from lowest to highest household income, and the vertical axis is the cumulative Share of Income (%) corresponding to these households. The Lorenz plot also generally shows a line-of-equality representing perfect equality of distribution on income. The plot can also include the value of the Gini-coefficient, which is the proportion of the area between the line-of-equality, and the actual Lorenz curve, as a proportion of the area under the line-of-equality.

The appropriate percentile corresponding to the top 1% of households (by household income) can be estimated either by the appropriate percentile of the kernel density estimate or by interpolation from the data. (Since you have a large amount of data, any reasonable technique will give similar values.) You can create the above plots either for the dataset as a whole, or for the top 1% of households, and so on.

Comparisons between the top 1% of households and the bottom 99% should be made on the basis of consistent measures of income for those groups. Hence, you are correct that it would not make sense to compare the median of the bottom group (or the overall median) to the minimum of the top group. Comparing these groups could be done using the median household income of both groups, or the mean income of both groups, etc., but you must use a consistent measure of income for the comparison.

$\endgroup$
0
$\begingroup$

I have another perspective to add to the nice answer of Ben.

With any sharp/hard cut in statistics, there come several caveats. Most of the issues arise from the underlying mathematics, especially analysis. Out of these general parts two are most important in that case:

  • cutting close to the end of an interval; which in your case is of course related to your original question

  • cutting at a point where there's a large change, or more precisely where the derivative is large

The first issue simply comes with the nature of the question. But it still is relevant, because it contains by definition the end of the scale.

The second issue directly touches your concern of accuracy. If the distribution were continuous instead of discrete, then you'd have a rather large effect on the "topmost 1%". So a small change $\pm \epsilon$ in the cut value leads to a rather large change in the income value to make the lower cut.

The core question "how can a distribution represented with a single number" is a general question of descriptive statistics. If you can find an approximate description of your distribution, this might be a shorter way to represent this. If this a parametric description, all the better -- but this in itself is a potentially complicated task. In the end you might end up with a result of describing the "top 1%" for example like:

Interval [350k, 1.1M], second order polynomial with coefficients $\alpha_0, \alpha_1$

$\endgroup$
1
$\begingroup$

This reminds me of a concept of value-at-risk (VaR) in financial risk management. If we applied it here, then the cutoff would be VaR and the average of all who made cutoff would be conditional VaR (CVaR) and an expected shortfall. Here's the formulae: $$\alpha=CDF(VaR)$$ first you solve for VaR value to match the quantile, or significance.

Second, you get the conditional on making the cut, what is the average income: $$CVaR=\frac 1 {1-\alpha}\int_{VaR}^\infty xPDF(x)dx$$

You can compare this to the expected value $E[x]$, which would be a mean of returns of entire population

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.