Calculating confidence in part of a sample (explain like I'm in the humanities please :) )

Question

Working on a two-part project. First, we're estimating the total size of a very large population. Each member of the population has a standardized random 15-digit identifier, and we have guessed using a brute force method until discovering enough extant identifiers (20,000) to estimate the population size, with a confidence calculated using standard error of the mean. That part is ok.

The second part is a description of the 20k sample: demographics, etc. What I need to figure out is how to calculate the CI/error% of particular subsets of that sample. So if there's a group of 1,000 within the sample of 20,000 with unusual characteristics, I want to be able to describe those characteristics. Let's say it's a sample of people where one dimension of the data is income. There's a group of 1,000 CEOs in the group with a much higher income than everyone else. What I want to know how to figure out is how statistically significant the mean income of that 1,000 would be. It seems like it would require some compounding of the standard error of the sample with regard to the population and the sub-sample of CEOs in relation to the sample. Am I overthinking this?

Answer 1

...explain like I'm in the humanities please :)

The paradox of "confidence" relating to inference from sampling is an expression of the hermeneutics of anxiety, tracable to the observations of Kierkegaard. "Statistical Science" puts forward a symbolic synthesis to internalise and "objectify" the antithesis of passion and paradox inherent in inference of the unknown. By examining the hermeneutical dimensions of Kierkegaardian anxiety we can place "Statistical Science" within the superego of the subject, heavily affected by the genealogy of the control relations of the capitalist production system.

The statistical inferential problem itself is a manifestation of the subject anxiety induced by the precarious production relations of late-stage capitalism. Obsesssion with a socially constructed "model parameter" is observably a manifestation of the Lacanian objet petit a generated by the anxiety of the subject navigating the "objectivity" of the "scientific" enterprise. The symbolic language of the statistical machinery allows the subject to confine this anxiety and express uncertainty entirely within the known Symbolic Order. As Lacan has observed, the Symbolic Order forms a part of the Big Other and the striving for "confidence"; inferring the aforementioned model parameter surely represents striving for return to the womb of the Mother.

In The Subversion of the Subject and the Dialectic of Desire in the Freudian Unconscious, Lacan recognises that desire is "...a defense against going beyond a limit in jouissance" (p. 699). Such a defence mechanism is amplified in the capitalist production system, itself in perpetual crisis and so developing an expanding motif of symbols and language with which to internalise crisis. Žižek presents this as capitalism "borrowing from the future by way of escaping the future". Thus, one is unsurprised that the Symbolic Order of statistical "confidence" should be conceived as a mode of pictorialism for those unwilling to recognise the hermeneutics of anxiety within the capitalist model.

And this brings us to the notion of "calculating" the "confidence" relating to inference from the finite sample. This exercise reflects the language of symbolism developed as internalisation of the hermeneutics of anxiety. To proceed according to this motif, one sets in motion the Symbolic Order at the expense of the Real (and plays out the dialectic of desire). As Nietzsche counsels in The Gay Science, "[o]ne should have more respect for the bashfulness with which nature has hidden behind riddles and iridescent uncertainties".

; )

Answer 2

I promised a serious answer (to supplement my gag answer) so here goes. I think at present you are mixing several issues, including considerations relating to descriptive statistics for a sample and some (not clearly specified) population inferences. The main thing I would recommend at this stage is to take some time to consider and describe your sampling method, and in particular, determine whether it can reasonable be regarded as a "non-informative" type of random sampling. If you are using genuine random sampling then you will have available the entire suite of standard inference methods, including methods for construting confidence interval for population means, quantiles, etc.

With regard to your desire to describe aspects of your sample, this would typically be done using standard visualisations with accompanying descriptive statistics. The usual rule here is to remember that "a picture tells a-thousand words", so you should primarily be thinking about how to illustrate as much useful information about your sample using appropriate plots. When describing the distribution of continuous quantities across multiple discrete groups, a violin plot is a useful tool. When dealing with heavily skewed variables like income/wealth, you might also consider showing these on a logarithmic scale.

At the moment your problem is not that you are "overthinking" things --- you just don't seem to have a clear description (or even consideration?) of your sampling method or a clear description of what population quantities are of interest in your inference problem, and what descriptive aspects of the sample you wish to illustrate. Once you have a clear formulation of those things you will be in a better position to choose appropriate statistical methods for these tasks.

Answer 3

Reading some of your follow-up comments, it sounds like you want to provide descriptive statistics with confidence intervals.

The uncertainty around the total population size generally isn't a problem. Typically, you'd just assume that the total population is infinite. This is a conservative approach. The you follow the standard approach for computing confidence intervals.

A more precise approach takes the size of the population into account. The idea here is that if you have a population of 100 and a sample of 99, you actually almost know the mean of the population. There's a formula to shrink your variance estimate by this ratio. It's called the finite population correction.

In your case, there's some uncertainty about the total population, but if your estimate of the total population is unbiased, then your estimate of this ratio is unbiased and you're okay. You can see this with a little math. See my answer here to a similar question.

So you have two options, 1) just assume your sample of 1,000 CEO's is from an infinite population of CEO's or 2) apply a finite population correction using your estimate for the population size.

Unless you've sampled a large ratio of the total population, 1) and 2) will be about the same. So if you just want to keep it simple, you should go with 1). In which case, yes, you're probably over thinking things. :)