PSU, or the primary sampling unit, is the object or a group of objects what you sample in the first stage of a multi-stage sample. Typically in large scale national studies, this could be a county or a census tract. Then you go down to the level of city blocks (secondary sampling units), dwellings, households, and individuals. So when you sampled Autauga County, Alabama (one of 3K+ counties in the US, the first that comes on the standard lists), you have to think of the 50,000 people that live in it as a single unit for variance estimation purposes. Of course, you would likely subsample this county, and end up interviewing may be 10 people. However, most of the contribution to the variance comes from the first stage, especially when observations with the PSU are similar to one another.
This is the standard formula for the variance of a clustered sample; a common knowledge, if you like. There is no simple explanation for it sans the derivation from the first principles. You would have to look at a standard survey statistics book, such as Lohr 2009, Korn & Graubard 1999 or Thompson 1997 (in an increasing order of complexity and mathematical rigor).
The first principles of finite population sampling are really orthogonal to anything you've learned in statistics (be it mainstream or Bayesian or machine learning). What you measure on the sample elements is considered fixed (someone's weight or height or color of their eyes; and that make sense, except for some measurement error: your height tomorrow should not differ from your height today, so how can it be random?). What is random, however, are the indicators of the finite population elements being taken into the sample. In other words, if you talk about sampling 1000 people from US population, you are talking about a 300-million dimensional vector that has zeroes for most people who did not make it to the sample, and ones for the 1000 people who were sampled. Thus, the probability spaces that you would encounter in the world of sample survey are discrete (although combinatorially huge), and so are the sampling distributions of the sample statistics, although the latter would sometimes be well approximated by the normal distributions. The CLT-type justifications, however, are way more complicated in survey statistics, as appropriate CLTs have only been proven in limited contexts of specific sampling designs. You would need to get used to thinking in terms of the totals (because they are the only linear statistics of the random elements); to the weighted mean being a biased estimator of the population mean (as it is a ratio estimator, i.e., a non-linear statistic); and to variance estimation being an order of magnitude more complex than point estimation.
While Phil Kott is a very wise guy who writes quite well, I doubt that this paper is a good starting point on survey statistics. You must have been thrown into this quite harshly, I gather, to have to read this out of blue sky.