0
$\begingroup$

I intend to conduct a multiple linear regression using large US Health Survey data. However, having looked at one of the variables relating to income, it seems that it has been top-coded at the 95th percentile.

As such I was wondering what the impact on the regression would be if I was to delete those observations which have been topcoded, and indeed whether this would be the best thing to do?

My understanding is that the top-coded data (which comprises 1.3% of the total data) would produce a bias in the data and therefore in the regression model. Therefore, as I mentioned, it should be removed?

Your help is much appreciated!

$\endgroup$
1
$\begingroup$

This top coding (called capping below) can be thought of as general strategy of 'treating outliers'. Imagine that an extremely rich person (example: Bill Gates) was part of the data. Then the income value for that person will be very high. But we sort of know that the person is not representative of general population.

There are two ways to handle it.

Capping/Top-coding Few Affected Variables

Capping is introduced so that model does not learn to correlate extremely high incomes with outcome variable. But at the same time, there could be other variables (say height) on which Bill Gates is not going to be an outlier. It is better to keep the record for the person but cap outlier variables like income.

Eliminating Rows

If you suspect that most of the variables/columns in the data are outliers for that person or record, you can eliminate the entire row. For example, because of high income, may be Gates can always get treated at best hospitals and so on. So if all other variables like health indicator are likely to be outliers for the person, you eliminate the record. Although this case is expected to be rarer, since we hope nature would not bestow special powers on people with more money :-)

Bottom line: check what other variables are and then take a call.

$\endgroup$
  • $\begingroup$ Thank you for your answer. Just to confirm; the 'capping' as you described is the top-coding which those who collected the survey data have already done to the data right? $\endgroup$ – dayleymart Jan 4 '18 at 4:26
  • $\begingroup$ Also, whilst those outliers have been accounted for by top-coding, isn't the fact that there are so many values (1.3% of total data) which take the same income value an issue which causes bias? Therefore requiring the removal of those entries? $\endgroup$ – dayleymart Jan 4 '18 at 4:30
  • $\begingroup$ ^ Yes, top-coding and capping are the same. I don't think 1.3% of total data having same value will introduce a bias. First, 1.3% is not a large percentage of data. Also, think of it like this: rather than saying 1.3% of the people have same income, we say that 1.3% of the people have so much money, that actual value of income matters little. All that matters is that they are (very very) rich. $\endgroup$ – hssay Jan 4 '18 at 4:33
  • $\begingroup$ Sorry to keep bothering you. But given that the top coded value is $2045 and the nearest value to it is $20043, I can't seem to concurr that that's what the data is saying.From my perspective being top-coded so close to the last non top-coded value doesn't show that those folks are much richer? $\endgroup$ – dayleymart Jan 4 '18 at 4:37
  • $\begingroup$ ^ top coding tries to ensure that high income values don't have undue influence on your model. Top-coding doesn't show that some folks are more richer, it says that all people above certain amount of money are treated equally by the model since beyond a (large) point more money doesn't matter (in this particular problem). Hope this helps. $\endgroup$ – hssay Jan 4 '18 at 5:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.