2
$\begingroup$

Recently over a phone interview I was asked this:

"I have data consisting of 102 points between 0 and 100 where the first 50 points, starting at 0, are incremented by 0.5 every time. The next two are 50 each. The last 50, starting at 100, are decremented by 0.5 every time. In this scenario, what points would be the outlier?"

I was really confused by the question, but asking him to repeat several times it seemed like the graph would look like this.Assumed Plot[1]

I stated that outliers are values that reside beyond 1.5 times the standard deviation from the mean, but that applies only to normal distribution.

What would be the outliers in this case?

$\endgroup$
  • 3
    $\begingroup$ I share your confusion, it seems like a lousy question to me. You might find this thread about what an outlier is interesting, even if it doesn't illuminate the thinking of your interviewers: stats.stackexchange.com/questions/7155/… $\endgroup$ – mkt - Reinstate Monica Aug 2 '19 at 9:09
  • $\begingroup$ Or you could simply say, 'There's insufficient information to answer this well' $\endgroup$ – mkt - Reinstate Monica Aug 2 '19 at 9:19
  • $\begingroup$ Yes, asking such elaborate questions on phone interviews isn't a good idea in my opinion. Thank you anyways. $\endgroup$ – Aakash Dusane Aug 2 '19 at 14:59
  • $\begingroup$ Did they provide the plot / are you sure it's correct? I would interpret, "The last 50, starting at 100" to mean the x-values 100 - 149. $\endgroup$ – gung - Reinstate Monica Aug 2 '19 at 18:22
  • $\begingroup$ They did not provide the plot. But I'm certain about the fact that the range is between 0 and 100. And the final condition was that the last 50 points starting at 100 are decremented by 0.5 every time. $\endgroup$ – Aakash Dusane Aug 5 '19 at 4:25
3
$\begingroup$

(Turning my comments above into an answer)

I share your confusion, it seems like a lousy question to me. I'd be inclined to simply say, 'There's insufficient information to answer this well', because we know nothing about the data-generating process.

You might find this thread interesting, even if it doesn't illuminate the thinking of your interviewers: Rigorous definition of an outlier?

$\endgroup$
1
$\begingroup$

There is no one answer to this question.

To begin to understand why, you must first understand that the word "outlier" can refer to two different things:

  1. an "outlier" in the dimension of the response. Typically this is measured by a studentized residual. If the absolute value of the studentized residual is over two or three, then you might say it's an outlier in $Y$-space.

  2. an outlier in $x$-space, otherwise known as leverage. I think it's safe to say you don't have any here. If you did have a point that was an outlier in x-space (a high-leverage point), then you would have a dot way to the right or left of the rest of your points, ignoring how high or low it was on the y-axis.

However, you could have some leverage, depending on how you transform your predictor into more predictors, enlarging $X$-space. To measure leverage of an observation in this case, you can use diagonals of the hat matrix (otherwise known as the projection matrix.) They are always bounded between $0$ and $1$, but there are many rules-of-thumb for what merits are closer look.

It's hard to say if you have an outlier in $y$-space. This is because the value of the residual will be highly model-specific. For instance, I could come up with some transformations of your independent variable here that will give you a perfect fit, and all of your residuals will be $0$, and so therefore you will have no outliers in $y$-space either.

I wouldn't say the question is lousy. It might be, or it might not be. Interview questions are designed to elicit responses from the interviewee; they are not necessarily designed to be answerable.

$\endgroup$
-3
$\begingroup$

Regardless the distribution underlying the data, generally a definition of outliers in strict sense is "those observations that lay outside the interval $[Q1 - 1.5*IQR, Q3 + 1.5*IQR]$" where Q1 denotes the first quartile, Q3 denotes the third quartile and IQR=Q3-Q1. I am saying a definition of outliers "in strict sense" because, in addition to outliers, you may also have extreme values, that are often defined as "observations that lay outside the interval $[Q1 - 3*IQR, Q3 + 3*IQR]$". For practical purposes (not the phone interview in particular, that I suppose was a bit more theoretical requiring the florist definition), the ones that you often want to get rid of are extreme values, and I generally follow the second definition provided above to identify them.

$\endgroup$
  • $\begingroup$ So in this case numbers close to 0 and 100 would be outliers? $\endgroup$ – Aakash Dusane Aug 2 '19 at 15:50
  • 3
    $\begingroup$ I don't think it's accurate to say that, 'generally a definition of outliers in strict sense is "those observations that lay outside the interval [Q1−1.5∗IQR, Q3+1.5∗IQR]"'. That's a rule of thumb Tukey took from the normal distribution to use in constructing boxplots. It isn't necessarily true for normals (cf, here), it certainly isn't necessarily true elsewhere, & it isn't the real definition of outlier. $\endgroup$ – gung - Reinstate Monica Aug 2 '19 at 18:17
  • $\begingroup$ @gung that is what I would have answered as required by the question.. I do not think in a phone interview (that is often about mental calculations as I suppose in this case), you can start making the history of statistics to avoid answering.. just take the definition that you have and is accepted and fire the answer for that specific case.. I apologize if I said “generally” but I don’t think that crashing answers based on “generally” is a constructive approach.. and I see No Answers yet, just people saying the question was wrong.. I agree but you have to answer!!!!!!!!!!!!!!!!!!!! $\endgroup$ – Fr1 Aug 2 '19 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.