26
$\begingroup$

I’m hiring for a mid-level data scientist with 2~ years of experience. The hire will be writing SQLs, working with stakeholders to define business problems and doing analytics whether an inference problem or predictive modeling.

One of our interview tasks is to examine the relationship between two variables in a table with 5 columns in total and quantify the impact of one on the other, which we mostly expect candidates to do EDA, fit a linear regression model, generate a regression table and interpret the statistics. And if they do well in the EDA, they will notice they need to include another covariate, hence fitting a multivariate linear regression, and interpret the statistics there.

I have noticed that multiple people with 5+ years of data science experience and advanced projects really struggle with this section, BOTH in generating a regression table, which I can fix by giving them the code to generate the table, AND interpreting the basic statistics (eg, why would the coefficient change for this variable when we add a covariate).

My question is do you think linear regression should be a basic skill for a mid-level data scientist and should it be part of the interview? Do you have a better recommendation on how to assess foundational skills of statistics?

$\endgroup$
13
  • 16
    $\begingroup$ Absolutely. I think linear regression is a fundamental skill in statistics, and I would question a candidate's capability if they can't do EDA, fit models, and interpret/diagnose them. $\endgroup$
    – Alex J
    Commented Mar 11 at 23:12
  • 14
    $\begingroup$ Do you need that employee to have that skill? Then it matters. $\endgroup$
    – Dave
    Commented Mar 11 at 23:41
  • 16
    $\begingroup$ Data science has always suffered from having high variability when it comes to roles definition and expectation. For a person to whom data science means working primarily with text or images, then linear regression may not be relevant. $\endgroup$ Commented Mar 12 at 1:08
  • 3
    $\begingroup$ Now I am curious to know if I would pass the interview. Might I ask: which is the element that should make you think to fit another covariate? A low R2? Analysis of residuals? $\endgroup$
    – rusiano
    Commented Mar 12 at 7:03
  • 3
    $\begingroup$ @MikeM that's a good way to find people who studied to the test and doesn't give you any indication if they actually know anything. but beyond that this isn't like expecting the person to have deep knowledge on how a random forest works, a 2 variable linear regression is so basic that it's a major red flag if someone struggles with it absent being told beforehand. i would go so far as to say giving explicit lists of topic items for an interview greatly diminishes the value of the interview from the company's perspective. $\endgroup$
    – eps
    Commented Mar 13 at 21:54

7 Answers 7

30
$\begingroup$

I think anyone who struggles with this is not a data scientist. Not an intermediate, not a beginner, not mid-level. Nothing. This is like not being able to connect to the internet, or not knowing how to program a FOR loop.

These are entry level skills.

And a company that hires people like this ... well, I think they are headed for disaster.

I've seen this. Not so much in my own work, but in reports from colleagues at statistics conferences where the friend will tell me about the huge messes they were called in to clean up.

$\endgroup$
1
  • 29
    $\begingroup$ One issue is that a lot of people today come up through a CS department, and many of them—I've interviewed a lot over the years—seem to have studied nothing but NN / deep learning technology, with maybe one stats course. There have, IIRC, even been questions on this site where the poster was surprised to learn you could do linear regression or logistic regression without stochastic gradient descent in an NN. $\endgroup$
    – jbowman
    Commented Mar 11 at 23:51
17
$\begingroup$

I have a somewhat contrarian opinion to Peter Flom.

This is a site for statisticians to answer questions about statistics. To come and ask "this person who is intended to work with data doesn't know how to interpret a basic model" will undoubtedly receive answers claiming the company hiring said individual(s) "are headed for disaster". That answer accords with our own beliefs that "data work = statistics", and I don't think that is necessarily the case at every org.

In my own experience, data science will mean 3 different things to 3 different companies, and that may mean that ability to do inference -- even basic inference -- is not the top skill to look for.

When I was interviewing, I found data science roles were largely placed into 3 broad categories:

  • Science Focused: These roles largely prioritized experimentation, hypothesis generation, and deep understanding of statistics and the scientific method

  • Engineering Focused: These roles largely prioritized modelling for accuracy instead of inference, deployment, and engineering skills.

  • Mixed Bag: These roles had a mix of each of the former to varying degrees.

In my opinion, it is silly to expect an engineering focused position to also have deep knowledge in statistics, just as I think it is silly to have a stats focused position have deep knowledge of engineering. To highlight this fact, I will gladly provide anecdotes of me failing interviews because the interviewer failed to appreciate statistical nuance that we here on cross validated nearly take as law.

Now, clearly the position you are hiring for is a science focused role, but your question was about mid level data science roles more generally. My answer is then that because data science still suffers from underdetermination, it may or may not be relevant to know linear regression beyond its predictive ability. It will depend on what kind of data scientist you are interviewing, their past experience, and their future plans to delineate their expertise.

If you find your job description is very clear that you need inference skills and not just machine learning skills, I think that might require a conversation with your recruiting team.

$\endgroup$
12
  • 24
    $\begingroup$ I disagree. Understanding a linear regression is not "deep knowledge in statistics". I would agree if the question was about working with GAMLSS or similar extensions, but I would absolutely say that there are foundational skills that every data scientist needs to have, and linear regression is absolutely one of them. Not knowing this is equivalent IMO to not knowing any of R, Python, Matlab or Julia. YMMV. $\endgroup$ Commented Mar 12 at 6:56
  • 5
    $\begingroup$ @StephanKolassa Your argument assumes that data scientists should be and should want to be scientists. My point is that a) despite the title, in practice data science seems to span the spectrum of research scientist to software engineer, b) software engineers may not need linear regression so they don't learn it, and c) if the applicants intend to remain and specialize at that end of the spectrum, then inference may not be as relevant to them just as some engineering concepts are not relevant to statisticians. $\endgroup$ Commented Mar 12 at 7:05
  • 2
    $\begingroup$ Not only this, but as jpbowman alludes to, applicants can enter data science from CS where the motivation for regression is not inferential. I would prefer for all data scientists to be scientists, but that just doesn't seem to be what is happening in reality. $\endgroup$ Commented Mar 12 at 7:09
  • 11
    $\begingroup$ I absolutely agree that a data engineer or ML ops engineer may not need to know about regression, and that there is no commonly accepted distinction between the different roles. But in the present case, the OP says they are looking for someone who will be "doing analytics whether an inference problem or predictive modeling". I do not understand how anyone can be expected to do predictive modeling without being expected to know pretty much the simplest tool for this task. $\endgroup$ Commented Mar 12 at 7:14
  • 3
    $\begingroup$ the key distinction here is whether your job is "statistics" or "machine learning". Yes, linear regression is useful for some "machine learning" problems but even then ML people very rarely think in terms of things like covariates and regression tables. They just use the model and they get a loss at the end. They care more about case specific metrics than general statistical measurements. If instead you want to actually have an objective description of how predictive a model is and you will be mostly working with simple non-ML based models, you are doing statistics. $\endgroup$ Commented Mar 13 at 2:01
10
$\begingroup$

Personal anecdote, when I interviewed for my current job as a data scientist I was asked how to compute a linear regression with pen and paper. I was able to do that and got the job. I never needed to do a linear regression with pen and paper on the job afterwards but I still believe it is a very good question to ask in an interview.

Namely it allows you to distinguish between the people who only know how to tell some stats software to compute a linear regression and those who understand what the software is actually doing when you ask it to do that. So ask yourself, do you need someone to only punch the numbers into a computer and tell you what the computer spit out or do you need someone who can check whether the computer actually did what they wanted it to do?

$\endgroup$
9
$\begingroup$

A different point of view, does understanding regression make you a good data engineer?

It is much more important to be a good programmer and for example be able to program in python and SQL and have experience with some fancy platform like hadoop, google cloud and visualization software like tableau or shiny.

Or at least, that is often in the descriptions. (This is not based on experience as a recruiter, but based on being someone that understands regression and occasionally looking out for vacancies in the field of data science. They are often very detailed, relating to specific practical skills).

Although regression is very basic and it would raise some eyebrows if knowledge about it is lacking, I believe that most recruiters are looking for simple muscle rather than brains.

Not being able to perform a simple regression analysis is a bit disappointing, yet is it essential for the data job? If you are asking this question then probably you already got the answer.

$\endgroup$
3
  • 4
    $\begingroup$ Upvoted (+1). I think the statement "they are headed for disaster" in one answer is clearly an exaggeration. I personally think engineering skills are way more important than very basic stat 101 knowledge to make you a good DS (here I interpret DS is different from data analyst or consultant). Plus, comparing with solid engineering skillset, being familiar with stat 101 knowledge is much less demanding. $\endgroup$
    – Zhanxiong
    Commented Mar 12 at 15:35
  • $\begingroup$ I agree that understanding linear regression isn't required for a good data engineer, but I would argue it's required for a good data scientist $\endgroup$
    – svavil
    Commented Mar 14 at 19:51
  • $\begingroup$ @svavil "The hire will be writing SQLs, working with stakeholders to define business problems and doing analytics whether an inference problem or predictive modeling." That's the description of an engineer. It is just being labelled 'scientist' because it is a popular term. $\endgroup$ Commented Mar 14 at 20:13
6
$\begingroup$

“Data scientist” is not a term with a clear definition. It’s also a sexy term, a job many people want or think they want, for some combination of (perceived) prestige and money. Thus, you get attention by calling a position “data scientist” instead of an analyst or an engineer. That might not really turn out to be the case, but plenty of companies sure seem to operate that way.

Depending on the exact skills required for the job, linear regression might be irrelevant. A “developer” role called “data scientist” might have minimal need for statistics skills. However, if the job involves regression analysis, not being able to answer basic questions is problematic, and regression fundamentals should not be considered beneath a “mid-level” or even “senior” hire, particularly if those skills matter to the work being performed.

$\endgroup$
1
  • 5
    $\begingroup$ As professionals in the field (or a related field), I think we have some leverage to stop propagating the mistake of calling every data-adjacent software job "data science". $\endgroup$ Commented Mar 13 at 1:41
3
$\begingroup$

Linear regression is for sure a basic concept in data science that forms the foundation for more advanced techniques. Understanding its principles is crucial for any data scientist, especially at a mid-level position where proficiency in statistical methods is

To me, the main point to take into account when addressing such questions is the complexity of data science solutions across industries and use-cases. It is crucial to tailor solutions to specific environments, considering factors like computational resources and above all, identify correctly end-users' profile. Therefore, it seems equally important to me to assess a candidate's ability to adapt and apply statistical methods effectively in diverse scenarios.

To further elaborate on that point, I would suggest to combine questions on linear regression with inquiries about more complex architectures like neural networks. This approach allows interviewers to gauge not only the candidate's understanding of basic statistical concepts but also their inclination towards more sophisticated solutions, and in which context do they consider using more complex models.

$\endgroup$
0
$\begingroup$

The question is appropriate as a basic skill to expect data science practitioners to have, BUT you might consider revisiting the interview style and preparation information you share so that candidates can be appropriately prepared.

By way of comparison, Software Engineering interviews are often focused on algorithmic puzzle solving but this is a skill that is very rarely used by practicing engineers, and many senior engineers find themselves in interviews with very rusty basics, struggling to answer what appear to be simple questions.

I believe there is a trend away from this style of interview these days because it's unrepresentative of what engineers actually have to do. Also programming with a hiring manager breathing down your neck is not a useful engineering skill to test for.

A few recommendations you might consider:

  1. Adopt more of a "work sample test" style where candidates are given a meaningful, real-world, and business-applicable problem to solve.
  2. Let candidates solve the challenge on their own terms such eg using their own tools.
  3. Consider being explicit and focused in the preparatory information you share, so that candidates know to revise and practice key skills.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.