Data scientist interview question - what statistical analysis should be used? This question was asked in an interview
At a company, say ABC, the hiring department collects data on interviews conducted by its employees. The data we have contains 4 variables, timestamp, employee_id, candidate_id and a candidate_score that ranges from 1 - 4 where each record represents an interview. We do not know if the candidate was hired or not.
Consider that each employee interviews 30 candidates and each candidate is interviewed by 6 employees.
The question: ABC wants to figure out what is wrong with the hiring process using this data. What kind of statistical analysis would you use?
Any ideas on how to approach this problem would be helpful. 
 A: As Alex R. pointed out, the question may be vacuous if you do not explain what defines anomalies. I will assume that the question to answer here is to explain the target to track its potential deviations, based on your variables, and eventually identify root causes. For the sake of simplicity, I will refer to candidate_score as target, and other parameters as variables. Since the question is designed for interviewing purposes, I assume that by no mean, answers from candidates should be exhaustive.

Analyzing and visualizing variations per variable

Since you have very few variables, first thing to do would be to analyze and visualize potential score variations based on one variable.


*

*timestamp: Calculate average score per period (week day, month, year), depending on the exhaustivity of your dataset. Start simple, look if you see any evolution across time (plotting time-series or histograms), or decompose the series if you are given a sufficient time-window into a trend/seasonal/remainder pattern. This would highlight seasonal phenomenon, e.g. potential keener notes on Friday than on Monday, or during holidays period.

*employee_id: Given the number of candidates interviewed per employee, it may be relevant to calculate the average score given per employee and examine the associate distribution through a boxplot. You will be able to track deviations in the way employees score candidates. A tight boxplot implies that they grade candidates in a uniform standard way (presumably according to rules clearly defined for the recruiting process), while a dispersed boxplot means that scores are subject to consistent variations depending on the employee. Additionally, if you have very few employees, you can directly calculate the average awarded score per employee and identify which ones are keener/harsher.

*candidate_id: This one would be trickier to evaluate, as candidates are supposed to be independent from the process. However, you can calculate the standard deviation of their score to see if they are perceived by employees in a uniform way (or not) / or that the grading rules make sense in the process. Again, you can boxplot, and examine the associated distribution, such as for average score per employee_id.



Examining interdependency

You may be interested in examining dependencies between your variables, and their influence over the target. You can combine your variables and perform the following analysis.


*

*Transform your timestamp into a categorial variable (e.g. month or weekday) and calculate the average score per employee per period. This would be close to performing a correspondance analysis. However, the pertinence of such analysis would be limited as you have very few grades per employee : 30 grades divided by 5 or 12 would return statistically insignificant results. 

*Dummify your variables and target, and look for correlation. This may highlight the fact that some employees are dedicated to specific period of time, but you would not gain much from what has been performed before.


Obviously, the analysis and reliability would clearly depend on the number of employees/candidates that were involved in the process, as well as the period covered by your dataset. Nonetheless, it can be a starting point, especially given the fact that in an interview, you will have limited time to develop your ideas.
