We're tracking users' hourly usage on our cloud service and have a risk model that uses aggregated usage data plus other signals to identify potential fraudsters. Basically, it's an anomaly detection model. The current approach uses simple filters, like checking if today's usage is suspiciously high on yesterday (around 2 sd from its past history) combined with a high-risk model score, before sending the users to our labeling team for review. However, accuracy is only about 50-60%. I suggested building a new model using deep neural networks and the past 60 days of data to uncover hidden usage patterns. My manager dismissed it, saying our existing usage model made it unnecessary. But then, a colleague proposed using scores from 3 current production models (include the usage one) as features for a new model, which showed some improvement, and suddenly my manager loved the idea. Honestly, I'm confused. Am I missing something, or is there a deeper issue with my manager?
3
-
2$\begingroup$ Could you help us understand why you conceive of this as a statistical question rather than as the employee relationship and psychological issues in which it is framed? $\endgroup$– whuber ♦Commented Mar 1 at 20:43
-
$\begingroup$ @whuber Thanks for the reply. I want to know if my approach is acceptable. The usage model we are integrating does not perform well. So, I want to dive deeper and develop a new model with flattened usage information. Is this a good approach to improve performance? Sorry, maybe I should change the title. $\endgroup$– Jiayu ZhangCommented Mar 1 at 20:56
-
1$\begingroup$ I'm afraid one would need much more info on which to base an answer. $\endgroup$– rolando2Commented Mar 1 at 21:54
Add a comment
|