0
$\begingroup$

I'm struggling with figuring how best to visually look at the trending the following.

I have the following data in rows

date, entity_type, row_count, duration_of_execution

I would like to visually look at how the row_count vs duration_of_execution are trending on a daily basis, entity_type wise. I expect the relation between row_count and duration_of_execution to be increasing. For example, if the row_count increases substantially, I expect an increase in duration_of_execution. However, would like to investigate if that is not the case.

What is the best way to represent this visually? Ideally, if there is an anomaly I would want to drill down to a particular day and entity. If I think more, I would like this chart to be visible for a larger audience to glance at it high-level and for me to investigate further if I observe an anomaly.

I'm open to any technology but importantly looking for ideas on what type of chart and formula to correlate the row_count vs duration_of_execution on a daily basis.

Improve this question
$\endgroup$
6
  • $\begingroup$ It sounds like you're describing two different things: one is a time-series graph that would be pretty messy ("...on a daily basis") and the other is a pretty straight-forward scatterplot with groupings. Since your question seems to be "Does duration increase as row count increases?", I would suggest you need the second and should drop the "on a daily basis" idea. You could still "drill down" on an anomaly, but the day is probably of more interest from an operations perspective than a statistical one and so wouldn't need to be in the plot. $\endgroup$
    – Todd Burus
    Jan 19 '20 at 8:05
  • $\begingroup$ Also, what language would you be making this visualization in? $\endgroup$
    – Todd Burus
    Jan 19 '20 at 8:05
  • $\begingroup$ @ToddBurus - Thanks. On the scatter plot suggestion, I assume there will be a new scatter plot to be generated everyday. It is fine and thinking more it does make sense. $\endgroup$
    – prabhu
    Jan 19 '20 at 8:40
  • $\begingroup$ For language, I was thinking about using python with matplotlib. Short term even an excel or google sheet would do. $\endgroup$
    – prabhu
    Jan 19 '20 at 8:41
  • $\begingroup$ do you have multiple entries per day or just one? Either way you could write a function that allows you to simply load in new data and generate daily. Python is a good choice, It would also be quick-and-easy to construct the visualization I had in mind in R using ggplot2. $\endgroup$
    – Todd Burus
    Jan 19 '20 at 8:58
1
$\begingroup$

I would make a scatterplot colored by grouping for entity_type. In Python you could use Seaborn with import seaborn as sns. The plot would then be executed by:

sns.pairplot(x_vars=['row_count'], y_vars=['duration_of_execution'], data=df_name, hue='entity_type', size=5)

You may need to adjust some of the calls based on how your dataframe is named in Python, but I think that should be clear enough.

If you see an anomaly you could then reference back to the dataframe and find the date. Including date on the scatterplot could be done as a label or something, but would probably add more clutter than necessary if you goal is clarity for a larger audience.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.