0
$\begingroup$

I am doing a sports analytics (Sabermetrics) project and wish to investigate the effect of jetlag on MLB players. Currently I am performing analysis on the individual player level rather than the team level. I have a set of players and their stats in non-jetlag games, and I also have their stats with 1 hour jetlag, 2 hours, etc. For each player I performed a t-test per statistic to determine if the average (for any chosen) metric significantly changed with jetlag (compared to no jetlag).

I would like to know the most correct way to present my results. Here is what I've tried:

  • % of players who statistically significantly improved or got worse (p < 0.05 in either direction)
  • % of players who improved or got worse (regardless of p-values)
  • An average of all the averages with and without jetlag, showing that the stats get worse overall when players are jetlagged
  • Heatmaps of players with significant improvement or degredation using k-means clustering
Improve this question
$\endgroup$
  • 1
    $\begingroup$ What's the rationale for doing individual comparisons? $\endgroup$ – AdamO Oct 13 '17 at 19:21
  • $\begingroup$ Why is this question taggeg "clustering"? $\endgroup$ – ttnphns Oct 14 '17 at 10:55
  • $\begingroup$ @AdamO I thought this was the best way to look at individual player performance as my impression of a single model was it can't 'faily' evaluate many different players each with different sample sizes for each metric. $\endgroup$ – Nihavent Oct 14 '17 at 14:21
  • $\begingroup$ @ttnphns As I stated I have also presented this data using k-means clustering $\endgroup$ – Nihavent Oct 14 '17 at 14:21
  • $\begingroup$ I doubt that k-means is meaningful here. Beware of multiple testing - you likely will see many "false significant" results. I'd focus on boxplots with and without jetpack, because these will make it easy to see if a difference in mean is small compared to the usual variation. $\endgroup$ – Has QUIT--Anony-Mousse Oct 16 '17 at 6:50
1
$\begingroup$

As @AdamO says, running lots of individual comparisons is not a good idea. You have a huge amount of comparisons that you have to correct for.

It would be better to include the amount of jetlag as a numerical variable in a model with performance as the dependent variable, possibly considering a spline transformation to include nonlinearity (see Harrell's Regression Modeling Strategies).

Once you have done this, the best plot would be simply to plot your raw data, performance against jetlag with one dot per observation. Potentially plot different dot symbols for the different players, or visually encode any other relevant information. Then overlay the model fit as a line.

Improve this answer
$\endgroup$
  • $\begingroup$ I like your idea. I do have further questions though. In this test my 'n' will become the number of players in the sample instead of the number of attempts each player has to achieve a metric (for example 'at bats'). How will this model handle greatly varying sample sizes of each individual player (ie. some players may have only been 'at bat' 5 times under the 3 hours jetlag condition and some may have been 'at bat' 1000+ times)? $\endgroup$ – Nihavent Oct 14 '17 at 14:15
  • $\begingroup$ Good point. I'd suggest using a repeated-measures model, which is typically done using a mixed-model. You can then model systematic differences between players (with a random intercept per player), or different responses to jetlag (with a random interaction between player and transformed jetlag), possibly even development of a player's ability over time (by using a AR or continuous AR covariance structure on residuals). $\endgroup$ – Stephan Kolassa Oct 14 '17 at 14:29
0
$\begingroup$

Are the data normaldistributet? If not, you could also show the median and you could do (for robust) the Wilcoxon test as a second t-test.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.