2
$\begingroup$

I am working with library(nycflights13)dataset in R and I'm trying to provide an answer to this specific question:

If I am leaving before noon, what are my top two airline options at each airport (JFK, LGA, EWR) that will have the least amount of delay time?

At first I created a summarised table that displayed each airport's top two airline departure times (smallest departure delay) on average. 

flights %>%
  filter(sched_dep_time < 1200, origin %in% c("JFK", "LGA", "EWR")) %>%
  aggregate(dep_delay ~ carrier + origin, ., mean) %>% 
  group_by(origin) %>%
  top_n(n=-2, wt=dep_delay)

But then I thought, means aren't really a good summary stat I should base my decision off of. So I started plotting my data

scatterplot

I can see there are quite a few outliers but my thought is to keep them in the dataset as they provide valuable information on how badly a departure can be delayed at times. After getting a glimpse of the entire dataset, I wanted to look closer at departure times that are negative (meaning departed early) or around zero.

enter image description here

These plots are valuable but don't really make it obvious which airlines and airport would be the best for me to take given all the information I have. My next thought is to estimate the CDF for each airlines departure delay and then compute $P(X \leq 0)$. At that point I can then select the airlines that have the highest probability of having a departure delay of at most zero.

My question is two parts:

  1. Is this sound statistical reasoning? Is there a test I can perform that would be better? I Really would just like some guidance on thinking this problem through.

  2. If this is sound statistical thinking, are there any resources you can direct me towards that would teach me how to implement it in R?

Improve this question
$\endgroup$
3
  • $\begingroup$ Time intervals can be naturally modeled using the Poisson distribution where the smallest mean would hint the smallest delay airport. On top of that, you want to do ranking since you wish to select top 2 (which Poisson distributions have the smallest means). My 2 cents. $\endgroup$
    – Vladislavs Dovgalecs
    Commented Dec 20, 2017 at 19:09
  • $\begingroup$ @VladislavsDovgalecs Thanks for the comment. Do you have any suggestions on resources I could look to do this and/or do it in R? $\endgroup$
    – dylanjm
    Commented Dec 20, 2017 at 19:26
  • $\begingroup$ I see a hierarchical Bayes model where each airline delay time is modeled using a Poisson distribution. The respective airport is modeled by a top level parameter \mu (Gaussian? Gamma?) that controls shrinkages for each airline and enables better parameter estimates. You can go a level higher for multiple airports. You can model that in Stan (see the 8 schools example). The Stan community on discourse is very helpful. John Kruschke's "puppies" book "Doing Bayesian Data Analysis" goes into great lengths explaining this type of models and offers many examples (R and JAGS). $\endgroup$
    – Vladislavs Dovgalecs
    Commented Dec 20, 2017 at 19:32

1 Answer 1

Reset to default
1
$\begingroup$

Data visualization is always a great way to start. Here is how I would approach it.

Since you are only interested in before noon - limit all data from 6am to noon.

Run three different ANOVAs (delay time as dependent variable, aircraft carrier as independent variable) for each airport. Or you could run one model including airport as another main effect along with the interactions.

Examine residuals for severe violations of model assumptions.

Investigate the output to see which airlines are significantly shorter in flight delay. Will need to do some multiplicity adjustments.

Googling ANOVA and R will give you plenty of material.

There are other questions to consider such as do I want to almost guarantee a delay time of less than 30 minutes compared to no delay time at all? This, I believe, requires more advanced models. To answer this I would use a Bayesian approach in JAGS and compute theses posterior probabilities. However, for your sake, I think you can just focus on least square means from the ANOVA models.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.