6
$\begingroup$

Modelling flight delays with negative values

I am working on a model to predict whether a flight will be delayed. The data consists of some explanatory variables for flights from a specific airport. I initially thought modelling this as count data would be a good idea, but as pointed out in the comments that is misleading.

The response variable is the number of minutes deviations from departure initial departure time. I have some explanatory variables about the flights to work with, i.e. date, distance traveled, etc.. I don't have any weather variables though.

The following is the histogram of the data. I have a positively skewed distribution and I am thinking what kind of a distribution would be a good candidate to model this.

Histogram of Data

I am now asking, what kind of model is appropriate for this kind of data? The main goal is to do predictions.

One idea I had was to train a classifier first to determine whether the flight will be delayed or not and then predict how late it would become with a regressions model, but I would also like to predict how early it went if that is the case.

I think that I will use logistic regression to predict whether a flight will be late or early and then construct a prediction model for these two classes. What ideas do you have for models that would be good to predict deviation from set take-off time conditioned on that it will be a delayed take-off or an early take-off?

Edited to remove my confusions about count data.

$\endgroup$
8
  • $\begingroup$ You could rescale your "delay" variable and add the maximum observed early departure. $\endgroup$
    – Karsten W.
    Feb 28, 2015 at 20:04
  • 2
    $\begingroup$ "Minutes" is a duration, not a count. Counts represent discrete things which cannot be subdivided, whereas a duration can be arbitrarily subdivided, even if it happens to be rounded to the nearest integer. $\endgroup$
    – whuber
    Feb 28, 2015 at 20:23
  • $\begingroup$ I am aware of that. I think it is still reasonable to model this as count data, since we can think about this as counting the minutes of failure until we have a success, i.e. take-off. The distribution of the data is also overdispersed which lead me to consider the negative binomial. If I have device that checks every minute until the flight has taken of, it is essentially counting the minutes as a discrete variable, do you see anything wrong with that? I am sure that I could model this as a continuous variable, but I am interested to see if I can the negative binomial or something similar. $\endgroup$
    – Gumeo
    Feb 28, 2015 at 20:33
  • 1
    $\begingroup$ You could, as a purely practical matter, attempt to model the data with discrete distributions such as a Poisson or negative binomial. But you still don't have "count data" (and there is no such thing as "count data with negative values"). Calling your data "counts" only risks confusing and misdirecting people who otherwise might be able to contribute helpful answers; and it certainly limits your analytical options, which ought to be expanded to include procedures appropriate for durations. $\endgroup$
    – whuber
    Feb 28, 2015 at 21:39
  • $\begingroup$ You could try with skew-normal (or skew-t). I am not sure if that can be used with as clear skew as this, but is worth a try. Can you post (a link to) the data? $\endgroup$ Jan 10, 2019 at 22:33

1 Answer 1

2
$\begingroup$

First, I agree that this is not count data.

If there are many flights that are canceled, then you might think of it as time to event data and look into survival analysis methods. This might depend on where and when you are: More flights are cancelled from Chicago in winter than from Phoenix in May.

Other than that, you might try quantile regression; I suggest this for two reasons: First, you might be particularly interested in long delays. If you are interested in this from a passenger POV, then a short delay in departure might not matter at all - these are often made up during the flight, and I think most passengers are more concerned with arrival time than departure time. But if you are the airport manager, then even a short delay might be a problem with scheduling runways and so on. Quantile regression lets you model the quantiles. Second, quantile regression makes no assumptions about the distribution of the residuals.

For the early departures, I think you have to figure out whether an early departure is better or worse or equivalent to an on-time departure.

$\endgroup$
1
  • 2
    $\begingroup$ Thanks for the answer! I agree, this should be treated as a continuous response in some sense. One thing to add is that when I did this ~3 years ago I scraped some weather data to try to help with the longer delays, that helped a lot. Sometimes it is more finding the right covariates rather than the modeling. $\endgroup$
    – Gumeo
    Jan 11, 2019 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.