5
$\begingroup$

I am working on a simple linear regression model that predicts cost-per-truck based on distance between origin and destination. Let's assume there's a good linear relationship between Cost and Distance. I have historical data for ~250K shipments which corresponds to around ~2000 lanes (origin, destination pairs).

The goal of this model is to predict cost for new lanes (for which I have no historical data) and get the confidence/prediction intervals.

Some random dummy data -

# A tibble: 10 x 4
     Lane   Distance ShipmentNo  Cost
    <chr>      <dbl>      <int> <dbl>
 1 A to B        100          1  812.
 2 A to B        100          2 1055.
 3 A to B        100          3  749.
 4 A to B        100          4 1479.
 5 A to B        100          5 1099.
 6 C to D        500          6  754.
 7 C to D        500          7 1146.
 8 C to D        500          8 1221.
 9 C to D        500          9 1173.
10 C to D        500         10  908.

I am confused between 2 approaches -

  1. Build a model at the Shipment level i.e. every shipment is a data-point (250K points) on Cost vs. Distance scatter plot. Data to be divided into train and test data-sets i.e. test data represents unseen (new) shipments. In this case, the regression line represents mean shipment cost for a given distance. How would this translate into lane level prediction? Would it make sense to use the predicted mean shipment cost along with 95% confidence interval as a prediction for new lanes?

  2. Build a model at the Lane level (aggregated level) i.e. average lane rate (Total Cost / Total Shipments for each lane) is a data-point (now 2000 lanes/points) on Cost vs. Distance scatter plot. Aggregated data to be divided into train and test data-sets i.e. test data represents unseen (new) lanes. I suppose, in this case, the regression line represents the mean of average lane rate for a given distance. Would it make sense to use this mean of average lane rates along with 95% prediction interval as prediction for new lanes?

Not sure but option 2 seems right in terms of level of detail I need. However, I am concerned that by aggregating the data I may be losing information on (or even misrepresenting) the variation in cost of trucks for individual shipments. Also, in option 2 all data-points (lanes) seem to have same weight in spite of some lanes potentially having lot more shipments than others which doesn't seem right.

Based on other answers on this site, general opinions indicate that aggregation is a bad idea - however I am curious to know if it is justified in my case considering the goal is to predict lane rates and not individual shipment rate.

Looking for answers to my above questions and advise on best way to handle this problem. Thanks for the help!

$\endgroup$
  • $\begingroup$ You could consider a linear model with a random effect for Lane. $\endgroup$ – Yves Jan 24 at 7:07
  • $\begingroup$ Not wanting to make more work for you, but there is something to be learned from doing both models, and comparing the results. If they give similar answers, you have a relatively well-behaved problem space (for example, different lanes having similar variability). On the other hand, if the two models give you very different answers, that in itself tells you something more than either one alone. $\endgroup$ – rossdavidh Jan 24 at 15:55
10
$\begingroup$

I would suggest taking advantage of all the data by fitting a linear mixed effects model. This is a model that includes fixed effects and random effects. If you aggregate the data then you lose information, while if you fail to account for the probability that costs in one lane are more similar to costs in that lane than other lanes then you will obtain biased results.

To do this, you simply fit random intercepts for lane. Using the lme4 in R package as an example the would look like:

lmer(cost ~ distance + (1 | lane), data = mydata)

The lane variable is treated as random in the sense that it can be thought of as a variable for which you have more than just a few (5-10 is often mentioned as being the minimum, although there is no consensus on the matter: you have 2000 so this is not an issue at all), and each lane then has it's own intercept which varies randomly around the global intercept (which is equivalent to just "the intercept" in a model without random effects). We do this in order to control for the non-independence of observations/measures within each lane, otherwise biased results are very likely.

You can also allow the effect of distance to be different for each lane by adding random slopes:

lmer(cost ~ distance + (distance | lane), data = mydata)

And finally you can also allow distance to have a non-linear effect on cost by including nonlinear terms in the model equation, or using splines.

The above models will help you to understand the association between distance and cost for existing lanes, and it will also help you to understand the variation in costs by lane, and in the case of the random slopes model, to also see if the association between cost and distance differs between lanes. However, you say that you are interested in prediction for new lanes that have not been seen before. To get to grips with this problem, more information is needed about what the lanes actually are.

$\endgroup$
  • 1
    $\begingroup$ Good advice, but few sentences about what mixed effects models are could be useful, since OP does not seem to mentioning it, so we can guess they are not familiar with mixed effect models. $\endgroup$ – Tim Jan 24 at 8:40
  • $\begingroup$ @Tim thank you - I have edited my answer accordingly $\endgroup$ – Robert Long Jan 24 at 8:44
  • $\begingroup$ @RobertLong Thanks for the answer. I'll read up on mixed effects model. As for "more information is needed about what the lanes actually are" - the analysis is limited to mainland USA. If it helps, the number of distinct origins is limited to ~50 i.e. new lanes will likely appear due to new destinations. Also, some additional attributes could be developed for all lanes like East-to-West or State-to-State or Zip3-to-Zip3 etc. Does that help? $\endgroup$ – Shree Jan 24 at 15:53
  • $\begingroup$ @Shree thanks ! I am still not quite clear about what exactly is a lane. $\endgroup$ – Robert Long Jan 24 at 18:40
  • $\begingroup$ @RobertLong Lane is origin-destination pair. For example - NYC-to-Chicago, NYC-to-Boston, Cleveland-to-Dallas etc. $\endgroup$ – Shree Jan 24 at 18:46
1
$\begingroup$

First, I'd try to answer the questions. Are the cost/distance relationships between each lane statistically different? If yes, are the differences large enough to matter for our application / problem? If no, use the aggregated data. Otherwise, we need to do some simple thought experiments to see what using aggregated / non-aggregated data means.

Suppose 50% of lanes cost \$10/km and 50% \$2/km. The average would be \$6/km. But does this mean anything for making predictions? I'd say a \$6/km prediction for a previously unseen lane is wrong. If all lanes are similar, than it's either going to cost \$10/km or \$2/km. But the average of hundreds of new lanes will be close to \$6/km.

Using aggregated or non-aggregated data answers different questions.

Hope this helps a little.

$\endgroup$
  • $\begingroup$ Thanks for the answer. Could you elaborate on "Using aggregated or non-aggregated data answers different questions" i.e. what question each answers. As mentioned in the question I am looking to predict the mean cost of truck and corresponding confidence/prediction intervals for a new lane with known distance. $\endgroup$ – Shree Jan 24 at 18:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.