1
$\begingroup$

I have the following preset;

A data frame on the format as follows:


df <- data.frame(prevArrivalTime = c(1676193057, 1676193112, 1676193180, 1676193277, 1676193358, 1676193469, 1676193581, 1676102575, 1676102613),
                 hours = c(1, 2, 3, 1, 2, 3, 1, 2, 3))

dummy <- model.matrix(~factor(hours) - 1, data = df)
df <- cbind(df[, 1], dummy)
colnames(df) = c("prevArrivalTime", "hour_1", "hour_2", "hour_3")
df <- cbind(df, actualTravelTime = c(55, 68, 97, 81, 111, 112, 126, 38, 73))

The prevArrivalTime is the recording for when a bus arrived at the previous stop, converted to a unix timestamp format, which measures the seconds which have passed since 1970-01-01. For instance, 1676193057 was "Sunday, February 12, 2023 9:10:57 AM". Dummy variable hour_1, hour_2 & hour_3 are dummy encoded hour recordings for the corresponding hour. The actualTravelTime is the variable that I'm trying to predict, namely how long it took for the bus to travel.

My data stretches over multiple days and weeks, meaning the variable prevArrivalTime will be strictly increasing as time passed by. I want to estimate $\hat{y}$ = estimatedTravelTime, by using linear regression.

The way I envision how I do this is through the formula;

$\hat{y} = b_0 * prevArrivalTime + b_1 * dummy(x) + \epsilon- prevArrivalTime$

Where I'd subtract the prevArrivalTime, which would give me a measurement of how long it took to travel from the previous stop to the next.

Otherwise, I think I end up with something where prevArrivalTime would be constantly increasing;

  • prevArrivalTime_1 = 1676193057
  • prevArrivalTime_2 = 1676193112
  • prevArrivalTime_500 = 1677829121

While actualTravelTime would be fluctuating;

  • actualTravelTime_1 = 55
  • actualTravelTime_2 = 68
  • actualTravelTime_500 = 58

My original question was how to implement the formula above in R, but as the comments suggested, it wasn't possible to interpret what I wanted to achieve. I hope this restructure explains what I wish to achieve in a better way.

$\endgroup$
22
  • 3
    $\begingroup$ Why not subtract first and then model? $\endgroup$ Mar 2, 2023 at 13:01
  • 1
    $\begingroup$ Subtract how you mean? $\endgroup$
    – OLGJ
    Mar 2, 2023 at 13:04
  • 1
    $\begingroup$ $y2=y_t-y_{t-1}$ and then model $y2$. $\endgroup$ Mar 2, 2023 at 13:06
  • 1
    $\begingroup$ @OLGJ your explanation is helpful in understanding your problem a bit better, but it is still not clear what you mean by 'subtracting' some variable from the model. $\endgroup$ Mar 2, 2023 at 15:30
  • 1
    $\begingroup$ As its written right now, the question appears to be an XY Problem. Your research question wants to know something about bus arrivals and you have some data about the buses (X), and you've decided to answer it using a regression that subtracts some variable (Y), but how these two ideas X & Y are related is very unclear. I suggest writing, in plain language, what you know, what you want to know, and where you are stuck. Code & notation can come later -- right now, they seem to be distracting & creating unclarity. $\endgroup$
    – Sycorax
    Mar 2, 2023 at 15:32

2 Answers 2

2
$\begingroup$

According to your comments it seems like you want to model some variable that has a periodic time dependency. When you model this as a linear function then subtracting the time is gonna make it still a linear function.

The model below

$$y = a + bt + \epsilon -t$$

is just equivalent to linear models

$$y = a + b^\prime t + \epsilon \quad \text{with $b^\prime = b-1$} $$ or

$$y^\prime = a + bt + \epsilon \quad \text{with $y^\prime = y+t$} $$


  • What you probably want to do instead is to include some periodic variables like Fourier terms.

  • Or alternatively you use a time variable modulo the time of the day. E.g. time stamps like t = 1.7 days, t = 2.7 days, t = 3.7 days will all become t=0.7.

    The image below shows an example with a time variable on the x-axis that runs from t=0 to t=7. On the left we have the variable $y$ plotted as a function of $t$. On the right we have the variable $y$ plotted as a function of $t$ modulo $1$ (the colour coding of the points is kept the same).

    example of plots

$\endgroup$
2
  • $\begingroup$ The first part actually addressed my concerns. Thank you for that and your patience. I will expand into different ways to encode the hour of the recordings, but I wanted to create this as a simple first model. Thank you. $\endgroup$
    – OLGJ
    Mar 3, 2023 at 14:32
  • $\begingroup$ However, If I may ask; how would I practically rewrite my linear model in this case? Even if it doesn't matter. $\endgroup$
    – OLGJ
    Mar 3, 2023 at 15:01
0
$\begingroup$

Using offset (hp is the lagged variable)

> lm(mpg~1,data=mtcars,offset=hp)

Coefficients:
(Intercept)  
     -126.6  

using I()

> lm(I(mpg-hp)~1,data=mtcars)

Coefficients:
(Intercept)  
     -126.6

subtracting first the modelling

> mtcars$mpg_hp=mtcars$mpg-mtcars$hp
> lm(mpg_hp~1,data=mtcars)

Coefficients:
(Intercept)  
     -126.6 
$\endgroup$
3
  • $\begingroup$ So offset takes the mean of hp in this case and subtracts it, is that correct? mean(mtcars$mpg)-mean(mtcars$hp) $\endgroup$
    – OLGJ
    Mar 2, 2023 at 14:11
  • $\begingroup$ @OLGJ Strictly speaking it's mean(mtcars$mpg-mtcars$hp). $\endgroup$ Mar 3, 2023 at 6:52
  • $\begingroup$ I see. I have now edited my question and tried be clearer about what I want to achieve @user2974951 $\endgroup$
    – OLGJ
    Mar 3, 2023 at 7:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.