1
$\begingroup$

I have the following preset;

A data frame on the format as follows:


df <- data.frame(prevArrivalTime = c(1676193057, 1676193112, 1676193180, 1676193277, 1676193358, 1676193469, 1676193581, 1676102575, 1676102613),
                 hours = c(1, 2, 3, 1, 2, 3, 1, 2, 3))

dummy <- model.matrix(~factor(hours) - 1, data = df)
df <- cbind(df[, 1], dummy)
colnames(df) = c("prevArrivalTime", "hour_1", "hour_2", "hour_3")
df <- cbind(df, actualTravelTime = c(55, 68, 97, 81, 111, 112, 126, 38, 73))

The prevArrivalTime is the recording for when a bus arrived at the previous stop, converted to a unix timestamp format, which measures the seconds which have passed since 1970-01-01. For instance, 1676193057 was "Sunday, February 12, 2023 9:10:57 AM". Dummy variable hour_1, hour_2 & hour_3 are dummy encoded hour recordings for the corresponding hour. The actualTravelTime is the variable that I'm trying to predict, namely how long it took for the bus to travel.

My data stretches over multiple days and weeks, meaning the variable prevArrivalTime will be strictly increasing as time passed by. I want to estimate $\hat{y}$ = estimatedTravelTime, by using linear regression.

The way I envision how I do this is through the formula;

$\hat{y} = b_0 * prevArrivalTime + b_1 * dummy(x) + \epsilon- prevArrivalTime$

Where I'd subtract the prevArrivalTime, which would give me a measurement of how long it took to travel from the previous stop to the next.

Otherwise, I think I end up with something where prevArrivalTime would be constantly increasing;

  • prevArrivalTime_1 = 1676193057
  • prevArrivalTime_2 = 1676193112
  • prevArrivalTime_500 = 1677829121

While actualTravelTime would be fluctuating;

  • actualTravelTime_1 = 55
  • actualTravelTime_2 = 68
  • actualTravelTime_500 = 58

My original question was how to implement the formula above in R, but as the comments suggested, it wasn't possible to interpret what I wanted to achieve. I hope this restructure explains what I wish to achieve in a better way.

$\endgroup$
22
  • 3
    $\begingroup$ Why not subtract first and then model? $\endgroup$ Mar 2 at 13:01
  • 1
    $\begingroup$ Subtract how you mean? $\endgroup$
    – OLGJ
    Mar 2 at 13:04
  • 1
    $\begingroup$ $y2=y_t-y_{t-1}$ and then model $y2$. $\endgroup$ Mar 2 at 13:06
  • 1
    $\begingroup$ @OLGJ your explanation is helpful in understanding your problem a bit better, but it is still not clear what you mean by 'subtracting' some variable from the model. $\endgroup$ Mar 2 at 15:30
  • 1
    $\begingroup$ As its written right now, the question appears to be an XY Problem. Your research question wants to know something about bus arrivals and you have some data about the buses (X), and you've decided to answer it using a regression that subtracts some variable (Y), but how these two ideas X & Y are related is very unclear. I suggest writing, in plain language, what you know, what you want to know, and where you are stuck. Code & notation can come later -- right now, they seem to be distracting & creating unclarity. $\endgroup$
    – Sycorax
    Mar 2 at 15:32

2 Answers 2

2
$\begingroup$

According to your comments it seems like you want to model some variable that has a periodic time dependency. When you model this as a linear function then subtracting the time is gonna make it still a linear function.

The model below

$$y = a + bt + \epsilon -t$$

is just equivalent to linear models

$$y = a + b^\prime t + \epsilon \quad \text{with $b^\prime = b-1$} $$ or

$$y^\prime = a + bt + \epsilon \quad \text{with $y^\prime = y+t$} $$


  • What you probably want to do instead is to include some periodic variables like Fourier terms.

  • Or alternatively you use a time variable modulo the time of the day. E.g. time stamps like t = 1.7 days, t = 2.7 days, t = 3.7 days will all become t=0.7.

    The image below shows an example with a time variable on the x-axis that runs from t=0 to t=7. On the left we have the variable $y$ plotted as a function of $t$. On the right we have the variable $y$ plotted as a function of $t$ modulo $1$ (the colour coding of the points is kept the same).

    example of plots

$\endgroup$
2
  • $\begingroup$ The first part actually addressed my concerns. Thank you for that and your patience. I will expand into different ways to encode the hour of the recordings, but I wanted to create this as a simple first model. Thank you. $\endgroup$
    – OLGJ
    Mar 3 at 14:32
  • $\begingroup$ However, If I may ask; how would I practically rewrite my linear model in this case? Even if it doesn't matter. $\endgroup$
    – OLGJ
    Mar 3 at 15:01
0
$\begingroup$

Using offset (hp is the lagged variable)

> lm(mpg~1,data=mtcars,offset=hp)

Coefficients:
(Intercept)  
     -126.6  

using I()

> lm(I(mpg-hp)~1,data=mtcars)

Coefficients:
(Intercept)  
     -126.6

subtracting first the modelling

> mtcars$mpg_hp=mtcars$mpg-mtcars$hp
> lm(mpg_hp~1,data=mtcars)

Coefficients:
(Intercept)  
     -126.6 
$\endgroup$
3
  • $\begingroup$ So offset takes the mean of hp in this case and subtracts it, is that correct? mean(mtcars$mpg)-mean(mtcars$hp) $\endgroup$
    – OLGJ
    Mar 2 at 14:11
  • $\begingroup$ @OLGJ Strictly speaking it's mean(mtcars$mpg-mtcars$hp). $\endgroup$ Mar 3 at 6:52
  • $\begingroup$ I see. I have now edited my question and tried be clearer about what I want to achieve @user2974951 $\endgroup$
    – OLGJ
    Mar 3 at 7:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.