3
$\begingroup$

So I am using a linear regression with time as a trend variable (specifically, I am taking the # of months since user's first activity as the linear feature of time, and also including the log, sqrt, squared, etc., transformations of that for non-linear time features) and dummy variables for 11/12 months of the year to account for seasonality. I will use forward selection of some sort to pick time features.

My time series is a bit ridiculous, so I find that trend often changes as time goes on. Check out the attached image for one that I am looking atA time series that shows a different trend with most recent observations

As such, does anyone know a way I could add an inverse of exponential decay of some sort to the weights in the regression? As in, let's say I have 60 rows of data, from the very first month of user activity to the most recent month. I want to train the regression on all of these, but I want the regression to weight the most recent observations more. There doesn't seem to be a good weighted regression package in Python besides Huber regression, but I cannot apply the weighting I want in that.

I do not think the fact that user_age_in_months is greater at more recent periods than in the first few periods makes a significant difference, especially since I am min-max scaling, and this is different from an exponentially weighted moving average.

This is the closest answer I've seen to this question, but it doesn't really give an example and this isn't as applicable to Python: Assigning more weight to more recent observations in regression

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ Some quick ideas: weighted regression is equivalent to OLS data scaled by the square root of your weights. You could also model your trend as an AR(1) $b_t = a_0 + a_1 b_{t-1} + \epsilon_t$ and work out what your estimate of $b_t$ would be based upon past data. On that approach some things to search for might be bayesian filter, Kalman filter, hidden markov model, particle filter. $\endgroup$
    – Matthew Gunn
    Apr 18 '17 at 19:17
1
$\begingroup$

Many ML packages give you an option to specify sample weights out of the box. One of the answers to the CV question that you cited gives an example of how this can be done in R. In Python ecosystem, scikit-learn provides an analogical parameter (sample_weight) that you can pass to many different regression models (linear, trees, SVMs, see e.g. here) while fitting.

You can use an arbitrary function to calculate the weights for each sample (in this case, that could be an exponential function of time), and pass it as a Numpy array.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.