Linear regression - iterative approach

Question

I have a single output variable $y$ and a number of inputs $x_1$, $x_2$, etc. These are time series. Each $x_i$ explains the changes in $y$ in specific circumstances, and the goal is to have a linear model that looks like $y=b_1x_1+b_2x_2+...$ The point is that each $x_i$ must be sampled on different criteria specific to its qualities, and its relationship to $y$ to be established only at those situations when that particular $x_i$ is the lead influence on $y$.

My approach is to take one sampling method $s_1$, sample $y$ and $x1$ on it, specific to $x_1$. Do the regression $y$ on $x_1$. Then, take the error $y-b_1x_1$, sample the error on another sampler $s2$, and regress this on $x_2$.

What's the name of this method, is there a regression of this sort? Does it sound like the right solution to my problem?

Due to practical constraints, I want my model to include all $x$s at the same time, rather than switch between different models on the fly.

Answer 1

I'm posting as an answer since it's too long for a comment.

If I understand correctly, you have data coming from, let's say, $N$ multiple sensors $ x_i^{(t)} \in \mathbb{R}^{T \times N} $ where each sensor is a time series with $T$ data points. You also have a target output variable $y^{(t)} \in \mathbb{R}^T$.

You would like to perform a prediction using a linear model and you would like to adjust the subsequence sample size (or sliding window size) for each $x_i$ sensor.

One option is to take a larger fixed size sliding window over your time series, $ X^{(t)} \in \mathbb{R}^{W \times N}$ where $W$ is the window size.

Then, there are methods that perform regularization for all $x_i$ such as group lasso or group elastic net or ridge regression. The subsequence for a sensor is one group.

In effect, this adjusts the $ b_i^{(w)} $ for all $x_i^{(w)}$ giving a specific weight to each one of the subsequence points in your window, effectively adjusting the window size or sample size separately per $x_i$.

Answer 2

If I understand correctly, you wish to use only specific sample values of $X_i^{j}$ ($i$ for the time period and $j$ for the predictor index) to predict $Y_i$.

Say you have the sampler values: $S_i^j = 1$ if predictor $j$ is selected in time period $i$. Then you could simply run the following regression:

$$Y_i = \beta_1 S_i^1 X_i^1 + \beta_2 S_i^2 X_i^2 + ...$$

and this would only fit to the regressors on the sampled time periods.

p.s. From your explanation it didn't seem to me that there was anything inherently specific to time series...