Linear regression - iterative approach

I have a single output variable $y$ and a number of inputs $x_1$, $x_2$, etc. These are time series. Each $x_i$ explains the changes in $y$ in specific circumstances, and the goal is to have a linear model that looks like $y=b_1x_1+b_2x_2+...$ The point is that each $x_i$ must be sampled on different criteria specific to its qualities, and its relationship to $y$ to be established only at those situations when that particular $x_i$ is the lead influence on $y$.

My approach is to take one sampling method $s_1$, sample $y$ and $x1$ on it, specific to $x_1$. Do the regression $y$ on $x_1$. Then, take the error $y-b_1x_1$, sample the error on another sampler $s2$, and regress this on $x_2$.

What's the name of this method, is there a regression of this sort? Does it sound like the right solution to my problem?

Due to practical constraints, I want my model to include all $x$s at the same time, rather than switch between different models on the fly.

• By time series, do you mean $y = y(t)$ and $x_i = x_i(t)$? Also, it's not clear what you mean by the statement "each $x_i$ must be sampled on different criteria..." From what I understand based on your description, your approach seems like boosting (en.wikipedia.org/wiki/Gradient_boosting). Dec 9 '15 at 5:16
• Yes, that's what I mean by time series. Sampled on different criteria means on different clock. Dec 9 '15 at 5:30
• Can you explain what the training data looks like? It seems like you have a set of $n$ training samples of the form $(y^{(1)}, x^{(1)}_1, x^{(1)}_2, dots), (y^{(2)}, x^{(2)}_1, x^{(2)}_2, \dots), \dots, (y^{(n)}, x^{(n)}_1, x^{(n)}_2, \dots)$ where $y^{(j)}$ and $x^{(j)}_i$s are times series and $x^{(j)}_i(t)$ may or may not be available for some $t$. Is this correct?
– Sobi
Dec 13 '15 at 5:32

I'm posting as an answer since it's too long for a comment.

If I understand correctly, you have data coming from, let's say, $N$ multiple sensors $x_i^{(t)} \in \mathbb{R}^{T \times N}$ where each sensor is a time series with $T$ data points. You also have a target output variable $y^{(t)} \in \mathbb{R}^T$.

You would like to perform a prediction using a linear model and you would like to adjust the subsequence sample size (or sliding window size) for each $x_i$ sensor.

One option is to take a larger fixed size sliding window over your time series, $X^{(t)} \in \mathbb{R}^{W \times N}$ where $W$ is the window size. Then, there are methods that perform regularization for all $x_i$ such as group lasso or group elastic net or ridge regression. The subsequence for a sensor is one group.

In effect, this adjusts the $b_i^{(w)}$ for all $x_i^{(w)}$ giving a specific weight to each one of the subsequence points in your window, effectively adjusting the window size or sample size separately per $x_i$.