I have a single output variable $y$ and a number of inputs $x_1$, $x_2$, etc. These are time series. Each $x_i$ explains the changes in $y$ in specific circumstances, and the goal is to have a linear model that looks like $y=b_1x_1+b_2x_2+...$ The point is that each $x_i$ must be sampled on different criteria specific to its qualities, and its relationship to $y$ to be established only at those situations when that particular $x_i$ is the lead influence on $y$.

My approach is to take one sampling method $s_1$, sample $y$ and $x1$ on it, specific to $x_1$. Do the regression $y$ on $x_1$. Then, take the error $y-b_1x_1$, sample the error on another sampler $s2$, and regress this on $x_2$.

What's the name of this method, is there a regression of this sort? Does it sound like the right solution to my problem?

Due to practical constraints, I want my model to include all $x$s at the same time, rather than switch between different models on the fly.

  • $\begingroup$ By time series, do you mean $y = y(t)$ and $x_i = x_i(t)$? Also, it's not clear what you mean by the statement "each $x_i$ must be sampled on different criteria..." From what I understand based on your description, your approach seems like boosting (en.wikipedia.org/wiki/Gradient_boosting). $\endgroup$
    – Vimal
    Dec 9 '15 at 5:16
  • $\begingroup$ Yes, that's what I mean by time series. Sampled on different criteria means on different clock. $\endgroup$ Dec 9 '15 at 5:30
  • $\begingroup$ Can you explain what the training data looks like? It seems like you have a set of $n$ training samples of the form $(y^{(1)}, x^{(1)}_1, x^{(1)}_2, dots), (y^{(2)}, x^{(2)}_1, x^{(2)}_2, \dots), \dots, (y^{(n)}, x^{(n)}_1, x^{(n)}_2, \dots)$ where $y^{(j)}$ and $x^{(j)}_i$s are times series and $x^{(j)}_i(t)$ may or may not be available for some $t$. Is this correct? $\endgroup$
    – Sobi
    Dec 13 '15 at 5:32

I'm posting as an answer since it's too long for a comment.

If I understand correctly, you have data coming from, let's say, $N$ multiple sensors $ x_i^{(t)} \in \mathbb{R}^{T \times N} $ where each sensor is a time series with $T$ data points. You also have a target output variable $y^{(t)} \in \mathbb{R}^T$.

You would like to perform a prediction using a linear model and you would like to adjust the subsequence sample size (or sliding window size) for each $x_i$ sensor.

One option is to take a larger fixed size sliding window over your time series, $ X^{(t)} \in \mathbb{R}^{W \times N}$ where $W$ is the window size.

enter image description here

Then, there are methods that perform regularization for all $x_i$ such as group lasso or group elastic net or ridge regression. The subsequence for a sensor is one group.

In effect, this adjusts the $ b_i^{(w)} $ for all $x_i^{(w)}$ giving a specific weight to each one of the subsequence points in your window, effectively adjusting the window size or sample size separately per $x_i$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.