0
$\begingroup$

For several weeks now, I'm stuck with this problem:

Given some $x_i$ for $i\in1...n$ I have to make a prediction (by some regression algorithm such as OLS regression or regression trees, since the response is continuous) for $y_i$. Not only the 'predicted value' is necessary to know but also the lower and upper bounds in order to say that $y_i$ lies within $x$% certainty within these bounds (hence, we have to create a prediction interval or use quantile regression I think?).

Now the difficulty comes in play:
We now have a prediction of $y_i$ including its upper and lower bounds but we eventually have to use these values to make a prediction(interval) of the total value $y=\sum_i^n a_iy_i$ (where $a_i$ is known for every $i$). I would simply think that we can just add the individual lower/upper bounds for the $y_i$ and arrive at the interval in which we can say that $y$ lies within $x$% certainty between these values?

In short: How do we create a $x$% prediction interval for $y$ by using information about the predictions of $y_i$?

Thanks in advance!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

No: building confidence intervals by using extreme values of previous confidence intervals does not work. For example, please take a look here: (pdf). The point is that such approach would work if confidence intervals contained the true parameter value with $100\%$ probability since, if two parameters are certainly between $2$ and $4$, then their sum is certainly between $4$ and $8$. But $($considering two variables $X_1$ and $X_2$ with the same variance $SE^2)$ if there is a $95\%$ probability that the interval $(2,4)$ contains the true values of both parameters, then there is no reason for the probability of the $(4,8)$ interval containing the true value to be $95\%$, unless the correlation between the two intervals is $1$ (so that you are basically considering a unique interval and multiplying it by two). For example, if the two quantities are independent, the variance of the sum will be twice the variance of single values, so the standard error of the sum will be given by the SE of single values multiplied by $\sqrt{2}$, so that: $SE (X_1+X_2) = \sqrt{2}*SE$, thus the confidence interval will be shorter than the sum of single confidence intervals. So, if the $Y_i, 1 \leq i \leq n$ have a multivariate normal distribution, you can calculate the variance of $Y=\sum_i^n a_iY_i$ by using variances and covariances. In case of independence of the $Y_i$, you would have that: $SE^2(Y)=\sum_i^n a_i^2 SE^2(Y_i)$, and from that formula you could derive your $SE(Y)$ to build your confidence interval for $Y$ (of course, you should add covariances to your formula, in case they are not null).

EDIT 24/03/2018

If your variables are not normally distributed, in case your observations $a_i*Y_i$ are i.i.d. you can still rely on the Central Limit Theorem if the sample is large enough (the sample size required for the CLT to hold depends on how far from Normality your distribution is, but $30$ is usually suggested as a rule of thumb). If you can group your dataset in sets of $30+$ i.i.d. observations, then you would still have an approximate Gaussian distribution if multi-normality holds for the sum of variables in each set (given your quantity $Y$ would be the sum of normally distributed variables). Otherwise you can't assume normality of your sum, unless you assume again multinormality of your $Y_i$'s. Also, you need to know the correlation structure of your variables $Y_1, \dots, Y_n$. For example, if the correlation between all variables is $+1$, or if it is $+1$ for some pairs and $-1$ for others, you would actually have just $1$ observation. The point is: to know the variance of a sum of normally distributed variables (when joint normality holds), you would need to know their covariance matrix.

EDIT 25/03/2018

Yes: if both the $X_i$ and the random errors $\epsilon_i$ are jointly independent (and obviously also independence between the $X$’s and the $\epsilon$’s hold, and $Y_i = \alpha_i + \beta_i X_i+\epsilon_i$ $\forall i$ in $(1, \dots, n)$, then you have $Y_i$’s are also jointly independent,thus the same holds for $Z_i=a_i*Y_i$ sequence (since $a_i$ is a sequence of constant values). And yes, I agree with you: at the end of the day you have to check whether you can apply the Lindeberg condition for your $Z_i$ sequence. Again yes, the modified version equally apply to sums or averages (given if Z has a normal distribution, then also W_1=Z/n and W_2=Z*n are normally distributed). As for your covariance matrix, the independence between $X_i$ and $Y_j$, the one between $X_i$ and $X_j$ and the one between $Y_i$ and $Y_j$, imply that their covariances are all zero. Thus, you just have to calculate the covariance of each ($X_i, Y_i$) pair.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for your Response! In fact I was targeting at the sum of the quantiles of some prediction interval. But, as far as I can understand, it's not possible to 'add the quantiles' of $a_i y_i$ to construct a prediction interval for $y=\sum^n_i a_ i y_i$ and I have to do something with the individual standard deviations.. Do you have any suggestions how i can construct a prediction interval for $y$ in case I do not certainly know whether $y_i$ and the corresponding $y$ are normal but I do know each $\sigma_(y_i)$ the corresponding $\sigma_y$?? $\endgroup$
    – Bas van der Reijden
    Commented Mar 23, 2018 at 19:01
  • $\begingroup$ Response on the edited version: The variables $Y_i$ are predictions made on the properties of (independent) $X_i$'s; furthermore n exceeds 10.000. I (can correctly?) assume that $Y_i$ are independent and hence I think I'll use the (modified version such as Lindeberg or Lyaponuv, which does not assume that the $Y_i$'s are ident. distributed) CLT to let $Y$ be normally distributed ;)! Lastly, I was questioning myself if it is possible to use the modified CLT for sums of random variables and if I can calculate the cov matrix by using info about the $X_i$ and $Y_i$? $\endgroup$
    – Bas van der Reijden
    Commented Mar 25, 2018 at 14:02
  • $\begingroup$ I edited my reply again. I assumed you asked about the covariance matrix of the set of $2n$ variables $X_1,\ldots, X_n, Y_1,\ldots, Y_n$: did I understand correctly? $\endgroup$
    – Federico Tedeschi
    Commented Mar 26, 2018 at 6:57
  • $\begingroup$ Thanks, I now think we come to the point and I can do something with this result! To summarize: I'll have to make estimates of the variances of $Y_i$, calculate the covariance matrix of $Y_1...Y_n$ (which is zero I suppose, because of the independence in observations?), consequently add the variances of $Y_i$ to obtain the variance of $Y$ and apply the CLT in order to make a prediction interval for $Y$? $\endgroup$
    – Bas van der Reijden
    Commented Mar 26, 2018 at 17:17
  • $\begingroup$ If the conditions of my edit of 25/03/18 are met, then you have that the $Y_i$ are jointly independent, thus their covariances are all zero's. Under the same conditions, their variance is given by $\beta^2*SE^2(X_i)+SE^2(\epsilon_i)$. The variance of $Y$ is then given by $\sum_{i=1}^{n}a_i^2*SE^2(Y_i)$. In case the Lindeberg condition is satisfied (I think possible questions on it are more fit on the Math forum), you can then apply the $CLT$, eventually deriving your CI for $Y$ from its (approximate) normality. $\endgroup$
    – Federico Tedeschi
    Commented Mar 27, 2018 at 7:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.