0
$\begingroup$

I have a time series dataset. The,

X (Independent variable) is time and is denoted as 1,2,3,4,5,6..1000.etc Y (Dependent variable ) is a percentage scale as 99%, 98.7%, 96%, 91% ...etc. This is a continuous data set.

I have 1000 such data points. The first 700 data points used as training set and rest 300 is used for testing.

I tried to use simple linear regression but when predicting sometimes the prediction is more than 100%. And the case is even worse when I calculated the confidence interval and prediction interval.

So I tried to use logistic regression as there is a boundary ( from 0% to 100%). But logistic regression can take only binary data. I am confused on how to appropriately convert my existing time series data so that I can try how logistic regression on that.

Improve this question
$\endgroup$
1
$\begingroup$

You're correct that logistic regression is only for binary response data, which is not applicable here. What you may be wanting to do is simply apply the logit transform to the response data (i.e. the $Y$ values) and then use linear regression on the transformed data. Then apply the inverse logit transform to predictions to put them back on the original scale.

However, if you are trying to forecast a time series, simple linear regression may not be the best approach. It may be better to fit a time series model to the data (after applying the logit transform to the response) and use that as a basis for prediction.

Improve this answer
$\endgroup$
8
  • $\begingroup$ There is also the variance stabilized arcsine square root transformations if $Y$ is a proportion. $\endgroup$ – Frank Harrell Mar 12 '16 at 16:37
  • $\begingroup$ A minor remark is that after such rescaling the MSE error changes significantly (algorithm will pay more attention to 'limit cases'), but this is solvable e.g. by adding weights $\endgroup$ – Alleo Mar 12 '16 at 16:40
  • $\begingroup$ Hello Brent, Thanks for your reply. Can we also logit transform the X (time scale) with Y and then use regression? $\endgroup$ – Muk Mar 12 '16 at 17:14
  • $\begingroup$ It's not clear why you would want to transform the time scale. You could do it, but unless you have a good reason for doing so, it probably wouldn't be recommended. $\endgroup$ – Brent Kerby Mar 12 '16 at 18:03
  • $\begingroup$ Hey Brent, Thanks again. I tried and it looks good now. But I was wondering on how do we calculate the confidence and prediction intervals the predicted Y's. If we do it after inverse logit then the Y crosses again 100%. Could we do it before logit? If so can you advice me on that procedure? $\endgroup$ – Muk Mar 12 '16 at 18:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.