0
$\begingroup$

Is it right to use linear regression to make a forecast based on social media impact?

Suppose you have the next dataset (events), where time delta is amount of minutes passed from the previous event:

+-----------------------------------------+
|event  time   current   # of related     |
| id    delta   price    Google's articles|
+-----------------------------------------+
|  1     1      50.60      110            |
|  2     15     50.71      120            |
|  3     38     50.85      120            | 
| ...    ...     ...       ...            |
| 100    4      80.70      120            |
| 101    8      80.71      120            |
| ...    ...     ...       ...            |
| 203    61     90.01      142            |

I'm confused about two things that make me hesitate about applicability of linear regression here:

  1. Social media (# of related articles) influence current price, but not immediately. The delay might be from few hours up to few days/weeks. That means that # of related articles is always outdated to current price. All I figure out - is to calculate average delay of social media impact and shift current price data relative to # of related articles. Is there any better solutions?
  2. current price is updating much frequently than social media impact (# of related articles). I.e. in data example below you see that price growth from 50.60 up to 80.71 thanks for increase of # of related articles from 110 to 120. But from event id 2 up to event id 203 # of related articles remains the same. Is linear regression able calculate correct coefficients based on this data? Any tricks?
Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Within the disipline of time-series analysis, the "shifting" you discuss is more traditionally referred to as lag — the term used by @Grosvenor below. It's quite common in time-series analysis to regress on lagged time series, and even on multiple lags of the same time series. Under #2, you have a missing data problem, which might be solved by interpolation or smoothing. Thus 'accessorized' by these additional techniques, regression remains technically a feasible and appropriate approach to your problem. $\endgroup$
    – David C. Norris
    Commented Sep 8, 2017 at 21:34
  • 1
    $\begingroup$ @DavidC.Norris thanks, your comment is really usefull $\endgroup$
    – VB_
    Commented Sep 11, 2017 at 9:45

2 Answers 2

Reset to default
2
$\begingroup$

For question 1, because the delay can vary, a linear mean might not tell the whole story. Instead, you could check the lagged residuals. A lag plot will tell you how large this effect is, and the degree of serial correlation. Then you could try a Durbin-Watson test.

Improve this answer
$\endgroup$
1
+50
$\begingroup$

For question 1, to add to the answer by @Grosvenor, you can also include the lagged values of the number of articles in the linear regression as predictors. The regression will figure out which lagged values or combinations to include. You move into the direction of time series here, and there is a lot going with autocorrelation and things like that, but, this could be a useful first step.

For question 2, linear regression will still calculate coefficients without problems. It always does, unless you have too few datapoints, then the system is underdetermined. These are based on minimizing the squares between all the differences between prediction and actual value (to pick a single explanation of linear regression), and the fact that a line doesn't fit through multiple points doesn't change much for that step. Whether they are "correct" is a bit of a philosophical question maybe, and I would say that they are never correct. A model is just a system that you came up with to describe what you see. How useful the model is remains to be seen.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.