I'm trying to figure out whether how the S&P500 reacts from the change in bullish sentiments of people and the change in people's allocation in stocks. The bullish proportion and stock allocation are my explanatory variables (in percentage) and monthly S&P500 closing price is my independent variable. Below is an example snapshot of my data:

Date             SP500 Stocks Bullish
1 2016-07-30  100      51         25
2 2016-08-30  109      40         32
3 2016-09-30  107      42         29

If I detrended my S&P500 monthly prices using the loess() function in R and got the cyclical components, how would I interpret the coefficients in regression model? If the coefficient for Stocks was 3, would it be that for every 1% increase in stocks allocation, that S&P500 prices would increase by 3 points? Because the prices are detrended, I think that I am interpreting the units of the regression output incorrectly, especially for the independent variable.


1 Answer 1


Quick comments:

  • Stock prices almost certainly aren't stationary. If you try to do regression with a process containing a unit root, you probably won't be estimating what you think you're estimating. I personally don't understand what you're doing with stock prices and loess.

  • Generally when one works with stocks etc..., one works with:

    • Returns: $ R_t = \frac{S_t + D_t}{S_{t-1}}$ where $D_t$ includes all distributions and $S_t$ is the stork price.

    • Log returns: $r_t = \log R_t = \log\left(S_t + D_t \right) - \log S_{t-1}$

      You might also have some ratio like the dividends to price ratio, earnings to price ratio or some other ratio that is stationary.

  • Timing is incredibly important. You need to be meticulously precise about the timing of variables and match what you're doing to the question you're asking.

    • For example, if you regress the stock return for January on a survey result for the end of January, a statistically significant result could be due to people reacting to what the stock return was! (eg. what if your end of January survey question was, "how much did the market go up?")
    • If you're trying to predict returns at time $t$, it is cheating to have information on the right hand side of your regression that isn't available at time $t$.
  • $\begingroup$ I used the Loess function in R since I was partially emulating the method of extracting the cyclical component used in ionides.github.io/531w16/midterm_project/project20/midterm.html: "The mid-range frequency component can be considered as the cycle or the perturbation caused by the state of the economy, unlike the meaningless noise component" $\endgroup$
    – user112947
    Feb 28, 2017 at 21:16
  • 3
    $\begingroup$ @user112947 I would very strongly caution against using "smoothing" methods which are not causality-preserving if you intend on using the output for further analysis. In particular, LOESS will smear out effects over time and leak information from the future. Another method which is often incorrectly used by economists for a similar purpose is the HP filter. Do not do this. $\endgroup$
    – Chris Haug
    Mar 1, 2017 at 1:35
  • $\begingroup$ I am more used to calling $R_t=S_t+D_t-S_{t-1}$ as "returns" whereas $\frac{S_t+D_t}{S_{t-1}}-1$ as "percentage returns". But maybe that is not the common thing. $\endgroup$ Mar 1, 2017 at 8:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.