1
$\begingroup$

I would appreciate some methodology and R implementation help for a thing I'm working on.

I have daily observations of $Y$ for several countries over several years, I will use quite a few independent variables $X$.

My hypothesis is that there was a break at a specific time that has affected the determination process of $Y$ and altered the coefficients of a regression. I’m interested in knowing what the effects of $X$ on $Y$ were before and after the break and whether there was a significant change in these relationships after the break.

My idea is to use multiple regression for each country of the form:

$$Y = a + B_1 X_1 + B_2 X_2 + B_3 X_3 + D + B_4 D X_1 + B_5 D X_2 + B_6 D X_3 + e $$

where $D$ is a dummy variable equal to 1 after the suspected break.

My test for whether the coefficients are different after the break is then simply to test the significance of the coefficients on the interaction terms: $B_4, B_5, B_6$. I know how run this regression for individual countries.

Q1. Will this tell me what I’m looking for or should I use something else like a Wald or Chow test?

Q2. Is this called a natural experiment?

Q3. If I want to run this as a time fixed effects panel regression, how is this done in R?

The suspected break is the introduction date of a new financial regulation. It’s possible that there was a gradual change over a few months in anticipation of the regulation.

Q4. Could I cut out a few months of the data to remove the effects of a gradual change?

Q5. Should I use some method to look for a break before the suspected break date?

Bonus question: In R, the output of these two regressions are the exact same, I’m using the lm function:

$$\begin{array}{rcl} Y & = & a + BX + D + BXD \\Y & = & a + BX + BXD \end{array}$$

Both models return a coefficient specific to the dummy variable D, even though the second model only uses D in the interaction term, why is this and can prevent it?

$\endgroup$
1
$\begingroup$

There is a lot going on in the question. You'll get better responses if you narrow the focus of your question. Further, you'll better understand the question yourself.

Here is some suggests which will lead you in the right direction:

Theory:

Chow Test

You're going to want to do a simple chow test first around the suspected break data. This is a good start, however, you're example suggests that you'd rather be agnostic about the exact break data, which is a good idea.

Endogenous Testing

The problem with your suggested approach is that the distribution of the test statistics is not going to follow a typical normal distribution. To begin to understand why this is, image you have 100 different potential breaks and you test each of the resulting 100 dummy variables, using a 5% significant level. By pure luck you're going to fail to reject that dummy variable is different from zero for 5 breaks on average.

So what do you do? Fortunately, some very smart people figured out the correct distribution of the test statistics. The testing procedure is as follows, roll a chow test over your data, compare each of these test statistics to critical values obtained by the aforementioned very smart people. Break date which corresponds to the maximum test statistics, which is statistically significant, is the most likely break date

Casual Inference

Imagine that you find a break date. Good! One way to estimate the effect of the break is to now run a model which includes a dummy variable which is zero before the date and 1 after. However, the question now is this event casual?

Let's look at a simple example, assume a country deploys some stimulus package on a given date and you want to measure the impact of the stimulus on GDP. Does this coefficient corresponding to the occurrence of the event measure a casual relationship? The answer is, maybe. But probably not. Presumably, the stimulus was dispensed because the country was doing poorly. Therefore, poor GDP could have caused the stimulus, not the other way. This is known as an endogenity problem. The stimulus package is not truly a natural experiment.

Implementation in R

To implement this in R, you're going to want to use the strucchange package. The documentation is pretty good, here is the vignette

$\endgroup$
  • $\begingroup$ Great answer, thank you. I read up some more and I have a few quick follow-ups: Is rolling a chow test over the data also called Quandt likelihood ratio/sup-Wald? I guess the function Fstats from strucchange is what I need. In the example in the link you provided they feed an ECM model into this function, do I need that or can I just use a normal lm(Y ~ X)? Could I feed a panel regression into the Fstats function to consider all my countries simultaneously? $\endgroup$ – Mr. T-stat Mar 25 '17 at 15:59
  • $\begingroup$ I like the point you made about endogeneity problems. Are such issues addressed by theoretical reasoning? I have an argument for why the regulation (break-causer) would affect Y but the other way around doesn’t make that much sense, but I will think about it and include it in my paper. $\endgroup$ – Mr. T-stat Mar 25 '17 at 15:59
  • $\begingroup$ Yes, you're going to want to use the Fstats. You don't need to feed it any ECM model, any model will work. However, your data many necessitate that you use an ECM model. For example, if your data follows a stochastic trend, (i.e. unit root) then this will impact your testing. You'll want to read about stochastic trends a bit to see if you think you have such a problem. It will depend on you situation and data. $\endgroup$ – Jacob H Mar 25 '17 at 21:12
  • $\begingroup$ Endogeneity is a very thorny problem and there are not great ways to test if it is present. Generally, the best way to convince people you don't have such a problem is by arguing it. Therefore including an argument within the paper would be a great idea. However, remember, there are other causes of endogeneity. For example, omitted variable bias. $\endgroup$ – Jacob H Mar 25 '17 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.