2
$\begingroup$

I am new to programming as well as econometrics and would like to ask some guidance for learning.

I am planning to calculate the effectiveness of a public works scheme in country A. By effectiveness I mean the rate of success by which participants were able to return to the open labour market compared to registered job seekers who did not participate in the programme.

I have found out from economists that for this type of analysis I can use propensity score matching (PSM) if I have panel microdata. I have already submitted a data request to public authorities, and I expect to receive a tidy, unblanaced dataset with several thousand observations for the past three years.

I was also recommended some books from which I could learn how to conduct the analysis (Woolridge 2012, World Bank 2009). All of these use Stata, but, if possible, I would prefer to stay with R that I have been learning since the past year. Unfortunately, however, I have not found a book yet that would comprehensively treat PSM in R (eg. among the useR series).

I am a beginner in R. At the moment, I can manipulate data by indexing, but not yet by loops. In order to avoid the risk of taking on something that I am unable to do, I would like to ask whether I can expect to conduct such an analysis by basic R programming skills, and reading about packages on panel data (eg. plm) and PSM (eg. MatchIt), or it is essential that I also learn some more advanced skills before, such as loops, apply, or writing functions? If the latter, then concretely which ones?

$\endgroup$
  • 1
    $\begingroup$ Pause and think through the analysis goals before getting into those questions. State the goal clearly, and state the sample size and number of possible adjustment variables. See if covariate adjustment works for you. Propensity score analysis is needed only as a data reduction technique when you have lots of exposures but have a small effective sample size on the outcome variable. $\endgroup$ – Frank Harrell Jun 18 '17 at 13:00
  • $\begingroup$ I will know more about data parameters when I receive them. But just to clarify your last sentence: Do you mean that PSM is only needed when there are many registered unemployed, but only a few of them have taken part in the public works programme? $\endgroup$ – malasi Jun 19 '17 at 20:43
4
$\begingroup$

I don't mean to get your hopes down, but expecting "tidy data" from "local (government) authorities" is probably a pipe dream. Government built/owned datasets are usually a mess. If I were you, I'd spend a day learning the basics of R with any of the free web-tutorials that are available. Outside of the classroom, I've never worked with a clean dataset -- especially ones that came from government entities -- that didn't require some cleaning and data manipulation before the analysis began. That way you'll be fairly well prepared to handle any data issues that come your way (which there will almost certainly be some, even if promised a tidy dataset). I'm not sure you'll need to use looping in R, since R has vectorized operations which eliminates the need for most looping.

The books you are using are probably books by economists. Economists tend to use stata more often, so, I'm guessing that's the why you find these books are using stata. Try looking for statistics books, by actual statisticians on the subject of propensity scores, if you'd like to gain a better understanding of both the theory of propensity scores as well as learn how to apply them in R (or SAS). I'd recommend the following books for you which contain plenty of R examples:

  1. [Propensity Score Analysis, 2nd. Guo, Shenyand, and Mark W. Fraser.]
  2. Propensity Score Analysis: Fundamentals and Developments. Pan Wei and Haiyan Bai PhD (Editors).
  3. Using Propensity Scores in Quasi-Experimental Designs. William M. Holmes. (a bit more basic/introductory book).
  4. Practical Propensity Score Methods Using R. Walter L. Leite.

With regard to packages, in R, I really like the twang package. It was developed by the statistics group at RAND and in developing the package, they put together a lot of really great resources to train others how to use the package. Since it was federally funded, I believe, you can access these materials and learn the package for free. This RAND website contains documentation, examples, and even video instruction on how to use the package. You can even match the methods they teach to papers that have been published so you can see how they've incorporated the findings from the package into their public research. From personal experience, I can tell you that without these materials, it would have taken me significantly longer to complete my thesis which incorporated their boosted regression trees to obtain propensity scores. The downside to this is that the original author of the package is no longer with RAND, so I'm not sure if it is being well-maintained, although, I've thought about making some enhancements to the package myself.

In addition, there is some evidence in the literature that shows boosted regression trees (used in twang) outperform the propensity matching of other propensity score models (e.g. logistic regression).

Lastly, I agree with Frank's suggestion of thinking through the problem. You don't necessarily have to use propensity score matching. There are other causal methods that might suit your needs better and build a stronger case for your argument. Consider other casual methods such as what economists like to call Fixed Effects Regression (not to be confused by what statisticians called fixed effects) or interrupted time series methods might work too.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Thank you very much for the plenty of resources and the hints where to look for them! The RAND learning site just looks perfect for a newbie like me: it is free, works with R and even has video presentations! Also, I just completed a Data cleaning class online, so I hope all goes fine. I will have to find out more about the alternative models though that you and Frank suggested. $\endgroup$ – malasi Jun 19 '17 at 20:36
  • $\begingroup$ A couple of comments (also to @malasi): propensity score methods are no more causal than regression; both depend on observed covariates. Second, if you use flexible additive spline models for propensity scores you don't need to check for balance. $\endgroup$ – Frank Harrell Jun 19 '17 at 23:00
  • $\begingroup$ But propensity score methods check for covariate balance, whereas regression will not do this, unless you do you it as a diagnostic check. I suppose one could say the same about propensity scores, but the use of propensity scores requires covariate balance checks before proceeding with modeling. $\endgroup$ – StatsStudent Jun 19 '17 at 23:07
1
$\begingroup$

MatchIt, Matching, twang, and CBPS are all great packages for propensity score analysis. For balance assessment, I recommend cobalt. If your data are clean, I can't imagine what type of looping you might have to do. You might have to do indexing, but these packages are designed pretty well to prevent you from having to do too much work outside of running the important functions. These packages (except CBPS) all have good vignettes explaining how to use the functions for propensity score analysis.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ thank you so much for the hint! It is reassuring to read that I do not need much looping or function writing. Not that I do not want to learn those things, but it is good to know, I do not need to rush with that, and I can focus on the statistics part. $\endgroup$ – malasi Jun 19 '17 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.