Correcting for attrition in panel data I have a dataset of online Reddit posts for unemployed individuals in two neighboring U.S. states, where unemployed workers in a given state received higher unemployment insurance UI benefits than the other (i.e. my treatment variable), and I am trying to test if receiving higher UI benefits causally correlated with more positive sentiment.
Specifically, I track online sentiment for both groups for around 7 months before the treatment and around 9 months post-treatment. However, one challenge I am facing is that the 'treated' unemployed workers become less active on reddit once their state passes the higher UI benefits (i.e. similar to attrition bias in surveys where individuals from a treatment/control group drop out of the survey). I have read about techniques explained in the blog article above on how to address and reduce attrition bias with survey data, but I wonder if applying the same techniques makes sense also with administrative or observational online text data?
Here is a sample of the data's structure:
Each row represents an online post written by an individual.
The first variable refers to username, followed by the date of a given post, the predicted ML sentiment/mood for that post.
The column treatment_time refers is a binary variable telling us whether the post was written before or after treatment, while the variable treatment_status tells us whether the observation received treatment or is in the control group (i.e. the neighboring state where UI benefits were not extended during Covid).
For instance, I have observations such as Kenny and Cartman from the treatment group who only remain active for a few days post-treatment with the last posts they write trending more positive, relative to pre-treatment. However, I am not sure if I can interpret their attrition (i.e. leaving the data) as a proxy evidence that the policy was effective in that individuals no longer felt the need to describe their job situation on Reddit?
On the other hand, individuals like Mr. Garrison from the control group remain active on Reddit post-treatment and continue posting statements that are classified as negative or neutral.
# Treatment (i.e. extended UI benefits) occurs on Sept 6, 2020.
username        date         mood         treatment_time    treatment_status 
Kenny       2020-09-02.       negative       pre               treatment 
Kenny      2020-09-03.        negative       pre               treatment
Kenny      2020-09-07.        positive       pre               treatment
Cartman   2020-09-03.       negative       pre               treatment 
Cartman  2020-09-06.        negative       pre               treatment
Cartman  2020-09-08.        positive       post              treatment
Mackey   2020-09-03.       negative        pre               control 
Mackey   2020-09-04.       negative        pre               control
Mackey  2020-09-08.        negative        post              control
Mackey  2020-09-13.         neutral         post             control
Mackey  2020-09-14.         neutral         post             control
Mackey  2020-09-23.         neutral         post             control
Garrison   2020-09-03.      negative        pre              treatment   
Garrison  2020-09-04.       negative        pre              treatment
Garrison  2020-09-04.       neutral         pre              treatment
Garrison  2020-09-05.       negative         pre             treatment
Garrison  2020-09-14.       neutral         post             treatment
Garrison  2020-09-19.       negative        post             treatment

 A: A key issue in the analysis of longitudinal data with attrition is whether you can make the missing at random (MAR) assumption.  [Aside: this is related to the independent censoring assumption in time-to-event analysis.]  If you can't make that assumption there are not many available tools for analyzing the data unless you are in the once-in-a-lifetime situation of knowing how to model the dropout process.  To be able to assume MAR requires a certain frequency of data collection so that there are few long gaps.  That is because MAR means that conditional on baseline variables and on previously measured longitudinal responses, missingness of the response at the current follow-up time is completely random.  In other words, we need the only things that predict missingness at time $t$ to be baseline variables and responses between time zero and time $t$.  Thus previous responses are assumed to provided a "preview" of missingness/dropout.  Under these assumptions longitudinal models may be used as long as they are full likelihood models (i.e., generalized least squares, Markov models, random effects models, not GEE).
A: I can't point to relevant literature. But I have a suggestion. How about using outcomes positive/drops out/negative? Then, secondarily, if you can predict dropping out as an outcome, also report the average sentiment in last post.
It seems to me that this approach could result in a useful and convincing story from the data.
