Time data that aren't time series - except now, consider a duration aspect

Question

This is a follow-up to the question How to approach time data that aren't time series?.

I realized that my prior question was missing an important nuance, so I have to change the hypothetical scenario entirely.

Imagine the following scenario: suppose I, as a researcher, hypothesize that teachers who have more experience teaching tend to give their students more As. As with the prior question, significance tests and effect sizes are desired. Because teachers have to receive certification to teach through obtaining an academic degree, I suspect that there's an association between how long ago a teacher graduated from their certification program, and the rate at which teachers issue As to their students out of all students taught so far.

You may assume that I have available the following in a data set: points $\{(u_i, t_i, \mathbf{y}_i)\}_{i=1}^{N}$ where

• $u_i$ is the unique identifier for a teacher
• $t_i$ is the year at which the teacher graduated from their certification program
• $\mathbf{y}_i = (y_{1i}, y_{2i})$ is a vector consisting of two components for each teacher: the count of As ($y_{1i}$), and the count of grades that are not As ($y_{2i}$) since the teacher graduated from their certification program.

Like with the prior question, I don't think this is a time series problem, but upon re-reading the prior answer, I don't think this is a panel data or longitudinal data problem either. We run, however, into similar problems to the ones I pointed out in the prior question:

• We don't have an equal number of teachers in each year for which teachers graduate.
• If my sample is insufficient so that some calendar years were skipped, I will have data for some calendar years but not for others, breaking the usual "equal-spacing" assumption with time series.
• The data are probably not stationary if we were to consider using the graduation year as an index for time.
• We could attempt to bin graduation years together, but such a mechanism has no precedent (i.e., assume no similar study has been done), and would be quite arbitrary to implement.
• Given $t_i$, we also suspect that $\sum_{\{i : u_i \text{ graduated in year }t_i\}}\dfrac{y_{1i}}{y_{1i} + y_{2i}}$ (the rate at which As were given) would be similar for values of $t_i$ that are close to each other, so any binning of graduation years we do would ignore this to an extent that I would not be comfortable with without adequate justification.

Additionally, there's an aspect of this that should be controlled for: teachers who have graduated at an earlier year naturally have higher counts in $\mathbf{y}_i$ than those who graduated at a later year.

I suspect this is more complex than viewing this as a time series or longitudinal data problem. What technique(s) would you suggest for approaching this problem?

EDIT: As mentioned in the comments, assume all $\mathbf{y}_i$ are gathered at the same time and that all teachers in the data are still employed at this institution.

Answer 1

As you have described the situation, what you have is a binomial regression model (total A versus not-A counts per teacher) with year of teacher graduation as the only predictor. Although you have an ID value for each teacher, you seem to have only 1 entry (cumulative counts) for each teacher rather than entries year-by-year, so that ID doesn't even need to be included in the model. You don't have a correlation structure of outcomes over time that would require special care in a time-series regression (or at least, without year-by-year values for the $y_i$, you can't address them).

The only question is how you want to model the predictor of teacher year of graduation. If there's a large enough range of years you could simply treat it as a continuous predictor. In a logistic binomial regression, to allow for non-linear associations between log-odds of having given A grades and time since teacher graduation, you could model that predictor flexibly, for example with restricted cubic splines. That way you get an overall test of the significance of the model and tests of the significance of the non-linear terms. If there are only a few years of teacher graduation in your data, you might be better off evaluating it as an ordered categorical predictor.

The above regression approach naturally takes into account things like years having different numbers of teachers graduating, even down to 0-teacher years (at least for the continuous model). The results will tend to be more weighted toward years that had a greater numbers of teachers and toward teachers who have given out greater numbers of grades.

What your model doesn't take into account is the teachers who have left the field since graduation. That could wreak havoc with interpretation of your results. For example, say that stricter teachers who give out fewer A grades get so much blowback from parents of their students that they tend to resign in disgust. Then an observation of a higher A-grade proportion with longer time since teacher graduation might represent a type of survivorship bias. I understand that this question represents a hypothetical to represent your actual situation, but survivorship bias needs to be considered in any study involving time durations up to an observation time.

In terms of an "effect size," with a logistic regression you would need to decide whether to keep model results in the original log-odds scale or translate to, say, a probability scale. Depending on the results of your modeling there might not be a single "effect size" that covers all the data, so you might need to provide illustrative examples. A plot of predicted results versus year of teacher graduation, with confidence intervals, would be important to show.