1
$\begingroup$

I am working on a dataset where a new administrator joins a school, and I want to see if it affects the number of students at the school. There are also characteristics of the administrator (age, gender, years of experience) and types of students (split by grade and core subject). I have the number of each type of student for the 3 years before and the 3 years after the administrator joins the school.

I know I'll need to run many tests to determine which administrator characteristics are significant, which student types are most affected, etc, but what has me stumped is how to use the 3 years before and 3 years after data. We don't know how quickly students may leave. I was thinking of using a paired t-test with the 9 year pairs tht could exist (-3 years and 1 years, -2 years and 1 years, etc) but that doesn't seem most efficient. Any suggestions for how to tackle this?

Improve this question
$\endgroup$
5
  • $\begingroup$ Your options are limited, because although you might be able to detect a change in the number of students after the administrator joined the school, you have no information that permits you to attribute that change solely, or even in part, to the administrator. Surely many other things happened at the same time, both at the school, in the neighborhood, and the world at large, that plausibly could cause the student count to change. Moreover, if the administrator was brought in to slow an imminent change, you might have cause and effect exactly backwards and risk drawing a very wrong conclusion. $\endgroup$
    – whuber ♦
    Commented Feb 5, 2022 at 16:18
  • $\begingroup$ The conclusion of this analysis isn't to say that the administrator caused a student change, but rather that the administrator is a predictor of student change. $\endgroup$
    – Nomi
    Commented Feb 6, 2022 at 17:06
  • $\begingroup$ It is still invalid to draw such a conclusion, for all the reasons I gave. You can point out that the administrator's hiring preceded a detectable (ie, "significant") change, but that's as far as you can go with such data. In order to conclude that hiring the administrator is a predictor, you will need more than one event in your analysis! $\endgroup$
    – whuber ♦
    Commented Feb 6, 2022 at 17:09
  • $\begingroup$ I apologize for the confusing wording. My dataset includes 50 examples of a new administrator joining a given school, and the student population numbers for the years before and after. Then I have characteristics of each administrator and different types of students. $\endgroup$
    – Nomi
    Commented Feb 7, 2022 at 20:54
  • $\begingroup$ That's much better! Please consider editing your post to make your situation better known to readers. $\endgroup$
    – whuber ♦
    Commented Feb 7, 2022 at 21:03

1 Answer 1

Reset to default
1
$\begingroup$

The following is my personal opinion relating to how to answer your question. It is not meant to disparage or suggest it is singularly better than your or other suggestions, which may present interesting information content. It does, however, recommend for validation, a simulation exercise that can conveniently be performed in a spreadsheet environment.

My approach is actually analytically more simple, but does require data input from schools that one would consider as your peers. It focuses on the relative change in enrollment (a factor of interest) for your school versus the peer group across time. In essence, the data as such can be viewed as self-adjusted for events such as normal time trend, pandemics, economic up/down turns and such events likely to impair the goal of performing an accurate unbiased quantitative assessment of a newly appointed school administrator contribution to enrollment/retention across time.

Next, depending on the amount of data, fit a parsimonious model with, for example, a simple dummy variable. If the question relates to female vs. male, the peer group could perhaps be so grouped, and perform a test of this select group results vs. your school administration gender. Just keep the model selection/estimation process simple.

I am wary of expanding a model with limited underlying data including, for example, even a moderator variable specification. To verify my assertion, I recommend constructing a database with a precisely known specified generating model + noise term and, upon repeated simulations, see how good a selected estimation/testing methodology is performing given data constraints and noise level.

I hope this helps.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi Ajkoer, this is an excellent point. We do have data of average student change year by year in schools of the same type. Can you specify what technique or model can be used to a) validate that the schools in question saw significant student population change and b) run a test on a given variable such as gender? I'm wondering how I should use my data, as I have student totals from 3 years before and after the change in administration $\endgroup$
    – Nomi
    Commented Feb 6, 2022 at 17:05
  • $\begingroup$ I would recommend proceeding with the simulation exercise 1st, and try out naive models first and move up to other models with software available to you based on the simulated data. The model that is most intuitive (for explaining also to others), fulfills your goals and is verifiably accurate i(at least in a specified assumed generating model) is the one I would select. $\endgroup$
    – AJKOER
    Commented Feb 7, 2022 at 3:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.