I am a UK Clinical Psychology Trainee carrying out a small (5000 words) service improvement project based on where I am on placement. The title of the project is: “Quantifying and comparing uptake of CBT therapy for clients with psychosis in four complex needs teams”. The aim of the project is to see what percentage of clients are receiving 16 sessions or more of CBT for psychosis as per National Institute for Clinical Excellence (NICE) guidelines. After analysing the data I created 6 categories of therapeutic delivery: 1 received 16 or more sessions 2 received < 16 sessions 3 Therapy Ongoing 4 Intervention declined by client or service 5 No intervention offered 6 Family Therapy The descriptive analysis tables shows the percentage of each team’s clients who have received each category of therapy per team. So for example in team 1, 60% of clients received CBT>16 sessions, team 2: 40%, team 3: 50%, team 4: 10%. So the team is the independent variable and % of uptake of CBT > 16 is dependent variable. I now need a statistical test to tell me if there are significant differences between the scores for the four teams and if so where they are. I assume I will run the statistical test for each of the 6 categories. Initially at the proposal stage, my supervisor suggested Chi squared as I don’t need to know the degree of significance but given that there are 4 teams is this possible to do? Also I have no idea what expected results would be? It is not like throwing a dice where this can can be calculated. From my reading it seems one way ANOVA plus post hoc tests (not sure which one) is what is required but is this possible with percentages. I came across your forum when I googled percentages and ANOVA and there was some discussion whether the dependent variable was binary or proportion. I am not sure what the dv is in this case...Any advice appreciated!
1 Answer
You're looking at so many comparisons there once you adjust for multiplicity, your cut off for significance is going to be tiny. In my very humble opinion, it also weakens the study to have so many categories, even though I can understand clinically why they would be of interest. I would suggest you just do a z test for proportions to compare the percent of each team that actually achieved the >16 sessions (don't forget to adjust for multiplicity) (and you can only do z test if npq>5, otherwise do chi square). It also begs the question- is there a reason you're interested in comparing your teams to each other? Might it not be more meaningful to categorize the outcome as either met guideline recommendations or didn't and do logistic regression? If you have data collected on each team, you might want to run a logistic regression model to determine predictors of meeting the NICE guideline recommendations for CBT. Eg practice volume, years in practice of MRP, patient SES etc, as this will inform what modifiable factors exist in teams that either increase or decrease guideline uptake. Unless you implemented educational interventions to improve uptake of CBT and are trying to compare groups? Then you could do logistic regression but include group as a categorical predictor to see if team/group was associated with the outcome of interest. I too am more clinician than statistician so take this with a grain of salt. Just my two cents! I'm sure many more qualified people here will have more to offer.
