Timeline for If the categorical variable is retained in my final model in R, then why does the post hoc analysis say the levels do not differ?
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 29 at 18:12 | comment | added | Alan | @SextusEmpiricus sorry for taking so long to reply, I have been very busy these days...but thank you for your help! I'm still learning many things about statistics and data analysis | |
| Oct 21 at 22:08 | comment | added | Sextus Empiricus | I remember some other posts here about situations where n=1 and questions about whether the results are 'usefull'. Some better examples than my smog example where given. | |
| Oct 21 at 22:04 | comment | added | Sextus Empiricus | Also, you could have theoretical considerations to estimate the error in your results. By how much do the rivers typically variate? Example/analogy: If you measure smog in a city and in a village, do you really need to test multiple cities and multiple villages? The statistical test will estimate the variance based on the samples and requires a large amount to make a good estimate of the error. Possibly many people would already be happy with a small sample and consider the random effects based on theoretical estimates or considerations. | |
| Oct 21 at 21:56 | comment | added | Sextus Empiricus | @Alan based on 6 rivers, and only two per each factor level, it is difficult to generalise to other rivers. What you could do is simply make a table or plot of the results for the six rivers (possibly without random effects), and commentate on those results with a warning about interpreting the results as conclusive because of the low significance (with 6 rivers you cannot generalise and you need to be open for the possibility that your results are plain coincidence). Possibly there is more data elsewhere, and people can combine results, or use your data for priors in new research. | |
| Oct 21 at 21:34 | comment | added | Alan | You guys, sorry for taking so long to reply, I've been very busy lately...Well, in my project I could only work with 6 rivers, we couldn't find more rivers that met our requirements for the study, unfortunately. So, if I look into emmeans() documentation and don't find alternative ways of computing the degrees of freedom, do you think it would be acceptable to accept the results the way they are? I mean, to write in the paper that the category was important but the levels didn't differ? I've once heard that such cases do occur. Thank you very much for your help! | |
| Oct 17 at 17:50 | comment | added | Aaron - mostly inactive |
The lmerTest package is worth looking into, I believe it uses Kenward-Roger df in its ANOVA table. But do consider Greg's last paragraph about looking at the size of the effects first.
|
|
| Oct 16 at 16:27 | comment | added | Greg Snow | @Alan, I don't have a quick fix for you. But from looking at the summary I would still say that the degrees of freedom for the Chi-square distribution are too high (infinity) and the values less than 3 are probably lower than needed. You should look through the vignettes and documentation for emmeans to find alternative ways of computing the degrees of freedom that better capture the information that you have. | |
| Oct 16 at 11:58 | comment | added | Alan | Sorry, I had forgotten the summary. I just edited my question and included it in it. The variability between rivers is slightly smaller than the variability within them. How can this impact the post hoc test? | |
| Oct 15 at 15:19 | history | answered | Greg Snow | CC BY-SA 4.0 |