Getting ecological statistics right: Learning by yourself

Question

I'm not really asking for reference on ecological statistics (such as here or here), but more on how you can be sure you're doing a new analysis you just learned correctly. I'm often stuck trying to learn modelling techniques by myself without a mentor simply because I know a bit more stats than everyone else and how to use R.

That being said, it doesn't mean I know everything. I know the stats that I've used before, and not necessarily the stats that's required for the current project; much of the stats I learned were from formal coursework (post-grad, workshops, short-courses, undergrad modules, etc.). I can read all the books and go through all the exercises by myself, but I can never be sure if I'm doing the analyses for the project correctly or not.

How do you know if you're doing the analysis correctly? Is there a workflow or formula you use for your learning so you always know you're on the right track? Who do you reach out for help? How much reading do you do before you say "Right, I really have no idea what I'm doing and I'm just going to give up."?

Answer 1

I find myself in a similar shoe at times. My first thought on how to check yourself is to see if you can recreate your results using another statistical software or (groan) by hand. For example, one of my professors who is definitely more knowledgeable than myself in ecological statistics uses programs like SPSS and Canoco where I typically stick to R. He is often my goto for questions where I am unsure if I have set something up correctly.

It may be worth checking into whether your institution can give you access to a program (eg. SPSS) and then compare results from your R output with that. To me, that is helpful when you are unsure of your R code, especially when trying to learn on your own. The obvious drawback though is that most programs like I mentioned are not free, and this solution is probably not worth it if you have to pay, but again your department/institution may have access to a limited number of installs you could tap into.

Hope that helps, though I am curious to see what others think on this.

Answer 2

This is a hard problem, but one approach that generalizes well is learning how to simulate data. If you are able to generate datasets with different patterns at will, you can apply any new method you encounter to them and see how well they recapture the patterns that you know exist there.