Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am simply wondering how non-professionals practice the statistical techniques they have learned. I am a student and want to implement some of the techniques I am learning but don't know how to approach consistently finding problems that are at my skill level.
Thanks
I cannot stress this enough, look for real data.
If you are using a particular statistical method provided in an R stats package for example, often the package will contain a sample dataset which the authors will demonstrate their methods on. Similarly, if you look in R's datasets, a whole bunch of classic datasets are included.
But if you are looking to practice use of statistical methods, this is only the place to start. Datasets included in an R package are not a great representation of real data analysis for two reasons: first, the data is typically organized and clean. Second, the data is typically cherry-picked to make that particular statistical method look great. While it makes sense for an author to do this (why would you demonstrate your method on data where it is unnecessary?), when using statistics to answer real questions, it's not always clear or obvious which method to use.
As such, while these datasets are a good basic tutorial (just as anyone should start programming with "hello world") to get more real experience, I would suggest going to public repositories. You might be surprised how much data is freely available. Interested in climate modeling? Go to NOAA and pick what you'd like. Interested in income modeling? European Social Survey. I think it's really cool to actually be making real insights while learning a new method.
I do caution that getting and cleaning the data is a considerable amount of work. But that's not an inaccurate representation of what real data analysis is like.
If you want to "get your hands dirty" with statistics, there is an infinite number of ways to mine data, set up hypotheses, experiments, analyze data sets using various styles of analysis (i.e., Bayesian vs. frequentist), etc. However, your question indicates that you're having trouble figuring out where to begin.
Find a topic that interests you, be it sports, politics, science, commerce/economics, behavioral psychology, biology, and so forth. If there are phenomena in said topic that can be quantified, then more likely than not, someone has already dug up massive amounts of data. I highly recommend Nate Silver's blog FiveThirtyEight, (of which I hold no vested interest/position in) where there's a wide variety of topics with small studies and analyses featured. The topics are well-thought out, and the statistics used isn't terribly difficult to understand or grasp for beginners (no offense to Nate Silver). At the very least, you could build upon whatever articles that have been featured, and use the many data references in the articles to do your own analyses or run your own specific tests.
After you have figured out what topic you'd like, specify a particular question you have in said topic- for example, I just thought of "in sports, do high-margin wins correlate to championship titles, or fatigue?"- and then find your data. The resources on the internet are nearly endless, but you must remember that not all data is quality data (i.e., be wary where you find your data, and whether there are any ethical/quality concerns).
Some useful links include DataHub.IO, where you can find (and share!) many free datasets, and Data.gov, a source of all open data that the US GOV shares. If your programming skills are pretty good, I imagine you can also fetch data from the popular social media webpages, i.e. Twitter, Instagram, Facebook, etc.
Don't forget to have some sort of go-to statistical evaluation software. Most (myself included) would recommend the open-software standard, R, but you'd be surprised how far you could go with something like Microsoft Excel, if your data size isn't terribly large and complicated.
Good luck!
The best method in my opinion to practice the skills you have is getting some data (this is the last of your problem, you can find dataset on the internet or you can create your own dataset) and try to figure out if there are relations or difference in some groups of the data (assuming you want to practice with inferential statistics).
For example:
I'm learning regression or machine learning techniques: can i predict the outcome of an event currently evolving? (the oscars? the end of a tournament?)
For inferential statistics, can i analyze some data and find difference between some groups? Associations between variables?
Ans so on. The possibilities are endless
Several of the free (or at least inexpensive) certifications (Coursera, etc.) have small projects that can get you a lot of practice. You can do them quick and dirty, or you can do them in way more detail than is asked for. The advantage over doing it alone is there is a forum where you can discuss your results and often community experts who can offer advice as well. And of course they are well tested with beginners so there are no insurmountable obstacles which can discourage learning.
The Practical Machine Learning from Johns Hopkins comes to mind. But also the Stanford ML course from Andrew Ng is another possibility, and there are many more.