# Statistical methods for small samples

Can you please suggest some statistical methods for datasets with few observations (aprox 100 values) that I could research more? I know about bootstrapping but I was wondering what else there is.

To be specific, I have a sample of 20 dimensions but with 100 observations of which 30% I will use as a test set. How can I improve my model training when I have only 70 observations.

• with 20 dimensions and 100 observations there's not much to do. OLS will barely work with a few variables, use shrinkage or PCA to reduce dimensions – Aksakal Sep 28 '17 at 20:37
• yeah, I wanted to use Bayesian or GP regression but as I said the sample is very small. So, I am not sure if it will work. In general, as I come from an engineering background I am not very familiar with statistics, so I was wandering what similar methods to bootstrap exist. That allow you to perform inference in small samples. – Jespar Sep 28 '17 at 20:39
• If your data is from physical world, then you have a chance. There's quite a bit that can be done with 100 observations and a few variables. 20 vars is a stretch unless the process is very stable, which happens a lot in physical world – Aksakal Sep 28 '17 at 20:41
• Actually, I am playing with financial/macro data from free resources (central banks, OECD, IMF, indexes) which are monthly and they go back only 10 or 20 years. So the max volume of observations that I can get are about 300. Which I still consider few. – Jespar Sep 28 '17 at 20:44

## 1 Answer

Look at econometrics. It deals with economic data, which is usually very scarce. For instance, GDP is quarterly, inflation is monthly etc. So, there's a lot of methods to analyze this data. OLS and time series are the most common. However, with 20 variables you really need a lot of data. Moreover, when you collect this data by expanding the time span the question is whether it is still relevant.

As it is stated now your question is too broad. For instance, Greene's Econometrics, 7th ed is 1000 pages or so long, and it's all about the subject of your question. You need to narrow down your question to a manageable scope. For instance, sometimes you can run PCA to reduce he dimension of your dataset, so that OLS will work on small data set. PCA works very well in many cases, such as for sets of variables of similar type, like interest rates.