I have a small dataset of 37 observations with students' performance on both cognitive tests (5) and professional tests (6). My goal is to predict professional tests (DV) with cognitive tests(IV). To summarize, the characteristics of my data are:
- A very small dataset
- DV close to being normally distributed (W-S test) while IV don't have any clear form
- Both dependent and independent variables are limited to the range of [0,1]
- Multicollinearity between DV is very low
I would like to ask your advice regarding the following:
1 What is the most suitable regression model in my case? - I tried GLM and thought of Tobit model, but it doesn't assume the limitation over independent variables. Also are there models taking into account the bias in the distribution of DV (i.e., median of each Yi>>0.5)?
2 What is the policy regarding outliers when it comes to analysis of small N?- I tried to use Cook's distance and boxplot but it results with expensively big number of outliers.
3 What is the best way to train a model having small sample size? - I'm particularly interested in obtaining model coefficients and, therefore, would like to have the model coefficients that maximize the prediction power. I have tried to make use of K-fold validation, but it's unclear how to average model coefficients without introducing overfitting. What is the reasonable test set and what is the appropriate setting in my case?
Your response and help will be mostly appreciated