0
$\begingroup$

I am working on a project that predicts the Market Cap (value) of different crypto-currencies. My data is very small (51 observations) and I initially have 18 X-variables. I was hoping to get feedback on my modeling approach and results, and suggestions on improving the model (particularly by transforming variables / with a non-linear regression) technique. I will do my best to keep the post clear and brief, and hope it can be helpful to others working on a similar analysis. The post may seem long, but a lot of it is images. Also, I am doing this in R, and can share any code on request.

1.) My first action was to log-transform the Market Cap (Y-variable) into log(Market Cap). Here are 2 graphs, of Market Cap and of log(Market Cap) of my 50 observations: here we go

...the outlier point, with a $190B market cap, is bitcoin, and the model badly overfit to this data point if I did not log-transform. For this reason, I think this first action of log-transforming the Market Cap is a good action.

2.) 12 of the 50 observations were missing values for 6 of the X-variables. Rather than throwing these observations away, I predict missing values for these 6 X-variables by fitting, for each of these 6 X-variables, a simple linear regression between the (a) the X-variable with missing values, and (b) one of the remaining 12 X-variables with no missing values. For (b), I chose the X-variable with the highest pearson correlation to the missing-value-X-variable. I use this simple linear regression to predict missing values.

3.) I then fit an initial linear model with all 18 variables. However, since 18 variables will almost certainly overfit to 50 data points, and because there is some multicollinearity to the X-variables, I then use stepwise regression with backward elimination to fit a model with fewer variables. The R summary output of the model returned using backward elimination, with 9 variables, is here:

enter image description here

and the R diagnostics plots from this model are:

enter image description here

4.) Lastly, here is a grid with variable distributions and correlations for these 9 predictor variables + the log(Market Cap):

enter image description here

as well as a table of Variance Inflation Factors

enter image description here

My thoughts on next steps are to fix the remaining multicollinearity issue by removing additional variables, and also remove some outlier data points indicated in the Residuals vs. Leverage graph. I also plan to test the model on a test data-set (I did a 40 / 10 split), 5 times using 5-fold cross validation.

I am interested in anyone's thoughts on transforming the X-variables, which I currently do none of. The grid of histograms / correlations / scatter plots shows that many variables are right-skewed. Additionally, I have graphed the simple linear regression fit between log(Market Cap) and each of the 9 remaining variables, and received these plots (for 4 of the 9):

enter image description here

And noticed for the most part that none of the X-variables seem to have a great linear-fit with log(Market Cap).

Any thoughts on next-steps on my model building would be greatly, greatly appreicated. Apologies again for the long post, but I felt that thorough / lots of images / plots would be helpful here. Thanks!

$\endgroup$
1
$\begingroup$

just some quick thoughts:

re Q2: there is a large literature on imputation; i guess you're familiar with it and you have followed some standard. The proportion of missing is quite high however (20% for some variables?) and thus any approach will feel dubious. The estimates should account somehow for the uncertainty of the imputed values.

re Q3: i personally do not like those stepwise methods, they are data-depedendent and your result may be an artefact of that particular dataset. Selection of variables, if possible, could be informed by an understanding of things. Unless of course the purpose of the analysis is to identify key predictors, then selecting a priori may be tautologous

re Q4: i wouldn't transform the X variables. If a large no. of values are clustered at 0 (as in one of your plots) then it is not a great loss of info to categorise this variable, especially if it is meaningful and aids interpretation of the results to do so. So much depends on the context and the purpose of the analysis, and the type of data you're handling is unfamiliar to me

frank harrell has a great stats blog and is an expert on these things: http://www.fharrell.com/ Where he contradicts what i have said: believe him

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ appreciate the input @PaulBrownPhd re Q3: part, but not the entire, of the purpose of the analysis is to identify key predictors. Since the dataset is very small, a model trained on a different training set (based on a random split of the data) will return different predictor variables when that model is them passed into a backward elimination algorithm, and I am aware of this concern re Q4: appreciate your thoughts on this - i will attempt to categorize certain variables and see if it helps the fit. $\endgroup$ – Canovice Jul 19 '18 at 4:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.