# linear regression alternative in machining learning

I have a dataset with y1 and x1-x90 and up to 16 to 32 observations based on groups. all features are continuous, x1-x90 are on the same scale (i.e. same measurement units). I want to train a model that can predict future ys with a good performance(?)

I know my observation are too few, but I need to find a way to be able to predict future ys with a reasonable accuracy.

I know linear regression is not working because I have more features than observations, I even tried to average out x1-x90 to get a single X as a predictor or select top n highly uncorrelated features as predictors. None gives good results.

I even tried Partial Least Square Regression (Very bad performance). Tried PCA to reduce dimensionality as well.

I need your help and ideas, what are you suggesting to do for such a dataset? I highly appreciate your input.

Dataset 1: The correlations between y and x1 to x90 (nrow = 20) is as follow:

 [1]  0.0079539241 -0.0279275943 -0.1114396676 -0.1590408236 -0.0677582175
[6]  0.3198357522  0.5550218821  0.4207827747  0.3337311745  0.0880328640
[11]  0.0582322069  0.1329527795  0.5249859691  0.6119430065  0.4040458798
[16]  0.5197256804  0.2394168079  0.0328372270  0.1350753903  0.4226841331
[21]  0.4601538777  0.3012250364  0.3581069708  0.0180357601 -0.0334616899
[26]  0.0943980515  0.3850728273  0.3699024916  0.1878492087  0.3077335060
[31] -0.0061943345 -0.0691352487  0.0579523742  0.3800838888  0.4198089540
[36]  0.1880936623  0.3462918103  0.0414635949 -0.0384607017  0.0588478819
[41]  0.4183586836  0.4522091150  0.2062807828  0.3634050786 -0.0160356686
[46] -0.1462596953  0.0751051903  0.5285830498  0.7419052124  0.4810313284
[51]  0.5592122674 -0.0237595476  0.1189743504  0.2352182865  0.3201770782
[56]  0.4368945062  0.3130523562  0.2522094250  0.1986370236 -0.0941022485
[61]  0.0003422847  0.4270433486  0.6329046488  0.2996157408  0.2382975817
[66] -0.1419960111 -0.0503439382  0.1332624555  0.3348115087  0.4208407998
[71]  0.1538611650  0.1823946089 -0.1148024052  0.0137165384  0.1134363785
[76]  0.4747509956  0.5021457076  0.1080847234  0.0540098026 -0.1391987801
[81]  0.0561727136  0.1322503537  0.4147768617  0.2781975567  0.0748434141
[86]  0.1748948395 -0.0827810317  0.0202686433  0.1201836467  0.4095821083


Dataset 2: The correlations between y and x1 to x90 (nrow = 20) is as follow:

[1] -0.217111468 -0.288009048 -0.179295585  0.079473615  0.314476699
[6]  0.584163606  0.441214234  0.276073635  0.201611370  0.151737019
[11]  0.523968101  0.619692445  0.715748668  0.436169446  0.229508668
[16]  0.222058058  0.366066575  0.465534329  0.655115664  0.693607986
[21]  0.448599219  0.290985614  0.211018473  0.071099617  0.344351411
[26]  0.526696444  0.576362967  0.364139080  0.202315256  0.237566277
[31] -0.015099751  0.246654257  0.419696361  0.590263247  0.324345291
[36]  0.164461091  0.339820623  0.157946825  0.237868398  0.385671616
[41]  0.425716549  0.098423198 -0.056284141  0.148104876  0.163983539
[46]  0.218968794  0.383379936  0.421786606  0.082801215 -0.146509498
[51]  0.155663043  0.213395104  0.523463070  0.570869625  0.436641365
[56]  0.185691327 -0.218298882 -0.110195361  0.050366189 -0.064592123
[61]  0.039960079  0.281514674  0.119713552 -0.123772383  0.313329190
[66]  0.093149640  0.200279668  0.353609443  0.384276360 -0.081244007
[71] -0.289093107  0.061332718  0.025393009  0.324227542  0.384287328
[76]  0.396609217 -0.237784714 -0.467857599 -0.206880033 -0.188748807
[81]  0.143861279  0.243665114  0.062525459 -0.266858548 -0.323563337
[86] -0.003926389 -0.113620020  0.028944774  0.227235451  0.273633987


I have many datasets like that. I cannot combine the dataset because they represent very different groups and it has to be treated separately.

• I have a few ideas, but first a few clarification questions. You say that you have "up to" 100 observations. What does that mean? Exactly how many observations do you have? I mention this, because later you say you have "more features than observations," but it looks like this is not necessarily true: You have 90 features and 100 observations. This is still problematic, but I want to make sure I understand the question. Lastly, would you be able to post a correlation matrix between all 91 variables (e.g., all x variables + the y variable)? Or perhaps dput the entire data set? – Mark White Jan 15 '18 at 15:33
• Thanks, @MarkWhite. I have added correlations between y and x1 to x90 for two different datasets. I made a mistyping with observation number. I have 16-32 rows based on the dataset. – Zmnako Awrahman Jan 17 '18 at 8:27