0
$\begingroup$

This is a school exercise and I just don't get on the right track.

Data description:

  • +- 1000 Samples test data.
  • 1 Y column.
  • 15 X columns.
  • Y mean: ~6 max: 341 min: -221
  • X: mean of each column around +- 1 (0.9, 1.05, 0.99 etc.). max over all X: 4.7 min over all X: -2.7

The target rsme should be around 15 but I just can't get below 20. I used every tool every library there is. Things I tried with sklearn:
Models: LinearRegression,KernelRegression, KernelRidge, BayesianRidge.
Feature selection: PolynomialFeatures up to degree 5 but everything over 2 mostly the rsme gets worse. from 1 to degree 2 is the most significant reduction in rsme visible.

A few plots: Y plot Sorted Y plot mby more interesting?

The one below is sorted for Y

I tried to post more pictures below of the x y relations but I can't here because I'm only allowed to post 2.

I should mention that we had only linear regression already and it should be kinda solvable with that they told.

Improve this question
$\endgroup$
2
  • $\begingroup$ This is a question that requires the self-study tag. $\endgroup$
    – Michael R. Chernick
    Mar 30, 2017 at 12:44
  • $\begingroup$ did you try removing variables from the model that are not well correlated with y? did you try adding interaction terms (e.g. x1*x2)? $\endgroup$
    – tea_pea
    Mar 30, 2017 at 17:27

2 Answers 2

Reset to default
1
$\begingroup$

Look closely to the distribution of your data. The mean is around 6 while the max and min values are 341 and -221 respectively. This suggests that there a number of outliers in your dataset and nothing is more worse for a regression problem than outliers. So, you need to remove the outliers first. Then you should standardize the data values by using a scaler. After these, you should try a regression model.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hi. Thanks for your answer. I used the Scaler class from sklearn to Scale my X. And for the outliners i trained my model then removed the top 10% of wrongest predicted X and trained again. should i remove the outliners before i train? $\endgroup$
    – Sim
    Mar 30, 2017 at 12:41
  • $\begingroup$ Yes you can remove all the outliers in the way you are doing but I find Tukey's method more reliable. Remove outliers using Tukey's method before training $\endgroup$
    – enterML
    Mar 30, 2017 at 17:14
  • $\begingroup$ @Sim did you end up getting a better result after removing outliers? $\endgroup$
    – AaronDefazio
    Apr 3, 2017 at 1:16
0
$\begingroup$

Here are some more pictures to the relation between x & y enter image description here

enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.