3
$\begingroup$

I have a dataset, and i have to predict the flow of users at a certain city given some information like the day of the week, the month, the distance of the city of origin ecc..

First i decided to plot the heatmap of correlation, to see if there are correlations between the features, and this is the result:

enter image description here

As we can see there's no much correlations between the features.

I have done Linear regression obtaining very bad results (R^2 = 0.1).

I have done Lasso Regression in order to drop the bad features but the best result for Lasso is given by lambda=0, so the best result is using all the features.

My question is, is it possibile that the dataset is very bad and it's not a problem of linear regression tool? Are there other techniques in order to understand if there is a better model? I'm trying to understand why Linear Regression performs so bad.

OK i plotted the features with respect to the label and i think that the problem is the dataset.enter image description here The plot with the green X are the features i decided to drop, obtaining an average training error of 4200 (against the 22000 of before). Honestly i don't know what to do now.

Improve this question
$\endgroup$
2
  • $\begingroup$ of course it is possible that a simple linear regression is a bad model to describe whats going on; this has not necessarily to be a data problem but could instead be a modelling problem. $\endgroup$
    – BloXX
    Commented Feb 21, 2019 at 17:03
  • 3
    $\begingroup$ I'm afraid I'm unable to see "there's no much correlations" because the meanings of the colors are undefined. I would expect the darkest purple would be close to $-1,$ in which case evidently some features are strongly correlated. However, this has nothing whatsoever to do with $R^2,$ which is a measure of how these features are related to the response. For a better understanding, consider drawing scatterplots of the response against the regressors so you can see how they might be related. $\endgroup$
    – whuber ♦
    Commented Feb 21, 2019 at 17:07

1 Answer 1

Reset to default
7
$\begingroup$

There are lots of reasons why linear regression may perform "so bad". A linear regression model may in fact be appropriate but there is a lot of noise in the data. In other words, the explanatory variables that you have simply don't explain enough of the variation in the response. There may be non-linear associations, which could be modelled with linear model (by including non-linear terms in the model or by using an additive model) - alternatively a non-linear model may be more appropriate. There may be interactions among the explanatory variables.

To investigate further, you could plot the response variable against each of the explanatory variables in turn - this may indicate non-linearities, or indeed confirm that the linear model might be appropriate.

Also, before throwing away the model on the basis of $R^2$ (which is generally not a good thing to do) you should perform the usual regression diagnostics such as inspecting residual plots.

Improve this answer
$\endgroup$
4
  • $\begingroup$ How can i understand if for example i compute the MSE, if is it good? For example i obtain a MSE of 2000. How can i say that is good or bad? How can i define the ranges of the MSE given my dataset in order to have a better idea? $\endgroup$
    – FraMan
    Commented Feb 22, 2019 at 16:49
  • $\begingroup$ @FraMan You can't. You can use MSE for things like comparing different models running on the same dataset, but you can't evaluate MSE in isolation, for one model, on one dataset. $\endgroup$
    – Robert Long
    Commented Feb 22, 2019 at 16:57
  • $\begingroup$ Do you know how i can evaluate if my model have done the best possible work based on the dataset? For example, i have a bad dataset and using different method i obtain different results. How can i understand if those results are the best achievable, or if it is possible to do better? $\endgroup$
    – FraMan
    Commented Feb 22, 2019 at 19:17
  • $\begingroup$ @FraMan You can investigate the fit of your linear model with the usual regression diagnostics. This may inform how to improve it (e.g. non linearities). You could also fit other types of models such a tree-based algorithms and compare how they fit. $\endgroup$
    – Robert Long
    Commented Feb 23, 2019 at 8:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.