4
$\begingroup$

i know statsmodel library in Python and in R, lowess and loess functions are available for this but i have a few problems with them:

1- i can't seem to be able to make predictions on new data for either

2- it doesn't seem to support a feature space grater than 1

is there anyway to implement Locally Weighted Linear Regression without these problems? (preferably in Python)

UPDATE: according to @xavier-bourret-sicotte's answer i used Alexandre Gramfort's implementation for this but still it doesn't seem that i can predict on new unseen data (test set)? is there anyway to implement this in the context of sklearn so we can use predict method?

$\endgroup$
  • 2
    $\begingroup$ Why don't you fit a generalized additive model (GAM)? $\endgroup$ – Roland Jul 24 '18 at 6:05
  • $\begingroup$ thank you, GAM seems to be a very good alternative, but still its peculiar that for lowess algorithm which at least in theory seems like a very effective model there is not one complete and comprehensive implementation available for it. $\endgroup$ – john d Jul 24 '18 at 12:22
  • 1
    $\begingroup$ Re your comment on sklearn - it is not implemented in sklearn and i remember seeing a discussion on github which said it wouldn't be added to future releases - your options are to implement yourself or use R for example $\endgroup$ – Xavier Bourret Sicotte Aug 1 '18 at 7:45
5
$\begingroup$

Is there anyway to implement Locally Weighted Linear Regression without these problems? (preferably in Python)

Yes, you can use Alexandre Gramfort's implementation - available on his Github page. (Alexandre is a core developer of Sklearn)

You can also have a look a this blog post which shows the implementation on a toy example as well as the maths behind the vectorized implementation

enter image description here

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ thank you so much, really helpful links especially your blog post was gave great intuition and examples but when i tried to implement this i came across some problems. i updated my question i would really appreciate it if you can help with the problems. $\endgroup$ – john d Jul 26 '18 at 14:22
  • 1
    $\begingroup$ @johnd sure no problem - however stats.stackexchange is not the right place to ask for code review or bug explanations and your question as it stands will most likely be closed - could please revert to your previous question ? If you need help with this particular error please post a question on stackoverflow or a code review website - Also if you are satisfied with my answer please feel free to accept it and upvote :) $\endgroup$ – Xavier Bourret Sicotte Jul 26 '18 at 20:56
  • 1
    $\begingroup$ Re your error I think it comes form the fact you are using pandas - Alexandre's implementation will require numpy arrays and won't work with pandas as it involves matrix multiplication and inversion... $\endgroup$ – Xavier Bourret Sicotte Jul 26 '18 at 20:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.