3
$\begingroup$

I am working in statsmodels and I am trying to do linear quantile regression with the function QuantReg. I can however not figure out a way to tune any hyperparameters, to avoid overfitting, such as regularization.

An example is given below:

df_dum = pd.DataFrame(columns=['y','x1','x2'])
df_dum['y'] = np.random.normal(size=10)
df_dum['x1'] = np.random.normal(size=10)+2
df_dum['x2'] = np.random.normal(size=10)+10

df_train, df_test = np.split(df_dum, [int(0.8*len(df_dum))])

mod = QuantReg(df_train['y'], df_train[['x1','x2']]).fit(0.25)
mod.predict(df_test[['x1','x2']])

Any idea how to tune hyperparameters for this dummy example?

$\endgroup$
2
  • $\begingroup$ if it's linear, then why are you afraid of overwriting? $\endgroup$
    – rep_ho
    Jun 4, 2021 at 8:33
  • 2
    $\begingroup$ In normal linear regression, I would use regularization to prevent overfitting. So I am assuming that the same is applicable when doing quantile regression. $\endgroup$
    – andKaae
    Jun 4, 2021 at 8:54

1 Answer 1

2
$\begingroup$

quantile regression in statsmodels does not support regularization, but sklearn does https://scikit-learn.org/dev/modules/generated/sklearn.linear_model.QuantileRegressor.html

regularization by augmentation of the dataset is also an option, although there might be some specific consideration for quantile regression.

if your question is how to fit a model that is predicting quantiles well, than the answer might be different than just use scikit-learn

$\endgroup$
3
  • 1
    $\begingroup$ I cannot seem to get QuantileRegressor to work in sklearn, which version are you running in which it works? $\endgroup$
    – andKaae
    Jun 5, 2021 at 15:51
  • $\begingroup$ i never run it myself, i just found it in their documentation, i generally assume that sklearn things work. It says that it is from a development version of sklearn, so you will need to install that. I can't help you more with that. I think stack overflow is a better venue for questions about specific libraries $\endgroup$
    – rep_ho
    Jun 6, 2021 at 11:01
  • 1
    $\begingroup$ The sklearn implementation is very slow and RAM intensive compared to the statsmodel version. Sadly it seems the statsmodel version does not come with regularization. $\endgroup$ Feb 10, 2023 at 0:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.