1
$\begingroup$

I have a data set of 3000 observations with 9 variables, and I'm trying to predict whether water are safe for drinking. Regular multivariate logistic regression isn't that good at forecasting, and also none of the coefficients is significant, even if I run univariate logistic regression. This is why I thought of regularization, but I wasn't able to found an explanation of this and when it is appropriate to use. Also, if it exists, if be happy for a reference to R functions.

$\endgroup$
5
  • $\begingroup$ Is your measured response variable binary or some measure of contamination (e.g., 6 parts per million coronavirus). $\endgroup$
    – Dave
    Commented Jun 27, 2021 at 19:54
  • $\begingroup$ It is binary: safe or not safe $\endgroup$
    – Ift h
    Commented Jun 27, 2021 at 20:14
  • $\begingroup$ plenty of regularised glm out there i believe. Glmnet is quite popular and has vignette. However, do you have any expectation of what the relationship is between inputs and "safe"eg I could imagine not safe to drink is "legally" defined as chemical 1> conc1 or chemical2 > conc 2 or chemical 3 > conc3. I don't believe you can fit this in a logistic regression (without adding some nonlinearities). $\endgroup$
    – seanv507
    Commented Jun 27, 2021 at 21:10
  • 2
    $\begingroup$ Statistical significance has nothing to do with regularization and forecasting. What doesn’t work about forecasting with logistic regression for you? $\endgroup$
    – Tim
    Commented Jun 27, 2021 at 21:17
  • $\begingroup$ This is mostly an exercise at class. There are all kind of substances and measures like Chloramines and pH levels. The prediction is around 58% accuracy, which is quite poor in such cases, as it is health issues. $\endgroup$
    – Ift h
    Commented Jun 28, 2021 at 14:24

1 Answer 1

1
$\begingroup$

Regularisation aims at reducing the effects of design matrix being overdetermined or underdetermined, recall solving $Ax=b$, $A \in \mathbb{R}^{m \times p}$. Regularisation is appropriate to use if $p>>m$ (underdetermined) or $p<<m$ (overdetermined). Here the case is $m>>p$, overdetermined (m=3000, p=9 in this case).

Using LASSO or elastic net regularisation are recommended instead of plain logistic regression. Without regularisation, solution may not be correct. glmnet's introduction will give a good idea how to use LASSO and elastic-net regularisations.

See also rank deficiency.

$\endgroup$
2
  • $\begingroup$ Agreed on glmnet—and just to add on, for a general background into what the LASSO does conceptually, I recommend Introduction to Statistical Learning by James et al. The R labs are outdated, but the concepts are explained in an intuitive and accessible way. $\endgroup$
    – Mark White
    Commented Jun 27, 2021 at 23:00
  • $\begingroup$ It’s just been updated with a new edition. Haven’t checked the R labs but presumably they’ve been updated, too. $\endgroup$
    – Mooks
    Commented Sep 6, 2021 at 16:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.