1
$\begingroup$

A simple question that I just want to clarify for understanding: In logistic regression, if we have $n$ input features, will there therefore be $n$ weights used for classification, one for each feature?

EDIT: I wanted to edit this so it will show up if others search. It seems that a more common terminology for what I called weights is "coefficients". What I call "bias" would also be a coefficient, just the intercept one.

$\endgroup$
7
  • $\begingroup$ What would you define as a feature? $\endgroup$
    – Dave
    Commented Apr 26, 2023 at 3:45
  • $\begingroup$ Say we are using height and weight to classify if someone is a male or female. I would say a feature is height and weight, so there are 2 features. $\endgroup$ Commented Apr 26, 2023 at 3:48
  • $\begingroup$ What about ann intercept? $\endgroup$
    – Dave
    Commented Apr 26, 2023 at 3:49
  • $\begingroup$ I guess I forgot to include that as a weight, the way my notes are set up calls that a "bias" term $\endgroup$ Commented Apr 26, 2023 at 3:50
  • $\begingroup$ i have not come across the term "weight" in this context, but if it means "regression coefficient", then there will be $n+1$ weights, unless you have chosen to exclude the intercept. note that a feature which is $p$-categorical will actually result in $p-1$ features when turned wide form via 1-hot encoding $\endgroup$
    – Alex J
    Commented Apr 26, 2023 at 4:34

1 Answer 1

1
$\begingroup$

In a standard application of logistic regression, you have one weight (or coefficient) per input feature, plus one intercept for the whole model (in ML sometimes referred to as the "bias term"). So the number of weights = number of inputs + 1. Technically an intercept is not needed, but exclusion is rare and usually not a good idea.

We can have more features if we include the option of parameter expansion. As a very simple example, if we want to include a quadratic feature for $x$, then the input feature is just $x$, but then we create another term $x^2$ which gets it's own weight, so in a sense we have two weights that came from one input feature. Other examples of parameter expansion include interaction effects (features $x, y$ expanded to $x, y, x \times y$) and splines (piecewise continuous polynomials).

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.