1
$\begingroup$

A simple question that I just want to clarify for understanding: In logistic regression, if we have $n$ input features, will there therefore be $n$ weights used for classification, one for each feature?

EDIT: I wanted to edit this so it will show up if others search. It seems that a more common terminology for what I called weights is "coefficients". What I call "bias" would also be a coefficient, just the intercept one.

Improve this question
$\endgroup$
7
  • $\begingroup$ What would you define as a feature? $\endgroup$
    – Dave
    Apr 26 at 3:45
  • $\begingroup$ Say we are using height and weight to classify if someone is a male or female. I would say a feature is height and weight, so there are 2 features. $\endgroup$
    – Carter Karl Falkenberg
    Apr 26 at 3:48
  • $\begingroup$ What about ann intercept? $\endgroup$
    – Dave
    Apr 26 at 3:49
  • $\begingroup$ I guess I forgot to include that as a weight, the way my notes are set up calls that a "bias" term $\endgroup$
    – Carter Karl Falkenberg
    Apr 26 at 3:50
  • $\begingroup$ i have not come across the term "weight" in this context, but if it means "regression coefficient", then there will be $n+1$ weights, unless you have chosen to exclude the intercept. note that a feature which is $p$-categorical will actually result in $p-1$ features when turned wide form via 1-hot encoding $\endgroup$
    – Alex J
    Apr 26 at 4:34

1 Answer 1

Reset to default
1
$\begingroup$

In a standard application of logistic regression, you have one weight (or coefficient) per input feature, plus one intercept for the whole model (in ML sometimes referred to as the "bias term"). So the number of weights = number of inputs + 1. Technically an intercept is not needed, but exclusion is rare and usually not a good idea.

We can have more features if we include the option of parameter expansion. As a very simple example, if we want to include a quadratic feature for $x$, then the input feature is just $x$, but then we create another term $x^2$ which gets it's own weight, so in a sense we have two weights that came from one input feature. Other examples of parameter expansion include interaction effects (features $x, y$ expanded to $x, y, x \times y$) and splines (piecewise continuous polynomials).

Improve this answer
$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.