I've been recently working on the following problem:

Let $F = \{F_1, F_2,F_3\}$ denote a set of feature sets. For example, $F_1$ is comprised of 100 actual features. Before training a logistic regression classifier, I re-weight the individual feature sets (re-scaled between 0 and 1) as follows:

$$F_x = w_x * F_x,$$

i.e., by multiplying each feature set with a weight between 0 and 1.

My question is the following:

Assuming I get (for the aforementioned example) weights: $\mathcal{W} = \{w_1 = 0.2, w_2 = 0.9,w_3 = 0.000004\}$ (the index corresponds to an individual feature set from $F$).

Can I interpret this in the lines of: "The second feature set (2) contributes the most to the learning" etc.?

With other words: How sensitive is logistic regression to such changes in feature values.



Current experiments suggest LR is indeed susceptible to such feature re-weightings.

Further, there is the trivial case when a given $w = 0$, as here no signal is present.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.