1
$\begingroup$

I asked this question last night and Matt Krause explanation helped me a lot. (For more explanation please see my previous question). Now I have another problem. We are using RapidMiner studio and logistic regression classifier to train our model. And here is our new model and results :

enter image description here

weights:

enter image description here

I have two simple question:

  1. What dose negative weights mean? Is it okay, or something is wrong?
  2. How should we use this weights to predict new records? Consider that we are using logistic regression classifier. (For example a C# application to summarize news)

Edit 1
Here is RapidMiner logistic regression doc. Logistic Regression

$\endgroup$
  • $\begingroup$ You should take a look at RapidMiner documentation for help with interpreting the model output. It appears that these "weights" correspond to the coefficients for the predictors in the model, but I can't be sure about that without looking at their documentation. $\endgroup$ – Vishal Apr 5 '16 at 20:16
  • $\begingroup$ @Vishal docs.rapidminer.com/studio/operators/modeling/predictive/… thanks $\endgroup$ – Oli Apr 6 '16 at 3:47
2
$\begingroup$
  1. The negative weights imply that these attributes have negative impact on the outcome, after controlling for all other attributes that are in the model. For example, all else being equal, as SimilarityToContent increases (from 0 to 1) the likelihood of the statement being true decreases. Interestingly, there's only one attribute in the model that has a positive coefficient. As SimilarityToTitle increase, the likelihood of a statement being true also increases. All other attributes lower the likelihood of true sentence.
  2. Here's the formula that you can use to predict the probability that a sentence is true:

$$ p\ (statement=true) = 1 / (1 + exp(-z)) $$ $$ Where \ z = Constant -0.598 * SimilarityToContent - 0.113 * StopWordsEffect - 0.125 * SntcLengthEffect - 0.742 * SntcPositionEffect - 0.030 * CuePhrase + 0.022 * SimilarityToTitle $$

Note that the attribute with a zero coefficient is excluded from this formula.

Update: Although the intercept (constant) term is not shown in the sample output in the original question, it was added in the formula above based on feedback to this answer.

$\endgroup$
  • $\begingroup$ thanks for your nice explanation, we balanced our dataset with Weka and smote algorithm, and relearned the model with Weka tools. now we have an intercept value , how should we include it in above formula ? $\endgroup$ – Oli Apr 7 '16 at 4:38
  • $\begingroup$ Glad you found it useful. I've updated the equation in my answer above to include the intercept (constant) term. $\endgroup$ – Vishal Apr 7 '16 at 13:10
1
$\begingroup$

I am assuming the data is labeled +1 and -1 and the underlying model is linear logistic regression for classification.

It looks like 7 features are extracted from data (as listed in the "weights" table). During the training, 7 model coefficients will be learned. I assume that it is standard L2-norm regularization with some parameter set by user or set by default. The regularization parameter deserves a special discussion and will be left out of scope.

The learned model (the coefficients) are completely dependent on feature values and the labels. I am expressing my guess here.

It appears from almost all the learned model coefficients (except SimilarityToTitle and EsharePhrase) has more weight toward predicting the negative class. The only feature "SimilarityToTitle" relatively little contributes to decision in favor for the class +1. The feature "EsharePhrase" has no impact at all as it is 0.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.