Tagged Questions

26 views

Use fitted value from regression on subset of features as independent variable

I am working with a relatively large data set with 2K columns and many variables can be grouped together (a logistic regression). So I am thinking can I use fitted value from regression on subset of ...
18 views

Feature boosting via rescaling in logistic regression and linear SVMs

If I were expressing a problem in terms of binary features, all encoded as {0,1}, could I boost some features by encoding them as {0,2}? Would the effect change based on whether I used either of the ...
24 views

How to deal with categorical features.

Recently I am playing in the famous Big-Data website Kaggle. There is a Display Advertising Challenge. In this competition, you are giving a training file which include huge records. the records is ...
98 views

Interpreting conflicting results from Random Forest & Logistic Regression?

I am using SKLearn and Statsmodel in python to build a RF and Logistic Regression, respectively. I have a feature that the RF indicates is important (feature importance of 0.202, closely behind #1 ...
29 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
322 views

How to select a subset of variables from my original long list in order to perform logistic regression analysis?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
130 views

Interpreting the lasso coefficients

I have used lasso logistic regression on some data and I have some non zero coefficients for some of the features. I want to know based upon the coefficient values how do I rank the features?
25 views

Choice of 0 or -1 for failure in the independent variables of a logistic regression

I am performing some exploratory analysis on a dataset where the dependent variable is a dichotomous variable. I have ~10 explanatory variables, some of which are dichotomous observations. I am ...
143 views

Steps followed when Binary logistic regression when both dependent and independent variables are binary

I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which ...
299 views

Exploratory Factor Analysis for Binary Logistic Regression Variable Selection

I have a great interest in learning new methods(at least to me) of variable selection in regards to binary logistic regression when I am working with over 500 potential predictor variables and have ...
276 views

Using Mutual Information for Binary Logistic Regression Variable Selection

In addition to proc varclus, randomForest, and assessing multicollinearity among potential predictor variables, I am seeking ...
222 views

Stepwise versus L2 regularized logistic regression: dataset-specific performance

I have two data sets from different collections. The second data set is smaller. They were both analyzed with the same methods in order to derive feature sets of 10-30 features each. Each feature set ...
2k views

Significance of categorical predictor in logistic regression

I am having trouble interpreting the z values for categorical variables in logistic regression. In the example below I have a categorical variable with 3 classes and according to the z value, CLASS2 ...
423 views

Not all Features Selected by GLMNET Considered Signficant by GLM (Logistic Regression)

I wanted to create a predictive model of mortality after patients had undergone a surgical procedure. But I also wanted to avoid doing what most researchers do by first performing univariate analysis ...
347 views

Variable selection / Dataset reduction for large datasets (in R)

I'm working on a behavoural scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is ...
173 views

Sensible to include ratio as a variable in logistic regression?

I'm creating a generalised linear regression using a binomial link function for two variables A and B. From looking at the data it appears that A/B may have discriminatory effect. Is it sensible to ...
182 views

Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?

I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
2k views

Logistic regression performance with high number of predictors

I'm trying to understand the behavior of logistic regression in high dimensional problems (i.e. when you are fitting a logistic regression to data with a high number of predictor variables). Every ...