Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
10 views

Feature selection - number of features/levels when having categorical data

Denote a feature set $x_1,...,x_p$ and target variable $y$. Assuming a predictive modeling technique (some type of logit regression) is being used to predict the value of $y$. This algorithm is ...
0
votes
1answer
48 views

Performance of a classifier change heavily

I'm using a data set of 32 face persons and a svm-rbf to classify and a random group of 70% for train and 30% for test. The problem is that my results are heavily dependent of the random group used ...
1
vote
0answers
24 views

features selection - methods based on estimated feature importances vs. methods based on scores

I noticed that all feature selection methods implemented in sklearn are based on external estimator that assigns weights to features, AKA feature_importances. I ...
0
votes
0answers
32 views

variable selection before a decision tree

I want to built a predictive decision tree. I have a dataset with +/- 1000 observations and 1500 variables. Can I just built my decision tree (training + validation dataset) with all the 1500 ...
3
votes
0answers
57 views

How to do dimensionality reduction on a huge data set?

I am working with fMRI data of ~1000 subject. Each subject has a feature vector of ~150 million dimension. So I can only keep the feature vectors of ~10 subjects in memory. What are some algorithms ...
0
votes
0answers
15 views

Properties of data for mutual information as feature selection

I have a dataset $D = \{(x_1, y_1),...(x_n, y_n)\}$, $x$ is a vector while $y$ is a scalar. I want to select a subset of features of $x$. I want to use mutual information as the feature selection ...
0
votes
1answer
32 views

Selecting variables and fitting to bounded response (0,1)

I have a dataset with 15 binary covariates and a continuous response variable bounded between 0 and 1. The binary variables represent correct or incorrect answers on a short test and the response ...
0
votes
0answers
20 views

How do you statistically determine the point when a series becomes stationary (i.e. the y variable saturates)?

I am computing AUC of a model as a function of features in the model. In general it does very well but I wanted to optimize the number of features by looking @ the trends in the AUC. I know the AUC ...
1
vote
1answer
69 views

Using trees after variable selection using Lasso/Random

I am new into Machine Learning so please excuse me if my question is naive. My question is, is it possible to use trees for example rpart or ctree after variable selection procedures such as ...
0
votes
0answers
27 views

ROC change after variable selection with glmnet

I was using glmnet in caret to select important variables. The code is like ...
2
votes
2answers
87 views

Ranking features in logistic regression

I used Logistic Regression. I have six features, I want to know the important features in this classifier that influence the result more than other features. I used Information Gain but it seems that ...
2
votes
1answer
35 views

Feature selection using cross-validation and multiple linear regression

I would like to predict discrete (mean) values from discrete (mean & std) values of the extracted features. My question is how you should perform feature selection when you are applying K-fold ...
0
votes
0answers
15 views

Correct evaluation/ comparison between undercomplete and overcomplete representations

Suppose I'm performing Unsupervised Feature Learning method to learn a representation of the data that is under-complete (e.g. 100 features) and use another algorithm to learn an over-complete ...
2
votes
0answers
51 views

Variable selection with multi-variate time series

I currently have a data.frame with 273 variables and 94 rows: ...
1
vote
1answer
33 views

Categorical with many levels for NN

I have a dataset which lists the amount of seconds a user held a session by browser_type. For example: ...
0
votes
0answers
20 views

How do I have select features which are influential for prediction?

I have a dataset which has dependent variable(label) as possible destinations and independent variable(features) as age,language, gender and many other categorical variables. How do i find which are ...
0
votes
0answers
77 views

What are appropriate feature selection techniques for binary features?

Suppose that we have binary features (+1 and -1 or 0 and 1). We have some well-knows feature selection techniques like Information Gain, ...
2
votes
1answer
81 views

Computing Cross-Validation Errors for Subset Selection: error in standard code in the literature?

I am currently trying to understand how to use cross-validation in order to choose among the "best" subsets of different sizes returned by the R function regsubsets (regsubsets returns the "best" ...
0
votes
1answer
26 views

Computing the Interaction gain. Is there an Error in the infotheo package in R?

In order to implementing a certain feature selection method for a classification problem I need to estimate the the interaction the interaction gain between two features and the target variable which ...
0
votes
1answer
106 views

VAR/VECM/ARDL optimal lag selection

Question 1: Is it necessary to consider AIC and the BIC criteria when selecting the lag for a VAR, VECM or ARDL model OR can I use something else? Example: Can I pick 12 lags because the model simply ...
1
vote
0answers
24 views

Choosing between feature selection and regularization to overcome over-fitting in categorical regression

In order to overcome over-fitting during a regression process over categorical features, one can either 1) Apply L1/L2/Elastic regularization during the regression, for example as answered here ...
0
votes
1answer
13 views

How to check feature relevance and representativeness?

How to check whether features from one domain are relevant in other domain? How to evaluate whether features are representing that domain?
0
votes
0answers
40 views

Logistic regression - variable transformation

I have a continues variable(EntropyDistanceFromMean) which I would like to use in a logistic regression, the problem with that variable is that it starting to effect the output (MQL) found on the ...
0
votes
0answers
13 views

Content Based Document classification

I have a corpus of 10 million resumes. I want to add tags to these resumes like Software Engineer, Data Scientists, ...
1
vote
0answers
39 views

Variable importance in regression with large number of missing values

I have a dataset with multiple (approximately 20) categorical and ordinal predictors and a numerical outcome and I am trying to understand which and how each of these predictors affect the outcome ...
1
vote
0answers
35 views

grouping attributes in RF and GBM

i have a dataset with 1000 samples and ~11k features (SNP markers). i have identified 100 additional binary features describing the markers themselves so i have a ...
1
vote
1answer
119 views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
0
votes
1answer
40 views

Is feature complementarity different from feature interaction?

I am writing a conference paper in which I have a sentence like "...complementary/interactive features...". This sentence ...
2
votes
1answer
61 views

What happens to multi-category variables in algorithms like Random Forest that sample the feature space?

Suppose I have a multi-level categorical variable like color (say, with 7 levels). Some software libraries only allow numeric matrices to train models, so we need ...
0
votes
1answer
49 views

In regression, what is the limit of independent variables?

After having taken the Coursera Data Science specialization, I am faced with my first "practical" problem which I plan on solving with some sort of regression. This is my first real world, ...
1
vote
2answers
62 views

What does “the process that generates the data” mean? and How does feature selection help in recovering it?

In [1], one of the motivations to use feature selection is stated to be: "to gain knowledge about the process that generated the data". What does this "process" actually mean? and How does feature ...
3
votes
2answers
123 views

random forests for optimal variable selection/feature selection

Gurus, I just came across this tutorial (http://blog.datadive.net/selecting-good-features-part-iii-random-forests/) about using "random forests" for optimal variable selection/feature selection. The ...
0
votes
0answers
61 views

Feature selection using RFE in SVM kernel (other than linear eg rbf, poly etc)

At this link, there is an example of finding feature ranking using RFE in SVM linear kernel. If I want to check feature ranking in other SVM kernel (eg. rbf, poly etc).How to do it? I have changed ...
0
votes
1answer
68 views

Split dataset by categorical variable or use as a dummy/factor variable?

I'm looking for any sort of best practice or ways to go about this situation. Often I come across datasets that have a categorical variable that I am tempted to split off the main dataset into ...
0
votes
2answers
71 views

is linear regression/polynomial regression sensitive to irrelevant features/noise

is linear regression/polynomial regression sensitive to irrelevant features/noise will their respective weights/coefficients be automatically be tuned down? or is it a straight nail in the coffin? ...
0
votes
0answers
29 views

Analysis of wrapper feature selection ouptput in Weka

I am using Weka to select important features from a dataset. I am using the wrapper method in this application. I chose a decision tree (j.48) for my classifier and Genetic search for the search ...
3
votes
2answers
132 views

Cross validated penalized logistic regression - one standard deviation rule

I am new to this topic and would like to understand it better. I want to build a binary classifier based on penalized logistic regression. I have 10 features and 23 observations: 16 from class "0" and ...
1
vote
0answers
41 views
0
votes
1answer
15 views

Using SVM's to output binary 0 or 1 to data

I am using a very large ~700,000 sample training set and ~700,000 sample testing set and training an SVM with the training set. When I run the SVM (SciKit-Learn) on the testing set it outputs only ...
0
votes
2answers
78 views

Feature selection clustering customer segmentation

based on customer data I want to perform a clustering using different clustering algorithms (K-Means, Expectation Maximization, etc.) in R. The most attributes were engineered pursuing the goal to be ...
2
votes
2answers
89 views

Subset selection features acquired from randomized logistic regression

I learned about the concept of randomized logistic regression(or randomized lasso) recently. My data, biological data called Microarray, usually has large features but small samples - 10000 features ...
3
votes
1answer
43 views

restrict splitting variable number in random forest?

Background: I have a set of ~100 features (input) that predict 25 variables (output). My input variables are integers in {1,2,3,4,5,6,7}, my output is continuous. I have ~100K data rows available. I ...
0
votes
1answer
39 views

Appropriate model for feature subset selection

I am working with a feature selection problem. What I am trying to do is find optimal subset of features for classification. My data consist of 100 features and 300 instances, and class label is ...
0
votes
0answers
18 views

Doing backward feature elimination using a classifier different from the one used to classify test set: does it make sense?

I'll make an example: if I use Naive Bayes for backward elimination and then I use optimal attributes discovered from that procedure to classify test set using SVM, does it make sense?
4
votes
3answers
151 views

Can I use linear model on each variable to determine which variables are important?

Suppose we have a n*p matrix X and a n*1 matrix Y, where n is the number of samples and p is the number of variables. p>>n. Also suppose this data is from a biology field experiment. My goal is to ...
7
votes
3answers
139 views

Is it better to do exploratory data analysis on the training dataset only?

I'm doing exploratory data analysis (EDA) on a dataset. Then I will select some features to predict a dependent variable. The question is: Should I do the EDA on my training dataset only? Or ...
0
votes
0answers
35 views

Feature Selection Order

I am implementing univariate feature selection from feature selection!. I have several features among which I am intending to select some features and proceed. Should I scale my data before applying ...
1
vote
1answer
21 views

Can a value appearing more frequently for a class help predict it?

I am trying to analyse my data before doing multi-class classification with SVM. I have several variables. I pick one of them and study it. This is a categorical variable. It can have the value 0 or ...
5
votes
1answer
148 views

how does multicollinearity affect feature importances in random forest classifier?

I have a random forest binary classifier, but the results from the feature importances are somewhat erratic. Here's what I want to know: Does multicollinearity ...