Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
78 views

What are appropriate feature selection techniques for binary features?

Suppose that we have binary features (+1 and -1 or 0 and 1). We have some well-knows feature selection techniques like Information Gain, ...
2
votes
1answer
85 views

Computing Cross-Validation Errors for Subset Selection: error in standard code in the literature?

I am currently trying to understand how to use cross-validation in order to choose among the "best" subsets of different sizes returned by the R function regsubsets (regsubsets returns the "best" ...
0
votes
1answer
26 views

Computing the Interaction gain. Is there an Error in the infotheo package in R?

In order to implementing a certain feature selection method for a classification problem I need to estimate the the interaction the interaction gain between two features and the target variable which ...
0
votes
1answer
160 views

VAR/VECM/ARDL optimal lag selection

Question 1: Is it necessary to consider AIC and the BIC criteria when selecting the lag for a VAR, VECM or ARDL model OR can I use something else? Example: Can I pick 12 lags because the model simply ...
1
vote
0answers
24 views

Choosing between feature selection and regularization to overcome over-fitting in categorical regression

In order to overcome over-fitting during a regression process over categorical features, one can either 1) Apply L1/L2/Elastic regularization during the regression, for example as answered here ...
0
votes
1answer
17 views

How to check feature relevance and representativeness?

How to check whether features from one domain are relevant in other domain? How to evaluate whether features are representing that domain?
0
votes
0answers
42 views

Logistic regression - variable transformation

I have a continues variable(EntropyDistanceFromMean) which I would like to use in a logistic regression, the problem with that variable is that it starting to effect the output (MQL) found on the ...
0
votes
0answers
13 views

Content Based Document classification

I have a corpus of 10 million resumes. I want to add tags to these resumes like Software Engineer, Data Scientists, ...
1
vote
0answers
44 views

Variable importance in regression with large number of missing values

I have a dataset with multiple (approximately 20) categorical and ordinal predictors and a numerical outcome and I am trying to understand which and how each of these predictors affect the outcome ...
1
vote
0answers
39 views

grouping attributes in RF and GBM

i have a dataset with 1000 samples and ~11k features (SNP markers). i have identified 100 additional binary features describing the markers themselves so i have a ...
1
vote
1answer
126 views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
0
votes
1answer
40 views

Is feature complementarity different from feature interaction?

I am writing a conference paper in which I have a sentence like "...complementary/interactive features...". This sentence ...
2
votes
1answer
63 views

What happens to multi-category variables in algorithms like Random Forest that sample the feature space?

Suppose I have a multi-level categorical variable like color (say, with 7 levels). Some software libraries only allow numeric matrices to train models, so we need ...
0
votes
1answer
51 views

In regression, what is the limit of independent variables?

After having taken the Coursera Data Science specialization, I am faced with my first "practical" problem which I plan on solving with some sort of regression. This is my first real world, ...
1
vote
2answers
67 views

What does “the process that generates the data” mean? and How does feature selection help in recovering it?

In [1], one of the motivations to use feature selection is stated to be: "to gain knowledge about the process that generated the data". What does this "process" actually mean? and How does feature ...
3
votes
2answers
142 views

random forests for optimal variable selection/feature selection

Gurus, I just came across this tutorial (http://blog.datadive.net/selecting-good-features-part-iii-random-forests/) about using "random forests" for optimal variable selection/feature selection. The ...
0
votes
0answers
73 views

Feature selection using RFE in SVM kernel (other than linear eg rbf, poly etc)

At this link, there is an example of finding feature ranking using RFE in SVM linear kernel. If I want to check feature ranking in other SVM kernel (eg. rbf, poly etc).How to do it? I have changed ...
0
votes
1answer
86 views

Split dataset by categorical variable or use as a dummy/factor variable?

I'm looking for any sort of best practice or ways to go about this situation. Often I come across datasets that have a categorical variable that I am tempted to split off the main dataset into ...
0
votes
2answers
76 views

is linear regression/polynomial regression sensitive to irrelevant features/noise

is linear regression/polynomial regression sensitive to irrelevant features/noise will their respective weights/coefficients be automatically be tuned down? or is it a straight nail in the coffin? ...
0
votes
0answers
37 views

Analysis of wrapper feature selection ouptput in Weka

I am using Weka to select important features from a dataset. I am using the wrapper method in this application. I chose a decision tree (j.48) for my classifier and Genetic search for the search ...
3
votes
2answers
149 views

Cross validated penalized logistic regression - one standard deviation rule

I am new to this topic and would like to understand it better. I want to build a binary classifier based on penalized logistic regression. I have 10 features and 23 observations: 16 from class "0" and ...
1
vote
0answers
44 views
0
votes
1answer
15 views

Using SVM's to output binary 0 or 1 to data

I am using a very large ~700,000 sample training set and ~700,000 sample testing set and training an SVM with the training set. When I run the SVM (SciKit-Learn) on the testing set it outputs only ...
0
votes
2answers
93 views

Feature selection clustering customer segmentation

based on customer data I want to perform a clustering using different clustering algorithms (K-Means, Expectation Maximization, etc.) in R. The most attributes were engineered pursuing the goal to be ...
2
votes
2answers
98 views

Subset selection features acquired from randomized logistic regression

I learned about the concept of randomized logistic regression(or randomized lasso) recently. My data, biological data called Microarray, usually has large features but small samples - 10000 features ...
3
votes
1answer
44 views

restrict splitting variable number in random forest?

Background: I have a set of ~100 features (input) that predict 25 variables (output). My input variables are integers in {1,2,3,4,5,6,7}, my output is continuous. I have ~100K data rows available. I ...
0
votes
1answer
43 views

Appropriate model for feature subset selection

I am working with a feature selection problem. What I am trying to do is find optimal subset of features for classification. My data consist of 100 features and 300 instances, and class label is ...
0
votes
0answers
21 views

Doing backward feature elimination using a classifier different from the one used to classify test set: does it make sense?

I'll make an example: if I use Naive Bayes for backward elimination and then I use optimal attributes discovered from that procedure to classify test set using SVM, does it make sense?
4
votes
3answers
154 views

Can I use linear model on each variable to determine which variables are important?

Suppose we have a n*p matrix X and a n*1 matrix Y, where n is the number of samples and p is the number of variables. p>>n. Also suppose this data is from a biology field experiment. My goal is to ...
7
votes
3answers
166 views

Is it better to do exploratory data analysis on the training dataset only?

I'm doing exploratory data analysis (EDA) on a dataset. Then I will select some features to predict a dependent variable. The question is: Should I do the EDA on my training dataset only? Or ...
0
votes
0answers
39 views

Feature Selection Order

I am implementing univariate feature selection from feature selection!. I have several features among which I am intending to select some features and proceed. Should I scale my data before applying ...
1
vote
1answer
21 views

Can a value appearing more frequently for a class help predict it?

I am trying to analyse my data before doing multi-class classification with SVM. I have several variables. I pick one of them and study it. This is a categorical variable. It can have the value 0 or ...
5
votes
1answer
197 views

how does multicollinearity affect feature importances in random forest classifier?

I have a random forest binary classifier, but the results from the feature importances are somewhat erratic. Here's what I want to know: Does multicollinearity ...
1
vote
0answers
119 views

Remove features with high correlation

In a classification problem using Linear SVM, I am trying to remove variables which have a strong correlation (Pearson) between them from a dataset. What is the usual threshold recommended? I ...
1
vote
1answer
47 views

Positive and negative impact of predictors on responses in data mining models

My question is an extension to the question asked here. How does one identify the parity of predictor/feature/variable impact on response/outcome in a data mining model. Is there a standard procedure ...
4
votes
1answer
260 views

Use of Random Forests for variable importance as preprocess before another analysis

the question Demonstrate the speed and accuracy of properly applied 'Random Forest' as a variable importance selection tool especially in handling very large data against alternative approaches such ...
0
votes
1answer
31 views

Measure influence of attribues on clustering

I don't have a specific example for my problem and maybe this is trivial, but I want to know how to measure the influence of specific attributes (or dimensions) of a dataset for clustering, like there ...
0
votes
1answer
107 views

pickSizeBest() for recursive feature elimination

I'm struggling providing my recursive feature elimination (RFE) function with valid arguments. This question is technically pretty specific so I hope I've hit the right Forum to ask it. I want to ...
0
votes
1answer
56 views

Linear Regression Model in R - Which variables should I use?

I would like to fit a linear regression model in R for predicting motorbike prices. My dataset has 13 variables, including number of kilometers driven, colour, month of the first registration, etc. ...
1
vote
0answers
55 views

SVM predicts everything in one class

I'm running a basic language classification task. There are two classes (0/1), and they are roughly evenly balanced (689/776). Thus far, I've only created basic unigram language models and used these ...
0
votes
0answers
11 views

Can we derive features from the output variable?

I know this sound weird, but can we use the output variable of the training data set to derive the some new features for feeding the model. If yes then how can it be statistically significant?
0
votes
1answer
56 views

Variable selection for multiple regression from large number of predictors

I have 20 response variables $Y = (Y_1, \dots, Y_{20})$, and 1600 predictor variables $X = (X_1, \dots, Y_{1600})$. There are 128 observations. I wanted to know which pairs of $X$ can best predict ...
0
votes
1answer
183 views

MATLAB function TreeBagger() (Random Forest classification) and different number of variables

I am using MATLAB function TreeBagger() for Random Forest classification, for an assignment. It gives error when the number of variables of the Test data is different from the number of variables of ...
0
votes
0answers
16 views

feature selection for clustering: wrappers

I am trying to understand if it is correct to perform a feature selection process using a wrapper method (for example using algorithms such as random forest, linear regression etc.) and then to extend ...
0
votes
0answers
14 views

How do i classify the feature set obtained using tf*idf

i have obtained the feature set for text classificaion using tf*idf method.I want to classify this feature set using SVM. How do i go ahead?
4
votes
1answer
84 views

How to fit weights into Q-values with linear function approximation

In reinforcement learning, linear function approximation is often used when large state spaces are present. (When look up tables become unfeasible.) The form of the $Q-$value with linear function ...
0
votes
0answers
17 views

To what degree does Mahalanobis distance account for correlations of the data?

I understand that Mahalanobis distance is used when your data are correlated (e.g., as with many environmental variables), but just how correlated can the variables be? Should I be screening and ...
3
votes
1answer
131 views

Best approaches for feature engineering?

I have a regression problem. The aim is to estimate the best fitting curve from a set of features. Now I have extracted a set of features that are relevant based on the literatures found. Now the ...
2
votes
2answers
97 views

Choosing predictors in regression analysis and multicollinearity

I would like to run a linear regression analysis and I'm uncertain about including predictors. I have three predictor variables available. One is based on a lot of previous research. Therefore I am ...