Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
1answer
111 views

What is the criterion value on sequential feature selection for binary classification?

I have a set of data represented by 16 features and a binary classification (true, false). I want to determine which features are important using forward and backward sequential feature selection, i.e....
2
votes
1answer
126 views

Is each of the PCA or PLS components just one of the original variables?

I am confused about what a component is in PCA and PLS. Are the components just the original variables but not necessarily in the same order? For example, in PCA, if I had 8 variables in my data, ...
0
votes
1answer
45 views

Feature selection based on mean, standard deviation and mean absolute deviation

Suppose we have a large dataset (~ 60000 entries, 58 variables, 4 class labels). For each variable mean, standard deviation and mean absolute deviation are calculated - separately for every class ...
9
votes
5answers
349 views

Understanding which features were most important for logistic regression

I've built a logistic regression classifier that is very accurate on my data. Now I want to understand better why it is working so well. Specifically, I'd like to rank which features are making the ...
0
votes
0answers
43 views

Perform various random iterations with feature selection in Caret R package, to select a constant subset of features

I would like to use the rfe function from the R package caret, for applying feature selection--with the custom pre-defined function rfFuncs--, in order to select a subset of features regarding a ...
1
vote
0answers
38 views

Feature binarization for RF/GBMs?

Are there any advantages to feature binarization for random forests or gradient-boosted machines? For example, suppose I am predicting snowstorms for the next day using various past measurements - ...
0
votes
1answer
34 views

How many features should I say I have in my model?

I am running machine learning using name features to predict Y (binary 0 and 1 labels). Using the name entity (eg: John Carter), I derive into 4 name substring features (1: first name = "John", 2: ...
0
votes
0answers
9 views

Multivariate 'group' regression where variables are grouped

I'd like to know a model which is useful for a multivariate regression problem where feature variables can be split into several groups. Formal problem setting is represented as follows: Let $G$ be ...
0
votes
0answers
12 views

What features make a set of data related?

I have a set of data samples(Let's call it $X$) which I know are somehow related. Let's say it is a set of anomalies that I have detected. Each sample $X_i$ has $n$ features. I want to know what ...
0
votes
0answers
6 views

Calculating F-score for feature selection on values of all the same sign

The F-score as defined on page 3 of this paper https://www.csie.ntu.edu.tw/~cjlin/papers/features.pdf doesn't apply for values of all the same sign, since either the positive mean or negative mean ...
2
votes
1answer
58 views

RReliefF algorithm for regression for feature selection with an example

How does the RReliefF algorithm for regression work? The original ReliefF algorithm for classification problems uses the concept of nearest hits and misses. I am confused how ReliefF can be used for ...
0
votes
0answers
13 views

Feature Extraction of motion in 3D

I have a data of object trajectory in 2D space with different length. Example: t(s) X Y 0 2 5 1 2 6 2 3 6 3 4 8 ... ...
0
votes
1answer
30 views

In a regression problem, can the weights estimated be used as a measure of importance of a predictor?

In a regression problem, a machine learning technique used estimates the continuous output as $ y= w^Tx+b$. Now doesn't the weight indicate the importance of a predictor as larger weight for a ...
3
votes
0answers
82 views

Important question regarding feature selection methodologies in R concerning the randomness of the results

I'm currently testing some feature selection methodologies/algorithms in R, like the Recursive Feature Elimination from the R caret package, and also the RRF R package, to select a subset of features ...
0
votes
1answer
30 views

Best subset algorithm for ridge regression in R [closed]

I'm searching for a best subset selection algorithm for ridge regression in R. There is a wide range of algoritms for an ordinary least squares fit. There also exists a function like ...
3
votes
2answers
101 views

Can we correctly identify all the non-zero coefficients in the linear regression model?

I have a conceptual question regarding linear regression. Assume our model is correct, i.e., the response variable $Y$ is indeed coming from the model $$Y=\beta_0+\beta X+\epsilon.$$ Here $X$ is ...
0
votes
0answers
17 views

Is there a way to easily select points out of a feature space?

Working in machine learning with images like (source: http://surenkum.blogspot.de/2013/03/feature-normalization-for-learning.html ) I want to apply different methodes to separate the data points ...
0
votes
0answers
16 views

Feature selection - number of features/levels when having categorical data

Denote a feature set $x_1,...,x_p$ and target variable $y$. Assuming a predictive modeling technique (some type of logit regression) is being used to predict the value of $y$. This algorithm is ...
0
votes
1answer
50 views

Performance of a classifier change heavily

I'm using a data set of 32 face persons and a svm-rbf to classify and a random group of 70% for train and 30% for test. The problem is that my results are heavily dependent of the random group used ...
1
vote
0answers
28 views

features selection - methods based on estimated feature importances vs. methods based on scores

I noticed that all feature selection methods implemented in sklearn are based on external estimator that assigns weights to features, AKA feature_importances. I ...
0
votes
0answers
45 views

variable selection before a decision tree

I want to built a predictive decision tree. I have a dataset with +/- 1000 observations and 1500 variables. Can I just built my decision tree (training + validation dataset) with all the 1500 ...
3
votes
0answers
63 views

How to do dimensionality reduction on a huge data set?

I am working with fMRI data of ~1000 subject. Each subject has a feature vector of ~150 million dimension. So I can only keep the feature vectors of ~10 subjects in memory. What are some algorithms ...
0
votes
0answers
16 views

Properties of data for mutual information as feature selection

I have a dataset $D = \{(x_1, y_1),...(x_n, y_n)\}$, $x$ is a vector while $y$ is a scalar. I want to select a subset of features of $x$. I want to use mutual information as the feature selection ...
0
votes
1answer
38 views

Selecting variables and fitting to bounded response (0,1)

I have a dataset with 15 binary covariates and a continuous response variable bounded between 0 and 1. The binary variables represent correct or incorrect answers on a short test and the response ...
0
votes
0answers
20 views

How do you statistically determine the point when a series becomes stationary (i.e. the y variable saturates)?

I am computing AUC of a model as a function of features in the model. In general it does very well but I wanted to optimize the number of features by looking @ the trends in the AUC. I know the AUC ...
1
vote
1answer
74 views

Using trees after variable selection using Lasso/Random

I am new into Machine Learning so please excuse me if my question is naive. My question is, is it possible to use trees for example rpart or ctree after variable selection procedures such as Lasso/...
0
votes
0answers
32 views

ROC change after variable selection with glmnet

I was using glmnet in caret to select important variables. The code is like ...
2
votes
2answers
112 views

Ranking features in logistic regression

I used Logistic Regression. I have six features, I want to know the important features in this classifier that influence the result more than other features. I used Information Gain but it seems that ...
2
votes
1answer
42 views

Feature selection using cross-validation and multiple linear regression

I would like to predict discrete (mean) values from discrete (mean & std) values of the extracted features. My question is how you should perform feature selection when you are applying K-fold ...
0
votes
0answers
16 views

Correct evaluation/ comparison between undercomplete and overcomplete representations

Suppose I'm performing Unsupervised Feature Learning method to learn a representation of the data that is under-complete (e.g. 100 features) and use another algorithm to learn an over-complete ...
2
votes
0answers
53 views

Variable selection with multi-variate time series

I currently have a data.frame with 273 variables and 94 rows: ...
1
vote
1answer
34 views

Categorical with many levels for NN

I have a dataset which lists the amount of seconds a user held a session by browser_type. For example: ...
0
votes
0answers
24 views

How do I have select features which are influential for prediction?

I have a dataset which has dependent variable(label) as possible destinations and independent variable(features) as age,language, gender and many other categorical variables. How do i find which are ...
0
votes
0answers
79 views

What are appropriate feature selection techniques for binary features?

Suppose that we have binary features (+1 and -1 or 0 and 1). We have some well-knows feature selection techniques like Information Gain, ...
2
votes
1answer
86 views

Computing Cross-Validation Errors for Subset Selection: error in standard code in the literature?

I am currently trying to understand how to use cross-validation in order to choose among the "best" subsets of different sizes returned by the R function regsubsets (regsubsets returns the "best" ...
0
votes
1answer
29 views

Computing the Interaction gain. Is there an Error in the infotheo package in R?

In order to implementing a certain feature selection method for a classification problem I need to estimate the the interaction the interaction gain between two features and the target variable which ...
0
votes
1answer
202 views

VAR/VECM/ARDL optimal lag selection

Question 1: Is it necessary to consider AIC and the BIC criteria when selecting the lag for a VAR, VECM or ARDL model OR can I use something else? Example: Can I pick 12 lags because the model simply ...
1
vote
0answers
24 views

Choosing between feature selection and regularization to overcome over-fitting in categorical regression

In order to overcome over-fitting during a regression process over categorical features, one can either 1) Apply L1/L2/Elastic regularization during the regression, for example as answered here ...
0
votes
1answer
18 views

How to check feature relevance and representativeness?

How to check whether features from one domain are relevant in other domain? How to evaluate whether features are representing that domain?
0
votes
0answers
44 views

Logistic regression - variable transformation

I have a continues variable(EntropyDistanceFromMean) which I would like to use in a logistic regression, the problem with that variable is that it starting to effect the output (MQL) found on the y-...
0
votes
0answers
29 views

Content Based Document classification

I have a corpus of 10 million resumes. I want to add tags to these resumes like Software Engineer, Data Scientists, ...
1
vote
0answers
55 views

Variable importance in regression with large number of missing values

I have a dataset with multiple (approximately 20) categorical and ordinal predictors and a numerical outcome and I am trying to understand which and how each of these predictors affect the outcome (...
1
vote
0answers
40 views

grouping attributes in RF and GBM

i have a dataset with 1000 samples and ~11k features (SNP markers). i have identified 100 additional binary features describing the markers themselves so i have a ...
1
vote
1answer
149 views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
0
votes
1answer
41 views

Is feature complementarity different from feature interaction?

I am writing a conference paper in which I have a sentence like "...complementary/interactive features...". This sentence ...
2
votes
1answer
67 views

What happens to multi-category variables in algorithms like Random Forest that sample the feature space?

Suppose I have a multi-level categorical variable like color (say, with 7 levels). Some software libraries only allow numeric matrices to train models, so we need ...
0
votes
1answer
52 views

In regression, what is the limit of independent variables?

After having taken the Coursera Data Science specialization, I am faced with my first "practical" problem which I plan on solving with some sort of regression. This is my first real world, business-...
1
vote
2answers
68 views

What does “the process that generates the data” mean? and How does feature selection help in recovering it?

In [1], one of the motivations to use feature selection is stated to be: "to gain knowledge about the process that generated the data". What does this "process" actually mean? and How does feature ...
3
votes
2answers
160 views

random forests for optimal variable selection/feature selection

Gurus, I just came across this tutorial (http://blog.datadive.net/selecting-good-features-part-iii-random-forests/) about using "random forests" for optimal variable selection/feature selection. The ...