Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
26 views

We should normalize (or standardize) data before feature selection tests (t-test, related matrix, etc.)?

We should normalize (or standardize) data before these feature selection techniques? Which one for every technique? normalization of standardization? t-test Related matrix Stepwise PCA Factor ...
2
votes
1answer
15 views

Is it necessary to use warm_start when tracking oob_score in scikit RandomForestClassifier?

I'm planning on doing feature-selection with RandomForestClassifier by using the feature_importances and ...
1
vote
1answer
28 views

How to select relevant features for binary classification from a given set of features using p values?

Lets say I have 50 features and want to select few relevant features (say around 20 features) for binary (2 class) classification. I have studied that we can use p values to decide which features are ...
3
votes
0answers
21 views

Does this pattern indicate over-fitting in machine learning?

I am working on a diagnostics project, and trying to improve the performance of a classifier(s). We have over a million features to choose from, so feature selection is a real challenge. To look ...
4
votes
3answers
200 views

For linear classifiers, do larger coefficients imply more important features?

I'm a software engineer working on machine learning. From my understanding, linear regression (such as OLS) and linear classification (such as logistic regression and SVM) make a prediction based on ...
3
votes
2answers
86 views

To select variables or not in logistic regression

I am trying to find predictors for an outcome. I was taught to perform univariate analyses & put significant variables into a multivariate logistic regression model. Then I remove variables one by ...
0
votes
1answer
53 views

What is the best way to select variables for clogit model?

I am doing clogit model (clogit of survival package) with around 150 independent variables which are highly correlated. I have to select the combinations of the variables so that the model will be the ...
0
votes
0answers
18 views

Forward search feature selection and cross-validation

I've a question regarding forward search for feature selection. Basically, I've found here and here that the procedure is the following: As the procedure suggests, the cross-validation is applied ...
0
votes
1answer
33 views

building up a predictive model with lots of features and missing data

I'm learning using R to build predictive models recently by myself and have many questions on how to attack a question. I'm given a data set of 8000 observations with 300 features. My goal is to build ...
3
votes
0answers
18 views

Feature Selection Among Groups

I'm trying to do feature selection along a dataset which has: ...
0
votes
0answers
10 views

Feature selection using GA with cross validation

I am working with feature selection on a dataset with 2100 instances.I have split my dataset into training(75%) and testing set(25%).I am using genetic algorithm for finding the optimal feature subset ...
2
votes
0answers
146 views

Coalitional effect in logistic regression and assessing explanarory variable contribution

I have a problem that could be described as logistic regression with all dichotomous variables: 1 response variable (DV) Y (I would call it later as a feature/violet star) and 5 explanatory variables ...
0
votes
0answers
14 views

Polynomial features on logistic regression [duplicate]

I'm working on a LR model, I'm currently trying to add some polynomial features of degree 2. Since the next step is choosing which polynomial features I have to discard, I've got a question: if I keep ...
0
votes
1answer
28 views

Randomizing Class Labels during classification to asses the feature selection results

I have a binary classification problem with thousands of variables and less than a hundred data points and class labels. The class is imbalanced (24 positive 51 negative samples). I have selected some ...
0
votes
0answers
26 views

Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
0
votes
1answer
59 views

Feature Selection for a Machine Learning problem

I have a Machine Learning problem at hand but I'm not sure how to approach it. I have a dataset which has around 5000 observations and around 250 features(most of them are numeric and around 3-4 are ...
5
votes
2answers
109 views

What's wrong with data-guided modeling in regression?

In the Regression Modelling Strategies of Frank Harrell, section 4.1, if I understood correctly, it is not recommendred to using the data to decide how to represent a predictor in a regression model ...
1
vote
1answer
87 views

Does it makes sense to use feature selection before Random Forest?

Everything is in the title, does it makes sense to use feature selection before using random forest? Thank you
0
votes
3answers
110 views

Suspicious results after clustering

I've done a clustering and I think that my results are too good to be trusted. Here is my pipeline: Inputs: a dataset of 208 images, distributed into 2 classes (99 and 109 images in each class). ...
0
votes
0answers
56 views

Lag order selection in error correction model (ECM)

I am building an Error Correction Model for monthly price data ($X, Y, Z$). I am deliberately using an ECM and not VECM and apply a two step approach (estimating cointegration relationship first, then ...
0
votes
0answers
11 views

RFE-RF feature selection

Would it make sense to use RFE-RF in caret for feature selection and then use the selected features to evaluate different model performances i.e SVM, GMB and compare it with the fitted RF model used ...
0
votes
0answers
23 views

What techniques can I use to perform feature selection in the context of classification with an highly unbalanced dataset ?

I'm dealing with CTR prediction, which is a classification problem with an highly unbalanced dataset (around 1 positive class for 200 negative class). Most of my features (>90%) are categorical. ...
0
votes
0answers
30 views

Does the Support Vector Machine favor datasets with fewer features?

I am a bit concerned, as there are so many questions asked and so few answers given. I take it, machine learning has become quite common to use, but only little is really understood about their ...
1
vote
1answer
56 views

What is the criterion value on sequential feature selection for binary classification?

I have a set of data represented by 16 features and a binary classification (true, false). I want to determine which features are important using forward and backward sequential feature selection, ...
2
votes
1answer
95 views

Is each of the PCA or PLS components just one of the original variables?

I am confused about what a component is in PCA and PLS. Are the components just the original variables but not necessarily in the same order? For example, in PCA, if I had 8 variables in my data, ...
0
votes
1answer
24 views

Feature selection based on mean, standard deviation and mean absolute deviation

Suppose we have a large dataset (~ 60000 entries, 58 variables, 4 class labels). For each variable mean, standard deviation and mean absolute deviation are calculated - separately for every class ...
9
votes
5answers
316 views

Understanding which features were most important for logistic regression

I've built a logistic regression classifier that is very accurate on my data. Now I want to understand better why it is working so well. Specifically, I'd like to rank which features are making the ...
0
votes
0answers
40 views

Perform various random iterations with feature selection in Caret R package, to select a constant subset of features

I would like to use the rfe function from the R package caret, for applying feature selection--with the custom pre-defined function rfFuncs--, in order to select a subset of features regarding a ...
1
vote
0answers
30 views

Feature binarization for RF/GBMs?

Are there any advantages to feature binarization for random forests or gradient-boosted machines? For example, suppose I am predicting snowstorms for the next day using various past measurements - ...
0
votes
1answer
25 views

How many features should I say I have in my model?

I am running machine learning using name features to predict Y (binary 0 and 1 labels). Using the name entity (eg: John Carter), I derive into 4 name substring features (1: first name = "John", 2: ...
0
votes
0answers
9 views

Multivariate 'group' regression where variables are grouped

I'd like to know a model which is useful for a multivariate regression problem where feature variables can be split into several groups. Formal problem setting is represented as follows: Let $G$ be ...
0
votes
0answers
12 views

What features make a set of data related?

I have a set of data samples(Let's call it $X$) which I know are somehow related. Let's say it is a set of anomalies that I have detected. Each sample $X_i$ has $n$ features. I want to know what ...
0
votes
0answers
4 views

Calculating F-score for feature selection on values of all the same sign

The F-score as defined on page 3 of this paper https://www.csie.ntu.edu.tw/~cjlin/papers/features.pdf doesn't apply for values of all the same sign, since either the positive mean or negative mean ...
1
vote
0answers
32 views

RReliefF algorithm for regression for feature selection with an example

How does the RReliefF algorithm for regression work? The original ReliefF algorithm for classification problems uses the concept of nearest hits and misses. I am confused how ReliefF can be used for ...
0
votes
0answers
13 views

Feature Extraction of motion in 3D

I have a data of object trajectory in 2D space with different length. Example: t(s) X Y 0 2 5 1 2 6 2 3 6 3 4 8 ... ...
0
votes
1answer
30 views

In a regression problem, can the weights estimated be used as a measure of importance of a predictor?

In a regression problem, a machine learning technique used estimates the continuous output as $ y= w^Tx+b$. Now doesn't the weight indicate the importance of a predictor as larger weight for a ...
3
votes
0answers
78 views

Important question regarding feature selection methodologies in R concerning the randomness of the results

I'm currently testing some feature selection methodologies/algorithms in R, like the Recursive Feature Elimination from the R caret package, and also the RRF R package, to select a subset of features ...
0
votes
1answer
19 views

Best subset algorithm for ridge regression in R [closed]

I'm searching for a best subset selection algorithm for ridge regression in R. There is a wide range of algoritms for an ordinary least squares fit. There also exists a function like ...
3
votes
2answers
87 views

Can we correctly identify all the non-zero coefficients in the linear regression model?

I have a conceptual question regarding linear regression. Assume our model is correct, i.e., the response variable $Y$ is indeed coming from the model $$Y=\beta_0+\beta X+\epsilon.$$ Here $X$ is ...
0
votes
0answers
17 views

Is there a way to easily select points out of a feature space?

Working in machine learning with images like (source: http://surenkum.blogspot.de/2013/03/feature-normalization-for-learning.html ) I want to apply different methodes to separate the data points ...
0
votes
0answers
10 views

Feature selection - number of features/levels when having categorical data

Denote a feature set $x_1,...,x_p$ and target variable $y$. Assuming a predictive modeling technique (some type of logit regression) is being used to predict the value of $y$. This algorithm is ...
0
votes
1answer
48 views

Performance of a classifier change heavily

I'm using a data set of 32 face persons and a svm-rbf to classify and a random group of 70% for train and 30% for test. The problem is that my results are heavily dependent of the random group used ...
1
vote
0answers
24 views

features selection - methods based on estimated feature importances vs. methods based on scores

I noticed that all feature selection methods implemented in sklearn are based on external estimator that assigns weights to features, AKA feature_importances. I ...
0
votes
0answers
28 views

variable selection before a decision tree

I want to built a predictive decision tree. I have a dataset with +/- 1000 observations and 1500 variables. Can I just built my decision tree (training + validation dataset) with all the 1500 ...
3
votes
0answers
55 views

How to do dimensionality reduction on a huge data set?

I am working with fMRI data of ~1000 subject. Each subject has a feature vector of ~150 million dimension. So I can only keep the feature vectors of ~10 subjects in memory. What are some algorithms ...
0
votes
0answers
15 views

Properties of data for mutual information as feature selection

I have a dataset $D = \{(x_1, y_1),...(x_n, y_n)\}$, $x$ is a vector while $y$ is a scalar. I want to select a subset of features of $x$. I want to use mutual information as the feature selection ...
0
votes
1answer
32 views

Selecting variables and fitting to bounded response (0,1)

I have a dataset with 15 binary covariates and a continuous response variable bounded between 0 and 1. The binary variables represent correct or incorrect answers on a short test and the response ...
0
votes
0answers
20 views

How do you statistically determine the point when a series becomes stationary (i.e. the y variable saturates)?

I am computing AUC of a model as a function of features in the model. In general it does very well but I wanted to optimize the number of features by looking @ the trends in the AUC. I know the AUC ...
1
vote
1answer
68 views

Using trees after variable selection using Lasso/Random

I am new into Machine Learning so please excuse me if my question is naive. My question is, is it possible to use trees for example rpart or ctree after variable selection procedures such as ...
0
votes
0answers
27 views

ROC change after variable selection with glmnet

I was using glmnet in caret to select important variables. The code is like ...