Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

2
votes
2answers
1k views

Mutual Information based feature selection

Suppose I have a feature matrix $F = [f_1^T,f_2^T,...,f_m^T]$ whereby $f_j^T \in \mathbb R^{n_t \times 1}$ is the $j$th column vector of $F$ ($n_t$ is the number of different events/trials and $m$ is ...
1
vote
2answers
2k views

Issues with feature selection in matlab

I am trying to use sequentialfs to do some feature selection in matlab. I have huge dimensional data of 22215 features. When I tried to use sequentialfs with svm as classifier so that it selects the ...
-1
votes
1answer
39 views

what are the dependent and independent factors and categorical for below project [closed]

my project is identification of factors influencing motor bikes for home to office trip. there are some attiributes like gender,age,marital status,education qualification,job type,working sector, no. ...
15
votes
5answers
749 views

What can cause PCA to worsen results of a classifier?

I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
1
vote
0answers
141 views

True and false discovery rate in variable selection

I have a question about how I can calculate true and false positive rate in a simulation study? I have seen some articles and thesis by different definitions. One of them is the following one: ...
1
vote
2answers
147 views

Sparse hyperspace clustering

I have a dataset of M elements where every item is represented by a feature vector of length N where N is very large and only a small subset of N is bigger then zero for every item. So I have a sparse ...
6
votes
1answer
666 views

Variable importance randomForest negative values

I am asking myself if it is a good idea to remove those variables with a negative variable importance value ("%IncMSE") in a regression context. And if it gives me a better prediction? What do you ...
2
votes
1answer
368 views

How to perform step() when n < p in R?

I am trying to perform stepwise regression for variable selection in R. In matlab, the stepwisefit function is able to work in ...
2
votes
0answers
44 views

Why do correlations matter in simulating data to compare classifiers when p >> N?

In genomics and computational biology, expression data sets contain a much larger number of features (p) than the number of observations (N). I wanted to simulate data where p>>N to compare the ...
4
votes
1answer
1k views

Using Adaboost for feature selection?

Is it okay to use Adaboost to do feature selection (selecting a subset of dimensions $S$ from a high-dimensional feature vector $V$)? I divided the samples into four non-overlapping sets: $A$ ...
2
votes
1answer
82 views

A buggy but effective feature?

Through error analysis I found that a quite effective feature actually has bugs in its implementation. Correcting the bugs actually decreased the classifier's performance. What do you do? Correct the ...
1
vote
0answers
48 views

How do I know if the differences in ICs among candidate models are significant?

I'm doing some exploratory modelling on a data set with 29 covariates and an additional 11 variables that are of interest to my research question. My strategy is to develop a model with a subset of ...
13
votes
2answers
581 views

Bayesian variable selection — does it really work?

I thought I might toy with some Bayesian variable selection, following a nice blog post and the linked papers therein. I wrote a program in rjags (where I am quite a rookie) and fetched price data ...
3
votes
1answer
498 views

LASSO vs forward selection

I have two questions: I use cross validation to select a LASSO model, does the step in which a particular variable enter, indicate its relative importance? Let's age enter in step 1 and sex enter in ...
16
votes
4answers
4k views

Features for time series classification

I consider the problem of (multiclass) classification based on time series of variable length $T$, that is, to find a function $$f(X_T) = y \in [1..K]\\ \text{for } X_T = (x_1, \dots, x_T)\\ ...
0
votes
1answer
733 views

Feature selection with Caret for data with more than one target

I am trying to do some feature selection, having around 3500 variables for about 200 samples. To each sample is associated two numerical values (the expected outcome). I can't manage to make the caret ...
1
vote
1answer
132 views

2D Shape Feature extraction

What are popular techniques for feature extraction of shapes? I'm doing image analysis, and I want to classify a smooth object (one with smooth boundaries) from a rough object (has a zig-zag like ...
1
vote
0answers
59 views

Relationship between threshold, number of features and accuracy/error? (text classification)

Are you aware of any research papers that explain a relationship between the following concepts? threshold (removal of features, whose frequencies are greater than or less than a defined ...
1
vote
0answers
160 views

Number of features and accuracy/f-measure - Who can explain this results?

I'm performing a binary sentiment classification (positive/negative) based on a Naive Bayes classificator and a SVM. To select top k features I use the MRMR algorithm. The model is trained using a ...
0
votes
1answer
168 views

Partial correlation

I want to create a regression model to predict state crime rate. There are two variables among 10 ( Vi= # of violent crimes per 100,000 population, Vi2 = # of violent crimes per 10,000 population) ...
0
votes
3answers
344 views

Classifier feature importance

If I train a GNB/LDA/kNN/other classifier I would like to know, in the model built, how important are features to classify or which feature(s) drives the classifier. For example in SVM models the ...
1
vote
1answer
1k views

How does scikit-learn perform $\chi^2$ feature selection on non-categorical features?

I'm experimenting with $\chi^2$ feature selection for some text classification tasks. I understand that $\chi^2$ test checks the dependencies B/T two categorical variables, so if we perform $\chi^2$ ...
2
votes
0answers
341 views

LASSO vs AIC for feature selection with the Cox model

I have some questions about the Lasso. After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
1
vote
1answer
213 views

'Forward Stagewise' option in LARS algorithm

Can anyone help me understand the forward stagewise part in the LARS algorithm? I was reading the R code and could not figure out what is ...
2
votes
3answers
9k views

How should I interpret the p-values (i.e. t-tests) in regressions, and can I use them for feature selection?

I'm trying to do an OLS regression with several independent variables, and want to better understand how to interpret the p-values from doing the t-tests on the independent variables within my ...
3
votes
4answers
2k views

Decision Tree as variable selection for Logistic Regression

I have to do a Logistic Regression, and have to use a subset of the variables. I received this "tip": do a Decision Tree first, and use the most relevant variables in the Logistic Regression. Is this ...
-1
votes
1answer
179 views

Random forest like procedure for regression or other statistical models

I'm wondering if there exist methods similar to one used in random forest algorithm - I mean taking simultaneously bootstrap sample and random subset of features, then building statistisal model. Have ...
2
votes
1answer
2k views

Feature selection using caret + repeatedcv

I am using caret and repeatedcv with repeats for feature selection. That is, ...
1
vote
0answers
712 views

How to select the best variables by RandomForest in R?

I have a table of mRNA levels of my target gene and it's transcription factors in many different condition. What I want to do is to select the most important ...
1
vote
0answers
352 views

Bootstrap randomized Lasso selection for a Cox model

I'm interested in variable selection for a cox proportional hazards model. I've read this article which slightly favours randomized bootstrap lasso selection over bootstrap lasso selection since it ...
3
votes
1answer
5k views

What is “feature space”?

What is the definition of "feature space"? For example, When reading about SVMs, I read about "mapping to feature space". When reading about CART, I read about "partitioning to feature space". I ...
1
vote
1answer
261 views

How to define in R the most important variables?

I have a data set with my target gene and more than thousand transcriptional factors somehow correlated with this gene. There is data of these factors in more than 70 variable conditions. What I'm ...
4
votes
0answers
356 views

time series with different length: feature extraction and classification [closed]

I have a binary classification problem, where each data point is multi-channel time-series, which can be represented as a matrix $T \times F$, where $T$ is the time-series length, and $F$ as the ...
3
votes
0answers
203 views

Fast algorithm for variable selection

The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations. The data is sparse, that is, many of the variables has many zeros and few positive ...
1
vote
0answers
262 views

Univariate feature ranking in classification

Scikit-learn has function to evaluate the F-statistics for univariate feature importance feature selection. According to the web page they are calculating ANOVA F value. If I understood correctly, ...
2
votes
1answer
186 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
1
vote
0answers
424 views

Feature importance

The extremely randomized trees classifier (scikitlearn) provides a (multivariate) feature importance measurement Ensemble methods/feature importance evaluation. For each feature, the classifier ...
2
votes
0answers
334 views

Kernel in PenalizedSVM R package

There is not option to select kernel in penalizedSVM R package. What kernel do they use? Is there some other R package with penalized SVM methods where I can choose various kernels?
3
votes
1answer
67 views

Methods for teasing apart the influence of different time series features on a target feature?

Are there any established methods for teasing apart the influence of different time series features on a target feature? To illustrate: The target: Sales volume of kittens. Features: Time of year, ...
1
vote
1answer
308 views

When is the PMI value good or bad?

Pointwise mutual information is calculated by this formula $pmi(x;y) = log(p(x,y)/p(x)p(y))$ , my question now is, When is this pmi good and when is it bad. I know if the value is low it is bad, but ...
2
votes
1answer
293 views

Multiclass classification with SVM a question about the feature vectors

I was told I had to direct my machine learning questions to this site. So here it goes. I'm trying to do Multiclass classification with SVM. I have 7 classes. Now I was wondering if the following is ...
3
votes
1answer
82 views

Changing variable values and examine the outcome difference between the altered and original data

I recently read an approach which is used to find the effect of changing an independent variable. They are doing a classification problem, so each data row (or record) is associated with an outcome ...
-1
votes
1answer
139 views

How logical to select features with respect to the correlation matrix and weigthing?

Is it logical to name low correlated features as valuable and choosing the low corelated ones for classification? Or it depends on the algorithm used for the purpose? How do I need to interpret a ...
1
vote
0answers
91 views

Overfitting a linear Linear Discriminant Function

I am estimating a Linear Discriminant function with 250 input variables over 4000 data records. Should I consider feature selection, am I over fitting the model? How do I know when feature selection ...
4
votes
2answers
131 views

Random search for the optimal number of input features and optimal number of hidden layers for a MLP?

I've performed a random search in hypothesis space $$\{(c,h)| c \in U[1,256]; h\in U[1,100];c \in \mathrm{Z} \text{ and } h \in \mathrm{Z}\}$$ that defines the parameters of a standard multilayer ...
2
votes
1answer
249 views

Selecting optimal number of input features and optimal number of hidden layers for a MLP?

What is the best way to select parameters for a binary neural network classifier? More specifically I have 265 features ranked according to Mutual Information Criterion. I have to determine the ...
2
votes
2answers
106 views

Can feature selection be considered a way to observe relationship between variables like correlation?

In correlation we can observe relationship between a pair of variables, let me call it X1 and Y. Now, considering I have the predicting variables X1, X2, ..., Xn and the variable Y. Does the ...
1
vote
1answer
269 views

highly correlated features and high ranking

I am classifying different texts and I wondering about some features that are highly correlated. I have 49 features. Some features are absolute counters (integers) but most features are relative ...
3
votes
1answer
370 views

Why can't Bayesian variable selection be used with categorical variables with more than 2 levels?

I am reading this article which is the first approach on Bayesian variable selection. In the discussion section it says that one of the major limitations of the particular method is that it cannot be ...
5
votes
2answers
552 views

Why does increasing the number of features reduce performance?

I'm trying to gain an intuition as to why increasing the number of features could reduce performance. I'm currently using an LDA classifier which performs better bivariately among certain features ...