Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
1answer
12 views

What does the varImp function in the caret package actually compute for a glmnet (elastic net) object

I am fitting an elastic net model with glmnet via the caret package with 189 predictors and a binomial criteria (a,b) ...
0
votes
1answer
69 views

Feature selection when bagging trees/random forest

I want to get a better understanding of feature selection and how the number of features affect performance when bagging trees. I am using Matlab's treebagger and I ...
1
vote
1answer
9 views

Multi Categorical Features vs multiple Features for categories

Say I am discretizing continuous data based on percentiles. (I realize this is generally frowned upon, but I am doing this for the sake of experiment) I am trying different percentiles, eg breaking ...
0
votes
0answers
11 views

How to predict the predict upper bound of a learning algorithm?

I am selecting features for a Logistic regression classifier. I have tested a lot of feature selection algorithms. however, it seems that there exist a fixed upper bound AUC value for a fix feature ...
0
votes
0answers
12 views

What is the best practices as to cropping the positive examples for training hog classifier?

I've been reading up on the literature on HOG training, but I can't figure out what the best practices are for cropping the positive examples. Question: Do you crop tight or with margins on the ...
1
vote
0answers
32 views

Different variable importance results with stabsel and mboost

I'm using glmboost in the mboost package to fit a boosted regression using linear models as the base learner. There are 13200 observations and about 75 variables, ...
2
votes
1answer
50 views

K-means clustering feature selection

I have a set of English and foreign language documents that I would to perform k-means clustering on to find document groups by topic. These documents are concatenated social media comments for ...
0
votes
1answer
96 views

How to normalize time series?

This is a general question on normalization of data so that all the variables are within the same range. Why do we normalize data in pattern classification? How to normalize time series which is ...
0
votes
0answers
10 views

Bayesian network overfit - number of features and examples

For a dataset consisting of 150 examples (mostly binary features) what would be the number of features needed so that a Bayesian network doesn't overfit? I know there is no exact answer and I've ...
1
vote
0answers
20 views

Poisson model with fractions

I have a simple website with a home page that has 5 different images on it. All images have a fixed set of 'features' associated with them (size, color, position etc.,). When a visitor comes to the ...
0
votes
1answer
43 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
1
vote
1answer
100 views

Feature selection and training on the same sample

Is feature selection and training on the same sample a bad idea? I want to emphasize that I am not going to use test set for feature selection. If I use the whole train set for feature selection and ...
0
votes
1answer
41 views

Selecting variables in multiple linear regression in R

Consider that we have a problem with 4 variables (y, x1, x2 and x3) and we want to do a multiple linear regression model. As we need to know which variables are the most important in the problem, we ...
0
votes
0answers
20 views

Collection of features that correlated with strength and elongation of nonwoven fibers

I'm working on a regression problem which estimates strength and elongation of a non woven fibers by using some features collected from processed images of the fibers. For this study I've collected 50 ...
0
votes
1answer
16 views

Using a set as a feature in decision tree classification

I'm faced with a data set where one of the features is a set of 4-5 categories (this number of categories isn't constant). I need to use this feature for building a decision tree. I searched online ...
2
votes
1answer
131 views

Feature construction for text mining

In the text mining, besides N-gram model, what are the state-of-art models for building feature space while capturing the dependence among the different words, or capturing the semantic meaning in the ...
1
vote
2answers
112 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
0
votes
1answer
88 views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
13
votes
5answers
451 views

What can cause PCA to worsen results of a classifier?

I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
2
votes
2answers
1k views

The disadvantage of using F-score in feature selection

F-score can be used to measure the discrimination of two sets of real-numbers and can be used for feature selection. However, I once read that A disadvantage of F-score is that it does not reveal ...
2
votes
1answer
23 views

Boruta score goes to minus infinity

I'm running the Boruta algorithm with a $179\times 36$ predictor matrix and a numerical response. Most of the variables have a score going to -Inf. Should I ...
0
votes
1answer
252 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
1
vote
1answer
85 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
6
votes
1answer
997 views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
2
votes
1answer
169 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
1
vote
1answer
208 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
3
votes
1answer
40 views

Comparing classification results from models selected differently with Kruskal-Wallis

For a seminar at university I'm reviewing a paper that proposes a new feature selection algorithm. The authors also evaluated their algorithm by applying it beside two other feature selection ...
1
vote
1answer
215 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
1
vote
1answer
19 views

Getting less number of features in weight vectors as were provided for SVM

I have trained a SVM with 18881 features and wanted to know the ranking of features. I tried the method given at http://stackoverflow.com/questions/7390173/svm-equations-from-e1071-r-package for it ...
1
vote
2answers
169 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
1
vote
3answers
2k views

Use of PCA analysis to select variables for a regression analysis [duplicate]

I have too many environmental variables to use in a multiple regression analysis. If I use all the variables the models are just too complex. The use of the PCA axes in the regression analysis was ...
0
votes
0answers
29 views

K Fold Cross Validation, Variable Selection in LDA

I'm currently working on a multi-class classification problem and I attempt to use lda for the same. I have 2 questions here. 1) Is it possible to perform k-fold ...
1
vote
1answer
67 views

How to handle missing data in a small $n$ large $k$ machine learning scenario?

I have a sample size $N=130$ and $1000$ variables. I am using machine learning techniques (SVM) for analysing the data. Some variables in the dataset have values that are so huge that they must be ...
0
votes
1answer
58 views

Large data variable selection

I'm looking for some methods of variable selection on large datasets.The number of variables are around 30-40, but the number of observations is quite large (around 36000000) Any methods which I ...
2
votes
1answer
72 views

If number of samples is smaller than number of features, how can all the variance in PCA be explained by less than $n$ components? [duplicate]

I believe I have a problem understanding PCA: I would like to use this technique to reduce the number of features of my problem. I originally have 10,000 features and 500 samples. However, the use of ...
0
votes
0answers
18 views

Contribution to the components of a Gaussian mixture by data features

My question is about modelling data with a GMM using EM. One can split the mean and variance of each component into parts as well when working with data with multiple features. My question is what ...
0
votes
0answers
30 views

How to check that the selected features are not overfitting

I'm trying to select best features for sentiment classification for a set of reviews, and using penalized SVM and Logistic regression to perform such task. by basically iterating over different ...
3
votes
1answer
66 views

“…if the data is linearly separable”

I keep hearing this phrase as a precursor to many algorithms, but I am not sure how exactly one goes about finding out if the data is indeed, linearly separable. Of course, if the data has ...
1
vote
1answer
17 views

How to find the cause of defect in a process

Suppose a product A undergoes a certain process. This product A is produced at a rate of 8000 per month and out of those in 75 cases defects are generated. In the data set, I have rows corresponding ...
0
votes
0answers
12 views

Performing interaction with a very significant predictor drives down p val of other predictors. But does it makes sense?

I'm observing a phenomenon that I can't understand. I have a linear regression setting with categorical vars. A couple of these elicit an highly significant coefficient and low p values. When used ...
0
votes
2answers
91 views

Choosing the best featureset for prediction

I have this N sets of features F each with $F_i$ number of features. All the feature sets have 20000 examples and we have 20,000 labels for them. Lets say feature set $F_1$ has 10 features and ...
2
votes
1answer
44 views

Difference of variable selection and importance estimation

Isn't variable importance estimation a necessary prerequisite for variable selection? Is there any use case where you want to select non-important variables for your model? So, why is variable ...
0
votes
0answers
47 views

Multple linear regression, adding one predictor with almost perfect fit make others irrelevant

I found something interesting while playing with some data and linear regression. I built a regression with various predictors, more or less correlated with the outcome. Then I added one predictor ...
1
vote
1answer
32 views

Variable selection and validation dataset

According to Hastie & Tibshirani, we shouldn't use validation datasets to do variable selection; otherwise, we will overestimate the model fit. However, it seems quite often to select variables ...
0
votes
1answer
27 views

Feature selection for unknown parametric model

Suppose one has about 500 points of 50 dimensional data that one knows a priori is derived from a parametric model (perhaps with some outliers). Does using this knowledge help in feature selection? I ...
2
votes
2answers
79 views

Do I need to take out any predictors from multiple regression if I put in some principal components as additional predictors?

I have an assignment which involves one area-level dataset made of $366$ scale variables. I have to perform PCA, compare it with rates of an additional response variable $X$, and comment on its face ...
0
votes
1answer
50 views

How to interpret random forest importance numbers

I ran randomForest in R package using 7 predictors variables (x1 to x7). I repeated the test with 4 dependent variables (y1 to y4). The importance numbers (IncNodePurity) are plotted in following ...
5
votes
3answers
170 views

Regression in $p\gg N$ setting (predicting drug efficiency from gene expression with 30k predictors and ~30 samples)

I have a dataset of 29 cell lines and the IC50 values of a test drug. I want to find a relation between the gene expression profiles of each cell line (nearly 31000 genes) and the IC50 values. My ...
1
vote
2answers
133 views

How to find important features in this problem

I was just thinking how ML techniques can be applied in the retail industry. Suppose we have data from a retailer who deals with apparel and cloth in this format and for each item there are ...
1
vote
2answers
90 views

How to build a predictive model with a billion of sparse features?

I am making a model to learn a dataset which has a big feature number and sparse samples (I am planning to use logistic regression). The feature number can be as big as 1,000,000,000. It is sparse ...