Methods and principles of selecting a subset of attributes for use in further modelling
5
votes
0answers
755 views
How do you select variables in a regression model?
The traditional approach to variable selection is to find variables that contribute the most to predicting a new response. Recently I learned of an alternative to this. In modeling variables that ...
4
votes
0answers
338 views
Feature selection and parameter tuning with caret for random forest
I have data with a few thousand features and I want to do recursive feature selection (RFE) to remove uninformative ones. I do this with caret and RFE. However, I started thinking, if I want to get ...
3
votes
0answers
62 views
How does LASSO select among collinear predictors?
I'm looking for an intuitive answer why a GLM LASSO model selects a specific predictor out of a group of highly correlated ones, and why it does so differently then the best subset feature selection.
...
3
votes
0answers
90 views
Fast algorithm for variable selection
The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations.
The data is sparse, that is, many of the variables has many zeros and few positive ...
3
votes
0answers
111 views
Sensible to include ratio as a variable in logistic regression?
I'm creating a generalised linear regression using a binomial link function for
two variables A and B. From looking at the data it appears that A/B may have
discriminatory effect. Is it sensible to ...
3
votes
0answers
121 views
Variable Selection One by One vs Simultaneously
The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
3
votes
0answers
237 views
When does LASSO select correlated predictors?
I'm using the package 'lars' in R with the following code:
...
3
votes
0answers
505 views
How exactly does Chi-square feature selection work?
I know that for each feature-class pair, the value of the chi-square statistic is computed and compared against a threshold.
I am a little confused though. If there are $m$ features and $k$ classes, ...
3
votes
0answers
444 views
Clustering probability distributions - methods & metrics?
I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
2
votes
0answers
31 views
Variable selection / Dataset reduction for large datasets (in R)
I'm working on a behavoural scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is ...
2
votes
0answers
34 views
Why do correlations matter in simulating data to compare classifiers when p >> N?
In genomics and computational biology, expression data sets contain a much larger number of features (p) than the number of observations (N). I wanted to simulate data where p>>N to compare the ...
2
votes
0answers
111 views
LASSO vs AIC for feature selection with the Cox model
I have some questions about the Lasso.
After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
2
votes
0answers
71 views
Multiclass classification with SVM a question about the feature vectors
I was told I had to direct my machine learning questions to this site. So here it goes.
I'm trying to do Multiclass classification with SVM. I have 7 classes. Now I was wondering if the following is ...
2
votes
0answers
64 views
Random forest like techniques (bagging, random feature subset) for SGD methods
Are there any well-known results/tools/literature on using bagging and random feature subset selection for regression or SGD-based methods?
2
votes
0answers
122 views
Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?
I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
2
votes
0answers
81 views
Feature construction for text mining
In the text mining, besides N-gram model, what are the state-of-art models for building feature space while capturing the dependence among the different words, or capturing the semantic meaning in the ...
2
votes
0answers
81 views
Non-linear (e.g. RBF kernel) SVM with SCAD penalties implementation
Is there one? I think there's a penalizedSVM package in R but it looks to use a linear kernel. Can't quite tell from the documentation. If it's linear, is there a R package that lets me calculate the ...
2
votes
0answers
291 views
Error using rfe in caret package in R
I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
2
votes
0answers
93 views
New development in variable selection in clustering using MCMC?
The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
2
votes
0answers
142 views
What should I do to compare different sets of data?
I am a beginner in statistics, and I want to learn machine learning :). Therefore, I have gathered some sample data to practice. But, the problem is I want to create a feature (or attribute), which is ...
1
vote
0answers
16 views
Select best set of binary variables for clustering known sample labels
I have a set of samples, for which I know the "true groups". For this samples I have about 200 binary variables, I would like to know a method to select the subset of variables, that gives me a ...
1
vote
0answers
16 views
Using MatchIt to match groups in a retrospective analysis
I am interested in using the R package MatchIt to preprocess my data as to obtain matched groups based on a predefined treatment variable. However I am facing a few issues.
The first issue is that ...
1
vote
0answers
13 views
Is there any meta-approach for variable selection based of measures of similarity between each two variables?
Is there any meta-approach ( or mayby I should say universal approach which works with different measures ) for variable selection which is based on similarity matrix which every entry ...
1
vote
0answers
19 views
Performance worse with new observations
I come from the computer science area but am new to machine learning / stats, so this question may be fundamental and easy.
I have time-series data (biological data), and, without getting into the ...
1
vote
0answers
39 views
True and false discovery rate in variable selection
I have a question about how I can calculate true and false positive rate in a simulation study?
I have seen some articles and thesis by different definitions. One of them is the following one:
...
1
vote
0answers
38 views
How do I know if the differences in ICs among candidate models are significant?
I'm doing some exploratory modelling on a data set with 29 covariates and an additional 11 variables that are of interest to my research question.
My strategy is to develop a model with a subset of ...
1
vote
0answers
70 views
Number of features and accuracy/f-measure - Who can explain this results?
I'm performing a binary sentiment classification (positive/negative) based on a Naive Bayes classificator and a SVM. To select top k features I use the MRMR algorithm. The model is trained using a ...
1
vote
0answers
259 views
How to select the best variables by RandomForest in R?
I have a table of mRNA levels of my target gene and it's transcription factors in many different condition. What I want to do is to select the most important ...
1
vote
0answers
104 views
Bootstrap randomized Lasso selection for a Cox model
I'm interested in variable selection for a cox proportional hazards model.
I've read this article which slightly favours randomized bootstrap lasso selection over bootstrap lasso selection since it ...
1
vote
0answers
80 views
time series with different length: feature extraction and classification
I have a binary classification problem, where each data point is multi-channel time-series, which can be represented as a matrix TxF, where T is the time-series length, and F as the channels number.
T ...
1
vote
0answers
65 views
Univariate feature ranking in classification
Scikit-learn has function to evaluate the F-statistics for univariate feature importance feature selection. According to the web page they are calculating ANOVA F value.
If I understood correctly, ...
1
vote
0answers
82 views
Feature importance
The extremely randomized trees classifier (scikitlearn) provides a (multivariate) feature importance measurement Ensemble methods/feature importance evaluation.
For each feature, the classifier ...
1
vote
0answers
115 views
Kernel in PenalizedSVM R package
There is not option to select kernel in penalizedSVM R package. What kernel do they use?
Is there some other R package with penalized SVM methods where I can choose various kernels?
1
vote
0answers
47 views
Overfitting a linear Linear Discriminant Function
I am estimating a Linear Discriminant function with 250 input variables over 4000 data records. Should I consider feature selection, am I over fitting the model? How do I know when feature selection ...
1
vote
0answers
110 views
Adaboost feature weight calculation
I thought I understood Adaboost, until code analysis made me realize that sample_weight is not an array of the feature weights... and after further investigation I am left confused as to how ...
1
vote
0answers
133 views
Guassian Process Regression - feature selection
I'm using guassian process regression to do some modeling. One issue I'm encountering is feature selection for some of my models, which often have many relevant features. I'm not sure what the best ...
1
vote
0answers
164 views
SVM importance of predictor variables
I am building a model in R using support vector machine (SVM) with KBF kernel.
The model seems to work quite well. I would like to assess the relative importance of predictor variables.
Can anyone ...
1
vote
0answers
52 views
What are some common tools, initial approaches to data in a prediction problem when facing too many predictors?
If one is given several hundred features (of both categorical and continuous type) what are some approaches to determining which features to keep or even drop? Data as such is difficult to visualize ...
1
vote
0answers
102 views
References about univariable vs multivariable variable selection
Suppose I have variables $X_j$, $j=1,\ldots,p$, some of which are correlated, and some continuous output $y$.
I want to rank the variables by importance. One way is to do an association test of each ...
1
vote
0answers
117 views
Measures of predictive power of attributes in data mining
What are the most widely used measures of predictive power of attributes in scoring models?
Motivation: I have a lot of attributes, more than I can study by myself and I want to select somehow the ...
0
votes
0answers
7 views
How to find the tendency of a feature in decision trees?
I've trained a decision tree binary classifier and I have the most informative features based on the sum of information gain weighted by the number of samples at the node (scikit-learn ...
0
votes
0answers
23 views
Comparison of feature selection algorithms
I have to compare some feature selection algorithms. Please suggest some study materials from where I can get the idea about the comparison method.
0
votes
0answers
14 views
Choosing prior distribution in LDA
how do you set prior distribution of K in LDA and can it be used for feature selection to improved selection accuracy of document. Abbey
0
votes
0answers
12 views
Which features to extract for classifying segmented zones of an image into two classes “handwritten text” and “graphics”
I have some chemical document images segmented into different zones, some zones represents "handwritten text" and others represent "graphics". I want to classify this zones into two classes, one for ...
0
votes
0answers
36 views
How many attributes to select for classification
I am aware that this question is vague, but after reading multiple standard books of the community, I am still wondering about the following applied problem.
Lets say I have a dichotomy problem and ...
0
votes
0answers
42 views
similarity of feature vector
What is a better measure of feature vector similarity, Euclidean distance or dot product/cosine similarity? I've read about cosine similarity being used with document vectors, but I've also seen ...
0
votes
0answers
22 views
Selecting features for text classification
I've been trying to improve the accuracy of my sentiment classifier. It basically uses n-grams of words as features for a Naive Bayes classifier. I've been using mutual information to select features ...
0
votes
0answers
14 views
How to get average two features from the front and next impressions of the current fingerprint impression?
I am working on a Fingerprint recognition scheme using Assembling Invariant Moments. At the time of feature extraction ROIs would be failed to acquire for computing the features, so we chose to ...
0
votes
0answers
46 views
Issues with sequential feature selection
I am trying to do some feature selection in gene expression data with 22215 features. I followed the tutorial here.
I initially applied filter method(ttest) to select the features having the best p ...
0
votes
0answers
54 views
How to perform step() when n < p in R?
I am trying to perform stepwise regression for variable selection in R.
In matlab, the stepwisefit function is able to work in ...
