Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

2
votes
1answer
280 views

Multiclass classification with SVM a question about the feature vectors

I was told I had to direct my machine learning questions to this site. So here it goes. I'm trying to do Multiclass classification with SVM. I have 7 classes. Now I was wondering if the following is ...
3
votes
1answer
82 views

Changing variable values and examine the outcome difference between the altered and original data

I recently read an approach which is used to find the effect of changing an independent variable. They are doing a classification problem, so each data row (or record) is associated with an outcome ...
-1
votes
1answer
132 views

How logical to select features with respect to the correlation matrix and weigthing?

Is it logical to name low correlated features as valuable and choosing the low corelated ones for classification? Or it depends on the algorithm used for the purpose? How do I need to interpret a ...
1
vote
0answers
90 views

Overfitting a linear Linear Discriminant Function

I am estimating a Linear Discriminant function with 250 input variables over 4000 data records. Should I consider feature selection, am I over fitting the model? How do I know when feature selection ...
4
votes
2answers
130 views

Random search for the optimal number of input features and optimal number of hidden layers for a MLP?

I've performed a random search in hypothesis space $$\{(c,h)| c \in U[1,256]; h\in U[1,100];c \in \mathrm{Z} \text{ and } h \in \mathrm{Z}\}$$ that defines the parameters of a standard multilayer ...
2
votes
1answer
241 views

Selecting optimal number of input features and optimal number of hidden layers for a MLP?

What is the best way to select parameters for a binary neural network classifier? More specifically I have 265 features ranked according to Mutual Information Criterion. I have to determine the ...
2
votes
2answers
99 views

Can feature selection be considered a way to observe relationship between variables like correlation?

In correlation we can observe relationship between a pair of variables, let me call it X1 and Y. Now, considering I have the predicting variables X1, X2, ..., Xn and the variable Y. Does the ...
1
vote
1answer
241 views

highly correlated features and high ranking

I am classifying different texts and I wondering about some features that are highly correlated. I have 49 features. Some features are absolute counters (integers) but most features are relative ...
3
votes
1answer
349 views

Why can't Bayesian variable selection be used with categorical variables with more than 2 levels?

I am reading this article which is the first approach on Bayesian variable selection. In the discussion section it says that one of the major limitations of the particular method is that it cannot be ...
4
votes
2answers
511 views

Why does increasing the number of features reduce performance?

I'm trying to gain an intuition as to why increasing the number of features could reduce performance. I'm currently using an LDA classifier which performs better bivariately among certain features ...
3
votes
0answers
204 views

Sensible to include ratio as a variable in logistic regression?

I'm creating a generalised linear regression using a binomial link function for two variables A and B. From looking at the data it appears that A/B may have discriminatory effect. Is it sensible to ...
6
votes
2answers
729 views

Fisher Distance for feature selection

I'm currently working for EEG signal classification from 3 electrodes. I want to have a simple feature selection algorithm that is independent with the classification process. From the feature ...
4
votes
2answers
3k views

Number of trees for Random Forest optimization using recursive feature elimination

How many trees would you suggest to pick to perform recursive feature elimination (RFE) in order to optimize Random Forest classifier (for binary classification problem). My dataset is very ...
0
votes
0answers
203 views

What are the good algorithms for feature extraction for large dataset?

I have KDD dataset for detecting fraud actions on networks but it has millions of lines and >20 feature columns. Thus it is not viable to process all these on my personal computer. I am thinking about ...
5
votes
5answers
2k views

Is using the same data for feature selection and cross-validation biased or not?

We have a small dataset (about 250 samples * 100 features) on which we want to build a binary classifier after selecting the best feature subset. Lets say that we partition the data into: Training, ...
1
vote
0answers
306 views

Adaboost feature weight calculation

I thought I understood Adaboost, until code analysis made me realize that sample_weight is not an array of the feature weights... and after further investigation I am left confused as to how ...
7
votes
3answers
12k views

How does one interpret SVM feature weights?

I am trying to interpret the variable weights given by fitting a linear SVM. (I'm using scikit-learn): ...
3
votes
2answers
411 views

How to do feature selection for learning from positive and unlabeled examples?

I have a binary classification task for German webpages for which I only have positive examples. That is why I use learning from positive and unlabeled examples as described on this page, also known ...
3
votes
3answers
2k views

How to reduce the number of variables in cluster analysis?

I've got 10 (yes, only 10) cases over 1000 variables (e.g. measurements of concentrations of 1000 different compounds at 10 different time points). I can group these cases into 3 clusters in ...
1
vote
2answers
774 views

Clustering time series with wavelets in R

Can discrete wavelet trasform be used for feature extraction from time series in order to cluster them? Any R code how to do this will be appreciated.
2
votes
0answers
123 views

Random forest like techniques (bagging, random feature subset) for SGD methods

Are there any well-known results/tools/literature on using bagging and random feature subset selection for regression or SGD-based methods?
5
votes
6answers
852 views

What machine learning algorithms are good for estimating which features are more important?

I have data with a minimum number of features that don't change, and a few additional features that can change and have a big impact on the outcome. My data-set looks like this: Features are A, B, C ...
6
votes
1answer
1k views

If p > n, the lasso selects at most n variables

One of the motivations for the elastic net was the following limitation of LASSO: "In the p > n case, the lasso selects at most n variables before it saturates, because of the nature of the convex ...
8
votes
4answers
3k views

Low classification accuracy, what to do next?

So, I'm a newbie in ML field and I try to do some classification. My goal is to predict the outcome of a sport event. I've gathered some historical data and now try to train a classifier. I got around ...
5
votes
3answers
560 views

Can I perform an exhaustive search with cross-validation for feature selection?

I have been reading some of the posts about feature selection and cross-validation but I still have questions about the correct procedure. Suppose I have a dataset with 10 features and I want to ...
2
votes
1answer
429 views

Advice for a sparse high-dimensional regression strategy

I have a regression problem where I would like to predict values given several thousand sparse features. The general data set is an $n \times m$ matrix where each row contains a sample with a value I ...
2
votes
1answer
149 views

What properties of a text makes it a spam/bad question?

I'm trying to identify numeric properties of a text message that make it a spam or, more specifically, a bad question on sites like this one. For example, would things like capital letter density ...
0
votes
1answer
187 views

R package for feature set algorithm selection

I want to train a binary classification NN and part of this will require data pre-processing. However, I have a choice of which pre-processing algorithm to use. Of course I'd like to choose that one ...
1
vote
0answers
388 views

Feature selection for SVM and Maximum Entropy

In text classification problems where the number of features >> number of documents, is it useful to perform feature selection with filters (e.g. Information Gain) when using Naive Bayes. However, ...
4
votes
0answers
168 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
8
votes
3answers
2k views

Feature selection using mutual information in Matlab

I am trying to apply the idea of mutual information to feature selection, as described in these lecture notes (on page 5). My platform is Matlab. One problem I find when computing mutual information ...
6
votes
1answer
1k views

Feature selection and parameter tuning with caret for random forest

I have data with a few thousand features and I want to do recursive feature selection (RFE) to remove uninformative ones. I do this with caret and RFE. However, I started thinking, if I want to get ...
22
votes
5answers
4k views

Detecting significant predictors out of many independent variables

In a dataset of two non-overlapping populations (patients & healthy, total $n=60$) I would like to find (out of $300$ independent variables) significant predictors for a continuous dependent ...
5
votes
1answer
960 views

Variable selection with LASSO

I am trying to fit a predictive gene-based model in survival analysis. My question is: Can I use LASSO as a variable selection method, and then run a multivariate Cox regression to get the ...
6
votes
3answers
1k views

How to avoid overfitting when using crossvalidation within Genetic Algorithms

This is a long set-up, but the pure intellectual challenge will make it worthwhile I promise ;-) I have marketing data where there is a treatment and a control (i.e a customer gets no treatment). The ...
3
votes
2answers
412 views

Variable analysis in multiple linear regression

I'm investigating how some weather variables (15) affect electricity demand in a specific area during the last 20 years. I was thinking to perform the following steps: 1. Perform Multiple Linear ...
3
votes
1answer
553 views

Are randomForest variable importance values comparable across same variables on different dates?

Are randomForest variable importance comparable across same variables on different dates? I have a data array X which is of size $T\times N\times K$, where $T=1500$, $N=1500$ and $K=10$. ...
6
votes
1answer
3k views

What's the forward stagewise regression algorithm?

Maybe it's just that I'm tired, but I'm having trouble trying to understand the Forward Stagewise Regression algorithm. From "Elements of Statistical Learning" page 60: Forward-stagewise ...
2
votes
0answers
275 views

Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?

I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
0
votes
1answer
130 views

Lucene-based text feature construction

When doing the feature construction for text mining, does Lucene has a better performance in terms of classification/clustering result than the traditional bag-of-word approach?
15
votes
3answers
279 views

Is building a multiclass classifier better than several binary ones?

I need to classify URLs into categories. Say I have 15 categories that I'm planning to zero down every URL to. Is a 15-way classifier better? Where I have 15 labels and generate features for each ...
4
votes
1answer
1k views

When does LASSO select correlated predictors?

I'm using the package 'lars' in R with the following code: ...
0
votes
1answer
79 views

Feature selection given non-normal variables?

I'm trying to reduce noise (improve separability) among groups in a data set with 26 numerical variables and 10.000 samples. Each sample is a chemical profile, with each variable indicating the ...
3
votes
1answer
102 views

How to determine significant subgroups of data inputs? [duplicate]

I have a large $(10000 \times 5001)$ table representing $10000$ samples and $5001$ different features of these samples. One of these features represents an output variable of each sample. In other ...
14
votes
3answers
998 views

Model stability when dealing with large $p$, small $n$ problem

Intro: I have a dataset with a classical "large p, small n problem". The number available samples n=150 while the number of possible predictors p=400. The outcome is a continuous variable. I want ...
10
votes
2answers
3k views

Finding the best features in interaction models

I have list of proteins with their feature values. A sample table looks like this: ...
3
votes
1answer
163 views

Choosing the number of features

I remember reading about a general rule of thumb for choosing number of features. It was something like $\sqrt{\log_2(n)}$, where $n$ is the number of samples or $\log_2(n)$. Does anyone remember the ...
4
votes
1answer
3k views

Information gain as a feature selection for 3-class classification problem

I am facing a sentiment analysis task where I am using Naive Bayes to classify documents as Positive, Negative or Neutral. I have thought of using Information Gain as my filter for feature selection. ...
1
vote
0answers
279 views

Guassian Process Regression - feature selection

I'm using guassian process regression to do some modeling. One issue I'm encountering is feature selection for some of my models, which often have many relevant features. I'm not sure what the best ...
2
votes
2answers
589 views

Variable selection in large datasets

I'm looking for an overview of some methods of variable selection. I use datasets with around 6000 variables (the level of missing values is satisfying i.e. there are no variables with 100% missing ...