Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

20
votes
4answers
1k views

How can SVM 'find' an infinite feature space where linear separation is always possible?

What is the intuition behind the fact that an SVM with a Gaussian Kernel has infinite dimensional feature space?
2
votes
0answers
31 views

Variable selection with multi-variate time series

I currently have a data.frame with 273 variables and 94 rows: ...
1
vote
1answer
20 views

Categorical with many levels for NN

I have a dataset which lists the amount of seconds a user held a session by browser_type. For example: ...
1
vote
1answer
382 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
2
votes
1answer
66 views

Computing Cross-Validation Errors for Subset Selection: error in standard code in the literature?

I am currently trying to understand how to use cross-validation in order to choose among the "best" subsets of different sizes returned by the R function regsubsets (regsubsets returns the "best" ...
0
votes
1answer
46 views

MATLAB function TreeBagger() (Random Forest classification) and different number of variables

I am using MATLAB function TreeBagger() for Random Forest classification, for an assignment. It gives error when the number of variables of the Test data is different from the number of variables of ...
0
votes
0answers
64 views

What are appropriate feature selection techniques for binary features?

Suppose that we have binary features (+1 and -1 or 0 and 1). We have some well-knows feature selection techniques like Information Gain, ...
0
votes
1answer
34 views

Is feature complementarity different from feature interaction?

I am writing a conference paper in which I have a sentence like "...complementary/interactive features...". This sentence ...
1
vote
1answer
440 views

Is F test used for feature selection only for features with numerical and continuous domain?

F statistic/test can be used for feature selection, e.g. from http://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.f_classif.html#sklearn.feature_selection.f_classif ANOVA ...
0
votes
0answers
28 views

PCA Transforming back to original space from a subset of principal components [on hold]

I applied PCA on a 12000 x 500 data set (12000 data points with 500 features). PCA gave me 12000 x 20 data (20 features). Is there any way to transform the results back to the original data space? (To ...
4
votes
1answer
120 views

Use of Random Forests for variable importance as preprocess before another analysis

the question Demonstrate the speed and accuracy of properly applied 'Random Forest' as a variable importance selection tool especially in handling very large data against alternative approaches such ...
2
votes
2answers
460 views

Event Prediction through Machine Learning

I have a large data set consisting of ca. 40 categorical data items and a few interval data items (real numbers, less than 5 such items). Most categories should have a lot of values that repeat ...
0
votes
0answers
12 views

How do I have select features which are influential for prediction?

I have a dataset which has dependent variable(label) as possible destinations and independent variable(features) as age,language, gender and many other categorical variables. How do i find which are ...
0
votes
0answers
28 views
0
votes
0answers
8 views

Identifying important differences between supervised learning datasets

The training data in a multi-class supervised learning task shows a significant dependence on time that is apparently not captured well by my learners. Specifically, the two learners I used (OvR ...
0
votes
1answer
23 views

Computing the Interaction gain. Is there an Error in the infotheo package in R?

In order to implementing a certain feature selection method for a classification problem I need to estimate the the interaction the interaction gain between two features and the target variable which ...
0
votes
1answer
36 views

VAR/VECM/ARDL optimal lag selection

Question 1: Is it necessary to consider AIC and the BIC criteria when selecting the lag for a VAR, VECM or ARDL model OR can I use something else? Example: Can I pick 12 lags because the model simply ...
0
votes
1answer
129 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
1
vote
1answer
95 views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
3
votes
1answer
587 views

Advice for a sparse high-dimensional regression strategy

I have a regression problem where I would like to predict values given several thousand sparse features. The general data set is an $n \times m$ matrix where each row contains a sample with a value I ...
2
votes
1answer
1k views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
0answers
17 views

Choosing between feature selection and regularization to overcome over-fitting in categorical regression

In order to overcome over-fitting during a regression process over categorical features, one can either 1) Apply L1/L2/Elastic regularization during the regression, for example as answered here ...
0
votes
0answers
10 views

How to check feature relevance and representativeness?

How to check whether features from one domain are relevant in other domain? How to evaluate whether features are representing that domain?
7
votes
5answers
512 views

How to prepare/construct features for anomaly detection (network security data)

My goal is to analyse network logs (e.g., Apache, syslog, Active Directory security audit and so on) using clustering / anomaly detection for intrusion detection purposes. From the logs I have a lot ...
2
votes
1answer
264 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
0
votes
0answers
35 views

Logistic regression - variable transformation

I have a continues variable(EntropyDistanceFromMean) which I would like to use in a logistic regression, the problem with that variable is that it starting to effect the output (MQL) found on the ...
0
votes
0answers
7 views

Content Based Document classification

I have a corpus of 10 million resumes. I want to add tags to these resumes like Software Engineer, Data Scientists, ...
1
vote
1answer
156 views

Most Parsimonious Elastic Net Model - choosing $\alpha$ and $\lambda$

How do I calculate which Elastic Net model is the most regularized/parsimonious? I am recreating GLMnet in another language as an exercise. I want to do a grid search over several values of alpha and ...
2
votes
2answers
156 views

Random Forests for predictor importance (Matlab)

I'm working with a dataset of approx 150,000 observations and 50 features, using SVM for the final model. To trim down the feature count, I decided to look into using RF so SVM optimization doesn't ...
0
votes
1answer
158 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
1
vote
0answers
20 views

Variable importance in regression with large number of missing values

I have a dataset with multiple (approximately 20) categorical and ordinal predictors and a numerical outcome and I am trying to understand which and how each of these predictors affect the outcome ...
1
vote
0answers
25 views

grouping attributes in RF and GBM

i have a dataset with 1000 samples and ~11k features (SNP markers). i have identified 100 additional binary features describing the markers themselves so i have a ...
2
votes
3answers
236 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
0
votes
0answers
33 views

What happens to multi-category variables in algorithms like Random Forest that sample the feature space?

Suppose I have a multi-level categorical variable like color (say, with 7 levels). Some software libraries only allow numeric matrices to train models, so we need ...
0
votes
1answer
42 views

In regression, what is the limit of independent variables?

After having taken the Coursera Data Science specialization, I am faced with my first "practical" problem which I plan on solving with some sort of regression. This is my first real world, ...
1
vote
2answers
56 views

What does “the process that generates the data” mean? and How does feature selection help in recovering it?

In [1], one of the motivations to use feature selection is stated to be: "to gain knowledge about the process that generated the data". What does this "process" actually mean? and How does feature ...
3
votes
2answers
55 views

random forests for optimal variable selection/feature selection

Gurus, I just came across this tutorial (http://blog.datadive.net/selecting-good-features-part-iii-random-forests/) about using "random forests" for optimal variable selection/feature selection. The ...
2
votes
1answer
198 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
0
votes
0answers
23 views

Feature selection using RFE in SVM kernel (other than linear eg rbf, poly etc)

At this link, there is an example of finding feature ranking using RFE in SVM linear kernel. If I want to check feature ranking in other SVM kernel (eg. rbf, poly etc).How to do it? I have changed ...
0
votes
1answer
28 views

Split dataset by categorical variable or use as a dummy/factor variable?

I'm looking for any sort of best practice or ways to go about this situation. Often I come across datasets that have a categorical variable that I am tempted to split off the main dataset into ...
2
votes
1answer
133 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
0
votes
2answers
42 views

is linear regression/polynomial regression sensitive to irrelevant features/noise

is linear regression/polynomial regression sensitive to irrelevant features/noise will their respective weights/coefficients be automatically be tuned down? or is it a straight nail in the coffin? ...
3
votes
2answers
69 views

Cross validated penalized logistic regression - one standard deviation rule

I am new to this topic and would like to understand it better. I want to build a binary classifier based on penalized logistic regression. I have 10 features and 23 observations: 16 from class "0" and ...
0
votes
0answers
20 views

Analysis of wrapper feature selection ouptput in Weka

I am using Weka to select important features from a dataset. I am using the wrapper method in this application. I chose a decision tree (j.48) for my classifier and Genetic search for the search ...
8
votes
1answer
2k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
1
vote
3answers
268 views

Evaluating features and similarity measures

I am currently developing a classifier, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
0
votes
2answers
40 views

Feature selection clustering customer segmentation

based on customer data I want to perform a clustering using different clustering algorithms (K-Means, Expectation Maximization, etc.) in R. The most attributes were engineered pursuing the goal to be ...
0
votes
1answer
14 views

Using SVM's to output binary 0 or 1 to data

I am using a very large ~700,000 sample training set and ~700,000 sample testing set and training an SVM with the training set. When I run the SVM (SciKit-Learn) on the testing set it outputs only ...