Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
17 views

Select the best point pair in the 2D grid [duplicate]

Suppose in the 2D space we have an array of points, and each point has a weighting factor, which is a float value ranging from 0 to 1. Each point also has a coordinate in the 2D grid. The following ...
3
votes
2answers
54 views

Why isn't the lasso selected variable even not significant?

I performed a lasso selection using lars::lars for a well normally distributed outcome using a pool of 86 predictors. Here is the plot of the output: The ...
0
votes
1answer
26 views

Handle large set of features using SVM

I have a biological dataset with 30.000 features (genes) and 1000 data points (cells). Basically I have two major classes of cells: 1 and 0 with a distribution of 90/10. Now I am trying to classify ...
0
votes
0answers
16 views

How to deal with categorical features.

Recently I am playing in the famous Big-Data website Kaggle. There is a Display Advertising Challenge. In this competition, you are giving a training file which include huge records. the records is ...
0
votes
0answers
6 views

libsvm with diffrent count of Keypoints [migrated]

I would like to use libsvm for a keypoint detection algorithm. Each keypoint has 36 features, but each sample of an Object has a diffrent count of keypoints... Is it even possible to train with ...
0
votes
0answers
16 views

feature selection for longitudinal data

I have a longitudinal data which looks like this. Number of time points are different for each ID. Y is the binary response variable (take values 0 & 1) and X1-X20 are either continuous or ...
0
votes
0answers
14 views

Comparision of feature selection cost functions

I'm using an optimization algorithm to feature selection besides choosing number of neurons in a neural network. I have 21 features and have ...
0
votes
0answers
23 views

different feature types for classification

There has a data set with several features. One feature is of the type of continuous numerical values; another feature is of the type of categorical values, such as A, B and C. If I want to build a ...
1
vote
0answers
7 views

Does within-group heterogeneity negatively impact random forest classification?

I have two rather conceptual questions about random forest classifiers. Before we get there, I quickly want to lay out the problem I am working on: I have large a large data set consisting of 300 ...
0
votes
0answers
20 views

Triple nested cross validation

I have read several very informative posts including the link about the nested/double cross validation, which can determine (sub)optimal hyperparameter values as well as make an unbiased estimate of ...
0
votes
0answers
13 views

Considering non-i.i.d. covariates in random forests

Random forests are theoretically funded on the assumption that the data are i.i.d. realizations from a multivariate random vector $(X_1, \ldots, X_p, Y)$. Does it make sense to use random forests (for ...
0
votes
0answers
21 views

How to deal with a feature relating to what type of expert labelled the data that becomes unavailable at point of classification?

Essentially I have a data set, that has a feature vector, and label indicating whether it is spam or non-spam. To get the labels for this data, 2 distinct types of expert were used each using ...
1
vote
0answers
30 views

categorical feature ranking

I would like to rank categorical features by the order or importance in a classification/regression setting. Input There are two features, which are survey questions: "how is your mood?": four ...
0
votes
2answers
59 views

Interpreting conflicting results from Random Forest & Logistic Regression?

I am using SKLearn and Statsmodel in python to build a RF and Logistic Regression, respectively. I have a feature that the RF indicates is important (feature importance of 0.202, closely behind #1 ...
0
votes
0answers
27 views

Find linear SVM feature weights using libsvm

I'm trying to use linear SVM to do some feature selection. I'm using libsvm, but I cannot figure out how to find feature weights. The model file created looks something like this: ...
0
votes
0answers
8 views

How can the quality of features be evaluated in high dimensional classification tasks?

I am currently experimenting with on-line symbol recognition for mathematics for my bachelors thesis. I have 369 symbols which I would like to distinguish. There are a lot of preprocessing methods / ...
0
votes
0answers
7 views

chi-squared feature selection

I have a feature vector of size 250 x 35. That is I have 250 images and each image has 35 features in it. I need to do chi-squared test to get the most significant features out of it. anyone know ...
0
votes
1answer
24 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
0
votes
1answer
12 views

Feature selection with a binary dependent variable

Given we have a binary dependent variable and 100s of features and ~50k observations, is there a generally accepted way to trim the features via some type of machine learning concept? I was trying a ...
-1
votes
0answers
18 views

How to: cross validation and scaling features using LibSVM – binary classification problem

I have a matrix of samples in rows and features in columns. I want to train this data matrix using LibSVM. How can I normalize or scale my features before running LibSVM? How can I perform cross ...
0
votes
0answers
13 views

On population variable importance

Consider we run a random forest on $n$ independent realizations of a random vector $(X_1,X_2,X_3,Y)$ assuming $Y$ is a numerical response variable. Let $f$ be the best theoretical classifier defined ...
0
votes
0answers
16 views

Featuring Engineering from Trends in the Training Set

I have a predictive model with just OK performance, and I'm trying to improve it with feature engineering. My question: is it valid to create new features by looking at trends in the training set? ...
0
votes
0answers
9 views

When to cluster features for supervised learning?

I'm doing a project on dog adoption patterns, and I realized that there are many (100 +) different breeds of dogs. I'd like to build a predictive model using breed as covariate, but I'm not sure ...
2
votes
2answers
51 views

What interactions to include in my GLM model?

I realize this might be a too general question, so I'll describe what I'm doing right now first. I'm working for a virtual insurance company and I have this dataset. It has severity (meaning ...
0
votes
1answer
26 views

Caret: customizing feature selection, nested inside cross validation

Using caret, I want to train a SVM classifier and estimate its performance using repeated cross validation. My dataset has a very large number of predictors (300K) and I want to reduce this number ...
0
votes
0answers
5 views

Design a feature with time and presence information

Context: I am working on a decision tree classifier, trying to classify businesses as to whether they are likely to have an event occur (default) in the next 90 days. One input I get is whether, and ...
0
votes
1answer
34 views

How to normalize time series?

This is a general question on normalization of data so that all the variables are within the same range. Why do we normalize data in pattern classification? How to normalize time series which is ...
0
votes
1answer
26 views

Feature Selection using (low) MCC

I have approximately 1200 input parameters that I am trying to whittle down with the following rough process: 1) Fit rbf SVM with n = 1200 parameters and calculate Matthews Correlation ...
0
votes
0answers
8 views

Boruta score goes to minus infinity

I'm running the Boruta algorithm with a $179\times 36$ predictor matrix and a numerical response. Most of the variables have a score going to -Inf. Should I ...
1
vote
0answers
16 views

How to assess the importance of the features which come from intersection of features of the two models?

I have two models from two different data sets. Model 1 contain 50 features and model 2 contain 40 features. the intersection of features of model 1 and 2 is 10. so how can I assess the relative ...
0
votes
0answers
18 views

How many features can be used for classification?

Asked a similar question the other day without an answer Link. I think maybe the question there is too big. Here I want to ask a specific one: 2 Class labels (Binary classification labelled with ...
0
votes
1answer
25 views

Including or discarding a variable based on its frequency

I am trying to dummy code a variable sector which have 20 levels. I am trying to combine some of levels. However I got this situation, if I have a say 2000 observations, how do I decide on the minimum ...
1
vote
2answers
72 views

Methods in R or Python to perform feature selection in unsupervised learning

What are the available methods/implementation in R/Python to discard/select unimportant/important features in data? My data does not have labels (unsupervised). The data has ~100 features with mixed ...
1
vote
1answer
26 views

Feature Selection - Mutual Information with response variable that takes three values

I am trying to calculate Mutual Information scores for Feature Selection. I have successfully implemented the Mutual Information to test each feature against the binary response variable. Each ...
0
votes
0answers
27 views

How many features to overfit the classification?

Recently, I have got some 'strange' comments from the reviewer of my paper. In my paper, I discussed a novel feature extraction method, and then I compared three classification methods for my binary ...
1
vote
1answer
41 views

Why does SAS Enterprise Miner keep all dummy variables for a coded categorical variable in stepwise logistic regression?

SAS Enterprise Miner nicely creates coded dummy variables for any categorical variables used in a logistic regression model. When it performs a variable selection using stepwise sequential selection ...
1
vote
0answers
31 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ ...
1
vote
0answers
21 views

Setting up feature vectors

I am working on a classification project and I want to use SVM's and/or Clustering Algs. What I am having trouble with is deciding how to set up my feature vectors. I have already decided what my ...
0
votes
0answers
17 views

Is subtracting the mean from PCA necessary when using an SVD result that is feature scaled?

I've applied SVD to the original data matrix and eliminated insignificant columns and rows from U and V^T respectively using the Sigma values. I multiplied together my optimized U, Sigma, and V^T ...
0
votes
0answers
11 views

Feature selection: all features vs a subset of them

I am doing a binary classification. The dataset has 3000 samples, and each sample has 10 features. But I find that the performance of using all 10 features is almost the same as that of using only the ...
0
votes
0answers
16 views

How to interpret merits in Weka with ChiSquaredAttributeEval and SVMAttributeEval?

I want to interpret the goodness of attributes using feature selection with 10-fold cross validation. With ChiSquared I get something like this (deletet attributes with merrit was 0 in all folds): ...
2
votes
0answers
36 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
0
votes
0answers
19 views

Feature selection from wavelet transformation in R

I am new to wavelets. Currently, I am developing a prediction model using time series data. I am using the wavelets package in R. I am taking part of the time ...
1
vote
1answer
81 views

Maximum Entropy Model for classification, what to use as context & feature?

I'm building a Maximum Entropy Model to classify some text, based on paper "A Maximum Entropy Approach to Natural Language Processing" by Berger et.al. It's similar to POS tagging. Below is some ...
0
votes
1answer
27 views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
1answer
88 views

Feature selection : how to select the Information Gain threshold?

I am trying to use Information Gain to select features when classifying text with a Support Vector Machine. For each word in our training data, we computed its information gain. Then, we should keep ...
0
votes
1answer
43 views

Feature Normalization/Standardization before or after Feature Selection?

Should the process of feature normalization/standardization be done before or after the feature selection process?
0
votes
0answers
39 views

recursive feature elemination in R with caret

i work with R caret software package to select the most important features from some set of data. My response is a factor of multiple classes (e.g. nominal ...
0
votes
0answers
7 views

Nonlinear functions of other features as new features in SVM model with RBF kernel

Can some one give me some conceptual insight on the potential advantages of disadvantages of adding features that are (nonlinear) functions of existing features in training an SVM model with an RBF ...
2
votes
2answers
90 views

Model Tuning and Model Evaluation in Machine Learning

Despite my readings (on stack 1, 2, or in literature (Cawley, 2010; Japkowicz, 2011)), I don't find a clear procedure for tuning and evaluating a model in a classification task. I want to perform a ...