Tagged Questions

Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
1answer
65 views

Feature Normalization/Standardization before or after Feature Selection?

Should the process of feature normalization/standardization be done before or after the feature selection process?
0
votes
0answers
112 views

recursive feature elemination in R with caret

i work with R caret software package to select the most important features from some set of data. My response is a factor of multiple classes (e.g. nominal ...
0
votes
0answers
9 views

Nonlinear functions of other features as new features in SVM model with RBF kernel

Can some one give me some conceptual insight on the potential advantages of disadvantages of adding features that are (nonlinear) functions of existing features in training an SVM model with an RBF ...
2
votes
2answers
123 views

Model Tuning and Model Evaluation in Machine Learning

Despite my readings (on stack 1, 2, or in literature (Cawley, 2010; Japkowicz, 2011)), I don't find a clear procedure for tuning and evaluating a model in a classification task. I want to perform a ...
1
vote
0answers
49 views

Unstable models, repeated crossvalidation, feature selection

I'm still trying to classify few (about 200) samples in a high dimensional feature space (dim=19) into 3 (very unbalanced) classes. I use an implementation of Least Squares SVM with one vs one coding ...
1
vote
2answers
245 views

Best feature selection method for naive Bayes classification

i want to make classification with naive Bayes. I have got about 100 Features. Numerical ones as well as categorical ones. Since i want only the most relevant ones to be included for the ...
0
votes
0answers
55 views

TF-IDF for text classification by taking into account the document class

I am looking for a TF-IDF weighting for text classification (not document ranking/retrieval) which takes also into account the document class. For example let's use the typical spam/not spam ...
1
vote
0answers
27 views

Removing insignificant variables? [duplicate]

Suppose you fit a linear regression model on some data with 10 variables. The F-statistic shows that 3 of them are significant (p < 0.05) , 2 are within trend (0.05 < p < 0.10), and the other ...
1
vote
0answers
71 views

Is this normal to have big value of Chi Squared?

I am using chi squared for feature selection in text classification. However when I compute it I sometime have very big values. Like 100, 1000 or even 20000. Is this normal ? I wonder because I ...
1
vote
1answer
156 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
1
vote
0answers
91 views

How to apply feature selection based on tf-idf threshold

Let's say we have the following matrix (typical VSM example): ...
0
votes
1answer
101 views

Find variables selected for each subset using caret feature selection

I am doing feature selection using the command 'rfe' in the caret package (http://caret.r-forge.r-project.org/featureselection.html). This command uses a metric to find the optimal amount of variables ...
0
votes
0answers
53 views

Dropping predictor variables, based on variable of importance, effect of Random Forest Accuracy

I am trying to use Random Forest to accurately predict forested land cover classes using Landsat 7, climatic and geographical data. I have 23 predictor variables and 1 response variable. When I drop ...
3
votes
2answers
131 views

Feature / attribute selection for k-means or other clustering

It seems to me that in literature it is assumed that one knows which features / attributes to choose to characterize an item in clustering. If I have a database with items which have many attributes, ...
0
votes
1answer
26 views

Importance of Time Features

if you have a time series and you want to do some predictions, what time feature should you use ? lets say we are trying to predict how many people visit a certain website, we have data for the ...
4
votes
2answers
140 views

Feature selection with partial permutation

For feature selection, permutation tests are biased in favor of those categorical variables with a large number of levels [White1994]. Besides, it has been proposed [Deng2011] that partial ...
0
votes
0answers
47 views

Regression by multiple dependent variables with constraints & feature selection

I have a data set of 1000 records. Each record has three dependent variables $y_1, y_2, y_3$ and 100 independent variables $x_1,...,x_{100}$, where the dependent variable $y_i$ satisfies: $0\le y_i ...
2
votes
2answers
376 views

How to select a subset of variables from my original long list in order to perform logistic regression analysis?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
2
votes
1answer
44 views

why feature scaling or weighting is important in surpervised learning?

I can understand feature scaling or weighting is important in unsurpervised learning case, because we want an good representation of "similarity". But why it is also important in surpervised learning ...
2
votes
0answers
56 views

Feature selection: permutation test Vs deleting a variable

In feature selection for predictive models, it is usually applied a permutation test. In this test, all the values of one variable are randomly permuted and the prediction accuracy is extracted for ...
0
votes
0answers
20 views

Comparing F-values of covariates in R

I'm new here and have a question regarding ANOVA in R. I have an ANOVA table like this from running anova(model) in R, where ...
1
vote
0answers
25 views

Is linear kernel SVM performance between features indicative of RBF kernel SVM performance?

I have feature set 1 and feature set 2. If a linear kernel SVM performs better ("better" meaning greater classification accuracy) when using feature set 1, does this guarantee that a properly tuned ...
1
vote
1answer
28 views

Transform a non-monotonic value before decision tree (concrete example)?

Newbie question here. I am building a toy decision tree to differentiate personal names from business, government, or organizational names, like: AAA ENTERPRISES LLC DBA AAA BBB SERVICE SMITH ...
0
votes
0answers
46 views

What method should I use for this optimization / feature selection project

I'm going to describe a problem and I'm not sure how to best solve it. I will describe the situation. When answering please recommend a method and maybe a software library. I'm using Python for my ...
0
votes
1answer
73 views

Values of the weights in Adaboost

I have implemented a simple Adaboost algorithm, using several weak classifiers, and when checking the values computed by it there are alphas with a negative value. Is that possible, or is there a bug ...
0
votes
1answer
60 views

Interpreting results of a factor analysis

I performed factor analysis on R using factanal. Following advice I found on this tutorial, I chose the number of factors as being the number of principal components that capture 90% of the ...
0
votes
1answer
38 views

Choosing a score for factor analysis

I want to perform factor analysis to reduce the number of variables in my dataset (the variables are very redundant). One of the parameters I need to supply to the R code is the number of factors to ...
0
votes
2answers
165 views

R package glmnet: Ridge selects variables

I am using the glmnet package in R. When I set the alpha value = 0, I would expect that no variables are selected. When I look at the coefficients some of them are set to zero. What could be the ...
0
votes
0answers
141 views

Clustering communication patterns to detect multiple identities

I have a data set of communication patterns between chatting agents. Each agent can have multiple profiles or identities. I am interested in developing a way to investigate the similarity between ...
1
vote
1answer
44 views

Feature selection based on information gain papers

I want to apply feature selection based on information gain: I have many features many of which are redundant. I am planning on selecting a feature and then iteratively add features that 'add the more ...
0
votes
0answers
15 views

Are features present in just one sample relevant for SVM learning?

I am building classification and regression SVM (RBF) models where the features for each sample are indicator values(0, 1) for a set of features exhaustively generated from all samples. There are many ...
0
votes
0answers
54 views

How to prove the significance of features in classification?

I have a binary classification problem. I have extracted 500 features from a set of 5000 samples using my domain knowledge. In other words, I have got hand crafted features. I wish to prove that ...
1
vote
1answer
62 views

Number of components in PCA

I believe I have a problem understanding PCA: I would like to use this technique to reduce the number of features of my problem. I originally have 10,000 features and 500 samples. However, the use of ...
3
votes
3answers
478 views

Anomaly detection: what algorithm to use?

Context: I'm developing a system that analyzes clinical data to filter out implausible data that might be typos. What I did so far: To quantify the plausibility, my attempt so far was to normalize ...
1
vote
1answer
497 views

Variablity in cv.glmnet results

I am using cv.glment to find predictors. The set-up I use is as follows: ...
2
votes
1answer
51 views

Feature selection for classifier

I'm using a supervised machine learning algorithm on some big data. There is much more features than observations. To reduce the number of features, I would like to do some feature selection. However, ...
1
vote
0answers
76 views

Feature extraction for customer churn data

I have a customer churn data, and would be implementing algorithms (decision tree, logistic regression, segment analysis). I have doubt on feature extraction procedure though. The training sample has ...
0
votes
1answer
16 views

Identifying feature values that influence an outcome

I have a data set which has data about 1 million people. Data about each person consists of a 'Score' and about 100 features (each of which refers to some characteristic of the person - example - age, ...
2
votes
1answer
39 views

Selecting features manually and proving the rest are redundant

I'm working with a gesture dataset, where each gesture has a variable number of frames, and each frame has the 3d position of 20 joints, so that each gesture is represented by a matrix of size frames ...
0
votes
0answers
22 views

Replacing categorical variables with historic response rate

In Linoff and Berry's "Data Mining Techniques" they mention reducing the number of categorical variables in a classification model by replacing the variable with the historic response rate. "When ...
0
votes
0answers
40 views

Methods for temporal patterns extraction

For example a video or series of images, or usage patterns data on a website, or a univariate time series, is there some flexible methods for extracting patterns of any length, such as head ...
1
vote
1answer
109 views

AICc results in R

I used the model: ...
4
votes
1answer
408 views

Gini decrease and Gini impurity of children nodes

I'm working on the Gini feature importance measure for random forest. Therefore, I need to calculate the Gini decrease in node impurity. Here is the way I do so, which leads to a conflict with the ...
1
vote
3answers
313 views

Feature Selection

I've a facebook users dataset in which each user has a "huge" set of attribute, i.e about 220 attributes like age, hometown, religion, and a set of facebook liked pages to store the users tastes. Now ...
0
votes
0answers
8 views

Redundant feature generation

I'm evaluating a few supervised feature selection algorithms, with a focus on redundancy. As a base correlation statistic, I'm using Mutual Information - so discrete variables are also a focus for ...
1
vote
0answers
12 views

What kinds of features are worth taking from objects on images?

I managed to label and get set of pixels for every single object of interest in the image. Now I'm looking into clustering and am not sure what features are worth using. I currently have two in mind ...
0
votes
0answers
44 views

Feature selection for one class SVM

I have around 300 features, i need to choose features for one class svm. can some one tell me the ideal algorithm for this use case. I know about that for feature selection regularised random ...
1
vote
1answer
95 views

Does Boruta feature selection (in R) take into account the correlation between variables?

I am a bit of a novice in R and feature selection, and have tried the Boruta package to select (diminish) my number of variables (n= 40). I thought that this method also took into account the possible ...
0
votes
0answers
10 views

How to train input feature weights for a path optimization problem?

I am working on a general path optimization problem. As many of you know, once the weights on all nodes are determined, one may solve this problem in many different means. Unfortunately, my raw input ...
1
vote
1answer
26 views

Using selected features from a wrapper algorithm to train another model

I was wondering if it can be useful to use selected features from a wrapper algorithm (for example SVM-RFE) to train another classification model like k-NN or Linear regression.