Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
18 views

Is subtracting the mean from PCA necessary when using an SVD result that is feature scaled?

I've applied SVD to the original data matrix and eliminated insignificant columns and rows from U and V^T respectively using the Sigma values. I multiplied together my optimized U, Sigma, and V^T ...
0
votes
0answers
11 views

Feature selection: all features vs a subset of them

I am doing a binary classification. The dataset has 3000 samples, and each sample has 10 features. But I find that the performance of using all 10 features is almost the same as that of using only the ...
0
votes
0answers
25 views

How to interpret merits in Weka with ChiSquaredAttributeEval and SVMAttributeEval?

I want to interpret the goodness of attributes using feature selection with 10-fold cross validation. With ChiSquared I get something like this (deletet attributes with merrit was 0 in all folds): ...
2
votes
0answers
37 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
0
votes
0answers
21 views

Feature selection from wavelet transformation in R

I am new to wavelets. Currently, I am developing a prediction model using time series data. I am using the wavelets package in R. I am taking part of the time ...
1
vote
1answer
92 views

Maximum Entropy Model for classification, what to use as context & feature?

I'm building a Maximum Entropy Model to classify some text, based on paper "A Maximum Entropy Approach to Natural Language Processing" by Berger et.al. It's similar to POS tagging. Below is some ...
0
votes
1answer
29 views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
1answer
119 views

Feature selection : how to select the Information Gain threshold?

I am trying to use Information Gain to select features when classifying text with a Support Vector Machine. For each word in our training data, we computed its information gain. Then, we should keep ...
0
votes
1answer
47 views

Feature Normalization/Standardization before or after Feature Selection?

Should the process of feature normalization/standardization be done before or after the feature selection process?
0
votes
0answers
53 views

recursive feature elemination in R with caret

i work with R caret software package to select the most important features from some set of data. My response is a factor of multiple classes (e.g. nominal ...
0
votes
0answers
8 views

Nonlinear functions of other features as new features in SVM model with RBF kernel

Can some one give me some conceptual insight on the potential advantages of disadvantages of adding features that are (nonlinear) functions of existing features in training an SVM model with an RBF ...
2
votes
2answers
97 views

Model Tuning and Model Evaluation in Machine Learning

Despite my readings (on stack 1, 2, or in literature (Cawley, 2010; Japkowicz, 2011)), I don't find a clear procedure for tuning and evaluating a model in a classification task. I want to perform a ...
1
vote
0answers
33 views

Unstable models, repeated crossvalidation, feature selection

I'm still trying to classify few (about 200) samples in a high dimensional feature space (dim=19) into 3 (very unbalanced) classes. I use an implementation of Least Squares SVM with one vs one coding ...
0
votes
2answers
78 views

Best feature selection method for naive Bayes classification

i want to make classification with naive Bayes. I have got about 100 Features. Numerical ones as well as categorical ones. Since i want only the most relevant ones to be included for the ...
0
votes
0answers
27 views

TF-IDF for text classification by taking into account the document class

I am looking for a TF-IDF weighting for text classification (not document ranking/retrieval) which takes also into account the document class. For example let's use the typical spam/not spam ...
1
vote
0answers
27 views

Removing insignificant variables? [duplicate]

Suppose you fit a linear regression model on some data with 10 variables. The F-statistic shows that 3 of them are significant (p < 0.05) , 2 are within trend (0.05 < p < 0.10), and the other ...
1
vote
0answers
66 views

Is this normal to have big value of Chi Squared?

I am using chi squared for feature selection in text classification. However when I compute it I sometime have very big values. Like 100, 1000 or even 20000. Is this normal ? I wonder because I ...
1
vote
1answer
100 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
1
vote
0answers
50 views

How to apply feature selection based on tf-idf threshold

Let's say we have the following matrix (typical VSM example): ...
0
votes
1answer
54 views

Find variables selected for each subset using caret feature selection

I am doing feature selection using the command 'rfe' in the caret package (http://caret.r-forge.r-project.org/featureselection.html). This command uses a metric to find the optimal amount of variables ...
0
votes
0answers
37 views

Dropping predictor variables, based on variable of importance, effect of Random Forest Accuracy

I am trying to use Random Forest to accurately predict forested land cover classes using Landsat 7, climatic and geographical data. I have 23 predictor variables and 1 response variable. When I drop ...
3
votes
2answers
94 views

Feature / attribute selection for k-means or other clustering

It seems to me that in literature it is assumed that one knows which features / attributes to choose to characterize an item in clustering. If I have a database with items which have many attributes, ...
0
votes
1answer
23 views

Importance of Time Features

if you have a time series and you want to do some predictions, what time feature should you use ? lets say we are trying to predict how many people visit a certain website, we have data for the ...
4
votes
2answers
127 views

Feature selection with partial permutation

For feature selection, permutation tests are biased in favor of those categorical variables with a large number of levels [White1994]. Besides, it has been proposed [Deng2011] that partial ...
0
votes
0answers
36 views

Regression by multiple dependent variables with constraints & feature selection

I have a data set of 1000 records. Each record has three dependent variables $y_1, y_2, y_3$ and 100 independent variables $x_1,...,x_{100}$, where the dependent variable $y_i$ satisfies: $0\le y_i ...
2
votes
2answers
307 views

How to select a subset of variables from my original long list in order to perform logistic regression analysis?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
1
vote
0answers
28 views

why feature scaling or weighting is important in surpervised learning?

I can understand feature scaling or weighting is important in unsurpervised learning case, because we want an good representation of "similarity". But why it is also important in surpervised learning ...
2
votes
0answers
48 views

Feature selection: permutation test Vs deleting a variable

In feature selection for predictive models, it is usually applied a permutation test. In this test, all the values of one variable are randomly permuted and the prediction accuracy is extracted for ...
0
votes
0answers
16 views

Comparing F-values of covariates in R

I'm new here and have a question regarding ANOVA in R. I have an ANOVA table like this from running anova(model) in R, where ...
1
vote
0answers
19 views

Is linear kernel SVM performance between features indicative of RBF kernel SVM performance?

I have feature set 1 and feature set 2. If a linear kernel SVM performs better ("better" meaning greater classification accuracy) when using feature set 1, does this guarantee that a properly tuned ...
1
vote
1answer
22 views

Transform a non-monotonic value before decision tree (concrete example)?

Newbie question here. I am building a toy decision tree to differentiate personal names from business, government, or organizational names, like: AAA ENTERPRISES LLC DBA AAA BBB SERVICE SMITH ...
0
votes
0answers
45 views

What method should I use for this optimization / feature selection project

I'm going to describe a problem and I'm not sure how to best solve it. I will describe the situation. When answering please recommend a method and maybe a software library. I'm using Python for my ...
0
votes
1answer
58 views

Values of the weights in Adaboost

I have implemented a simple Adaboost algorithm, using several weak classifiers, and when checking the values computed by it there are alphas with a negative value. Is that possible, or is there a bug ...
0
votes
1answer
53 views

Interpreting results of a factor analysis

I performed factor analysis on R using factanal. Following advice I found on this tutorial, I chose the number of factors as being the number of principal components that capture 90% of the ...
0
votes
1answer
33 views

Choosing a score for factor analysis

I want to perform factor analysis to reduce the number of variables in my dataset (the variables are very redundant). One of the parameters I need to supply to the R code is the number of factors to ...
0
votes
2answers
115 views

R package glmnet: Ridge selects variables

I am using the glmnet package in R. When I set the alpha value = 0, I would expect that no variables are selected. When I look at the coefficients some of them are set to zero. What could be the ...
0
votes
0answers
126 views

Clustering communication patterns to detect multiple identities

I have a data set of communication patterns between chatting agents. Each agent can have multiple profiles or identities. I am interested in developing a way to investigate the similarity between ...
1
vote
1answer
42 views

Feature selection based on information gain papers

I want to apply feature selection based on information gain: I have many features many of which are redundant. I am planning on selecting a feature and then iteratively add features that 'add the more ...
0
votes
0answers
14 views

Are features present in just one sample relevant for SVM learning?

I am building classification and regression SVM (RBF) models where the features for each sample are indicator values(0, 1) for a set of features exhaustively generated from all samples. There are many ...
0
votes
0answers
54 views

How to prove the significance of features in classification?

I have a binary classification problem. I have extracted 500 features from a set of 5000 samples using my domain knowledge. In other words, I have got hand crafted features. I wish to prove that ...
1
vote
1answer
58 views

Number of components in PCA

I believe I have a problem understanding PCA: I would like to use this technique to reduce the number of features of my problem. I originally have 10,000 features and 500 samples. However, the use of ...
3
votes
3answers
289 views

Anomaly detection: what algorithm to use?

Context: I'm developing a system that analyzes clinical data to filter out implausible data that might be typos. What I did so far: To quantify the plausibility, my attempt so far was to normalize ...
1
vote
1answer
230 views

Variablity in cv.glmnet results

I am using cv.glment to find predictors. The set-up I use is as follows: ...
2
votes
1answer
47 views

Feature selection for classifier

I'm using a supervised machine learning algorithm on some big data. There is much more features than observations. To reduce the number of features, I would like to do some feature selection. However, ...
1
vote
0answers
67 views

Feature extraction for customer churn data

I have a customer churn data, and would be implementing algorithms (decision tree, logistic regression, segment analysis). I have doubt on feature extraction procedure though. The training sample has ...
0
votes
1answer
15 views

Identifying feature values that influence an outcome

I have a data set which has data about 1 million people. Data about each person consists of a 'Score' and about 100 features (each of which refers to some characteristic of the person - example - age, ...
2
votes
1answer
33 views

Selecting features manually and proving the rest are redundant

I'm working with a gesture dataset, where each gesture has a variable number of frames, and each frame has the 3d position of 20 joints, so that each gesture is represented by a matrix of size frames ...
0
votes
0answers
18 views

Replacing categorical variables with historic response rate

In Linoff and Berry's "Data Mining Techniques" they mention reducing the number of categorical variables in a classification model by replacing the variable with the historic response rate. "When ...
0
votes
0answers
33 views

Methods for temporal patterns extraction

For example a video or series of images, or usage patterns data on a website, or a univariate time series, is there some flexible methods for extracting patterns of any length, such as head ...
1
vote
1answer
86 views

AICc results in R

I used the model: ...