Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
1answer
13 views

Feature selection in GBM

I am using gradient boosting (caret package in R). As far as I understand, the feature selection is already included in this package. However, I slightly ...
0
votes
0answers
9 views

learning if instances from a dataset are part of the same subset

I was wondering if there are some well-known machine learning methodologies for subset learning. In other words, to learn if two instances are part of the same subset or not (boolean label?). One ...
0
votes
1answer
47 views

R gbm package variable influence

I'm using the excellent gbm package in R to do multinomial classification, and my question is about feature selection. After deciding the number of iterations ...
2
votes
0answers
25 views

How to increase the performance of random forest classifier?

I have a text classification task. These are the metrics for different languages at present: class1: 0.6823 class2: 0.7450 class3: 0.66 class4: 0.6719 How can I ...
3
votes
0answers
51 views

Placement of earlier features in more complex features in CNN

I'm trying to understand convolutional neural networks better. I've been doing different tutorials, but there are some basics concerning how the hidden units represents features that I really would ...
0
votes
0answers
21 views

Feature selection for logistic regression [on hold]

Not sure if the feature selection is the correct term but assuming I have data x,y | z where x and y features and z is target. And the task is to classify z using x,y but I know that data is not ...
0
votes
1answer
311 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
8
votes
4answers
169 views

How can top $k$ principal components retain the predictive power on a dependent variable?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from ...
0
votes
0answers
17 views

Feature Selection for Regression Models in R [duplicate]

I’m trying to find a feature selection package in R that can be used for regression. Most of the packages implement their methods for classification using a factor ...
1
vote
2answers
125 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
9
votes
3answers
7k views

Using principal component analysis (PCA) for feature selection

I'm new to feature selection and I was wondering how you would use PCA to perform feature selection. Does PCA compute a relative score for each input variable that you can use to filter out ...
1
vote
2answers
177 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
6
votes
1answer
1k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
1
vote
1answer
44 views

GBM: Predict the response variable measured in {0,20}

I need to predict the response that has values in {0,20}. Should it be used as a factor or as a numeric value? How does it influence on the prediction error? I am using GBM with the Gaussian ...
6
votes
4answers
127 views

How to prepare/construct features for anomaly detection (network security data)

My goal is to analyse network logs (e.g., Apache, syslog, Active Directory security audit and so on) using clustering / anomaly detection for intrusion detection purposes. From the logs I have a lot ...
2
votes
1answer
316 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
0
votes
0answers
7 views

Need a statistic for comparing “strength” of Markov blankets in a Bayesian network

Working with Bayesian networks. I take a given network structure and fit its parameters on data. I am looking for a statistic based on those parameter estimates that allows me to compare Markov ...
1
vote
1answer
94 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
0
votes
0answers
27 views

How many samples are enough

I have objects with large number of attributes (about 60.000). Attributes are actually deviations of object part from model. I would like to cluster this objects, to get centroids that will represent ...
0
votes
1answer
22 views

Find entropy in WEKA

I am new in data mining so sorry for asking this kind of silly question. I am working on FAST feature selection algorithm and for that I need to find entropy of each attribute in dataset. But the ...
2
votes
1answer
171 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
1
vote
0answers
15 views

feature selection in a small sample size

I need an advice. I have a dataset consisting of 108 observations (27 subjects * 4 time points) and ~10000 features. The data represents intensity values (comes from continuous domain). When I run ...
4
votes
1answer
54 views

What are the disadvantages of using Lasso for feature selection?

As far as I understand, feature selection is difficult for classification problems because it's effectively impossible to identify an optimal subset of $k$ features in problems where the the total ...
3
votes
2answers
83 views

Appropriately selecting explanatory (independent) variables

My aim is to carry out a GLM. I have 400 sites where I have count data of animals (response variable) and environmental characteristics (explanatory variables). At the moment I have around 40 ...
0
votes
0answers
15 views

What must be the sample size for feature selection by coefficient correlation method?

I have eight features from which I want to select 2-3 significant features for classification. The method which I adopted for doing so is coefficient correlation. The problem I am facing is for ...
1
vote
0answers
12 views

Will the classification accuracy vary if we first classify based on a single variable and then use the rest?

Let's suppose I am doing classification and that I have 99 features and another feature that says if the person is male or female. I have two options viz to build one classifier using all the ...
3
votes
1answer
89 views

When should I use feature selection and when should I use dimensionality reduction techniques?

When should I use feature selection and dimensionality reduction? I know that feature selection is different from dimensionality reduction. But I don't know under what circumstances should I use ...
1
vote
1answer
227 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
0
votes
1answer
113 views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
2answers
60 views

Feature selection before neural network classification

I have a training set of 87 samples and 9480 variables. My predictors are continuous and my response variable is binary. I'd like to use the caret package in R to tune a neural network classification ...
1
vote
1answer
106 views

How to handle missing data in a small $n$ large $k$ machine learning scenario?

I have a sample size $N=130$ and $1000$ variables. I am using machine learning techniques (SVM) for analysing the data. Some variables in the dataset have values that are so huge that they must be ...
0
votes
2answers
44 views

Which variables to keep in my analysis based on loadings from PCA? [duplicate]

Could someone please explain me how I should decide which variables to keep in my analysis based on loadings from PCA. The output is: ...
1
vote
0answers
23 views

How to reduce the number of features for Gaussian Process regression?

Ridge regression reduces complexity of the model by scaling down the coefficient. Lasso reduces the complexity of the model by selecting the features used. For Gaussian Process, is there similar way ...
1
vote
1answer
53 views

Quantitative importance for interacting variables in Artificial Neural Networks?

Is there any common/sound method to quantify (similar to T-test or F-test in regression models) the measures of influence and significance of terms in Artificial Neural Networks? By terms I mean both ...
1
vote
1answer
22 views

Alternative to AIC for feature selection in classification

I want to know what are the most common methods for feature selection in classification problems (binary and mutli-class). I see in Chapter 6 of Zumel and Mount that they use AIC before they train ...
1
vote
0answers
14 views

Akaike information criterion for categorical and numerical data

How should I compute AIC for categorical and for numeric variables in classification problems? I see in Chapter 6 of Zumel and Mount that they use AIC before they train classification algorithms ...
3
votes
3answers
1k views

How to use principal components as predictors in GLM?

How would I use the output of a principal components analysis (PCA) in a generalized linear model (GLM), assuming the PCA is used for variable selection for the GLM? Clarification: I want to use PCA ...
0
votes
1answer
46 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
0
votes
0answers
12 views

R language: Can the function rfe of the package caret be used with a mixed effect model [migrated]

I would like to do feature selection with a mixed effect model in R, but I cannot manage to combine the function rfe of the package caret with the function me of ...
0
votes
0answers
19 views

How to find original features corresponding to the first two principal components? [duplicate]

I have a set of data described by $n$ features. I do a principal component analysis (PCA) to reduce it to just 2 dimensions so I can make a 2D plot of the data, with the first two Principal ...
0
votes
0answers
42 views

In PCA, can the values in the principle component vectors which are close to zero be removed to see the important features? [duplicate]

In PCA, when I extract the principle component vectors, I am choosing the first vector with the largest corresponding eigenvalue. I notice that some of the values in this vector are close to zero. Can ...
0
votes
1answer
258 views

How to normalize time series?

This is a general question on normalization of data so that all the variables are within the same range. Why do we normalize data in pattern classification? How to normalize time series which is ...
1
vote
1answer
489 views

Issues with sequential feature selection

I am trying to do some feature selection in gene expression data with 22215 features. I followed the tutorial here. I initially applied filter method(ttest) to select the features having the best p ...
0
votes
0answers
20 views

To be significant or to be stabile, what's more scientfically important?

Recently I discovered the techniques related to cross validation. Basically you can split up your data in n groups and then run your model on one part of the data and assess prediction reliability on ...
14
votes
3answers
896 views

Model stability when dealing with large $p$, small $n$ problem

Intro: I have a dataset with a classical "large p, small n problem". The number available samples n=150 while the number of possible predictors p=400. The outcome is a continuous variable. I want ...
2
votes
1answer
25 views

How to handle changing input vector length with neural networks

I want to train a neural network with a sequence of character as an input vector. Learning examples have different length and for this reason I don't know how to represent them. Let's say I have two ...
5
votes
2answers
1k views

Is it possible to use kernel PCA for feature selection?

Is it possible to use kernel principal component analysis (kPCA) for Latent Semantic Indexing (LSI) in the same way as PCA is used? I perform LSI in R using the prcomp PCA function and extract the ...
1
vote
1answer
104 views

Feature selection and training on the same sample

Is feature selection and training on the same sample a bad idea? I want to emphasize that I am not going to use test set for feature selection. If I use the whole train set for feature selection and ...
1
vote
2answers
1k views

Variablity in cv.glmnet results

I am using cv.glment to find predictors. The set-up I use is as follows: ...
1
vote
1answer
30 views

Proper variable selection: Use only training data or full data?

I'm going through the lab exercises in "Introduction to Statistical Learning" and am having difficulty understanding the proper way to do best subset selection. The book is available here ...