Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
21 views

Feature selection for logistic regression [on hold]

Not sure if the feature selection is the correct term but assuming I have data x,y | z where x and y features and z is target. And the task is to classify z using x,y but I know that data is not ...
2
votes
0answers
20 views

How to increase the performance of random forest classifier?

I have a text classification task. These are the metrics for different languages at present: class1: 0.6823 class2: 0.7450 class3: 0.66 class4: 0.6719 How can I ...
0
votes
1answer
43 views

R gbm package variable influence

I'm using the excellent gbm package in R to do multinomial classification, and my question is about feature selection. After deciding the number of iterations ...
0
votes
0answers
17 views

Feature Selection for Regression Models in R [duplicate]

I’m trying to find a feature selection package in R that can be used for regression. Most of the packages implement their methods for classification using a factor ...
0
votes
0answers
8 views

Feature selection in GBM

I am using gradient boosting (caret package in R). As far as I understand, the feature selection is already included in this package. However, I slightly ...
8
votes
4answers
169 views

How can top $k$ principal components retain the predictive power on a dependent variable?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from ...
1
vote
1answer
40 views

GBM: Predict the response variable measured in {0,20}

I need to predict the response that has values in {0,20}. Should it be used as a factor or as a numeric value? How does it influence on the prediction error? I am using GBM with the Gaussian ...
0
votes
0answers
7 views

Need a statistic for comparing “strength” of Markov blankets in a Bayesian network

Working with Bayesian networks. I take a given network structure and fit its parameters on data. I am looking for a statistic based on those parameter estimates that allows me to compare Markov ...
0
votes
0answers
27 views

How many samples are enough

I have objects with large number of attributes (about 60.000). Attributes are actually deviations of object part from model. I would like to cluster this objects, to get centroids that will represent ...
0
votes
1answer
21 views

Find entropy in WEKA

I am new in data mining so sorry for asking this kind of silly question. I am working on FAST feature selection algorithm and for that I need to find entropy of each attribute in dataset. But the ...
1
vote
0answers
15 views

feature selection in a small sample size

I need an advice. I have a dataset consisting of 108 observations (27 subjects * 4 time points) and ~10000 features. The data represents intensity values (comes from continuous domain). When I run ...
6
votes
4answers
127 views

How to prepare/construct features for anomaly detection (network security data)

My goal is to analyse network logs (e.g., Apache, syslog, Active Directory security audit and so on) using clustering / anomaly detection for intrusion detection purposes. From the logs I have a lot ...
3
votes
2answers
83 views

Appropriately selecting explanatory (independent) variables

My aim is to carry out a GLM. I have 400 sites where I have count data of animals (response variable) and environmental characteristics (explanatory variables). At the moment I have around 40 ...
0
votes
0answers
15 views

What must be the sample size for feature selection by coefficient correlation method?

I have eight features from which I want to select 2-3 significant features for classification. The method which I adopted for doing so is coefficient correlation. The problem I am facing is for ...
1
vote
0answers
12 views

Will the classification accuracy vary if we first classify based on a single variable and then use the rest?

Let's suppose I am doing classification and that I have 99 features and another feature that says if the person is male or female. I have two options viz to build one classifier using all the ...
4
votes
1answer
54 views

What are the disadvantages of using Lasso for feature selection?

As far as I understand, feature selection is difficult for classification problems because it's effectively impossible to identify an optimal subset of $k$ features in problems where the the total ...
1
vote
2answers
60 views

Feature selection before neural network classification

I have a training set of 87 samples and 9480 variables. My predictors are continuous and my response variable is binary. I'd like to use the caret package in R to tune a neural network classification ...
0
votes
2answers
44 views

Which variables to keep in my analysis based on loadings from PCA? [duplicate]

Could someone please explain me how I should decide which variables to keep in my analysis based on loadings from PCA. The output is: ...
1
vote
0answers
23 views

How to reduce the number of features for Gaussian Process regression?

Ridge regression reduces complexity of the model by scaling down the coefficient. Lasso reduces the complexity of the model by selecting the features used. For Gaussian Process, is there similar way ...
1
vote
1answer
53 views

Quantitative importance for interacting variables in Artificial Neural Networks?

Is there any common/sound method to quantify (similar to T-test or F-test in regression models) the measures of influence and significance of terms in Artificial Neural Networks? By terms I mean both ...
1
vote
0answers
14 views

Akaike information criterion for categorical and numerical data

How should I compute AIC for categorical and for numeric variables in classification problems? I see in Chapter 6 of Zumel and Mount that they use AIC before they train classification algorithms ...
1
vote
1answer
22 views

Alternative to AIC for feature selection in classification

I want to know what are the most common methods for feature selection in classification problems (binary and mutli-class). I see in Chapter 6 of Zumel and Mount that they use AIC before they train ...
0
votes
0answers
12 views

R language: Can the function rfe of the package caret be used with a mixed effect model [migrated]

I would like to do feature selection with a mixed effect model in R, but I cannot manage to combine the function rfe of the package caret with the function me of ...
0
votes
0answers
19 views

How to find original features corresponding to the first two principal components? [duplicate]

I have a set of data described by $n$ features. I do a principal component analysis (PCA) to reduce it to just 2 dimensions so I can make a 2D plot of the data, with the first two Principal ...
3
votes
1answer
88 views

When should I use feature selection and when should I use dimensionality reduction techniques?

When should I use feature selection and dimensionality reduction? I know that feature selection is different from dimensionality reduction. But I don't know under what circumstances should I use ...
0
votes
1answer
45 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
0
votes
0answers
42 views

In PCA, can the values in the principle component vectors which are close to zero be removed to see the important features? [duplicate]

In PCA, when I extract the principle component vectors, I am choosing the first vector with the largest corresponding eigenvalue. I notice that some of the values in this vector are close to zero. Can ...
0
votes
0answers
20 views

To be significant or to be stabile, what's more scientfically important?

Recently I discovered the techniques related to cross validation. Basically you can split up your data in n groups and then run your model on one part of the data and assess prediction reliability on ...
2
votes
1answer
25 views

How to handle changing input vector length with neural networks

I want to train a neural network with a sequence of character as an input vector. Learning examples have different length and for this reason I don't know how to represent them. Let's say I have two ...
1
vote
1answer
30 views

Proper variable selection: Use only training data or full data?

I'm going through the lab exercises in "Introduction to Statistical Learning" and am having difficulty understanding the proper way to do best subset selection. The book is available here ...
0
votes
0answers
10 views

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor?

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor when no test set is available, --when the input vectors are unknowns? Here's my problem, I have a massive ...
0
votes
0answers
49 views

In recursive feature elimination in random forest, why are all features selected?

I am trying to use the recursive feature elimination in caret package. Here's the code; ...
0
votes
0answers
9 views

Elastic net is being used in genome wide analysis. Similar approach would work for survey analysis?

I'm approaching the elastic net procedure for genome wide analysis (GWAS) because it allows for feature selection, groups detection and improved validity. It's a powerful technique when you have many ...
2
votes
0answers
21 views

Can I reuse the dataset set aside for performing t-test based on the following condition?

I have a small number of samples and large number features. For doing the feature selection I'm going to divide my total set into a feature selection set and a test set.I run the t-test on the former ...
3
votes
1answer
73 views

What is the difference between feature selection and dimensionality reduction?

I know that both feature selection and dimensionality reduction aim towards reducing the number of features in the original set of features. What is the exact difference between the two if we are ...
0
votes
0answers
22 views

SVM In text classifcation

I am learning about SVM in text classification. However, here i am posed with a problem. I have a dataset of documents which have 3 class labels. First Question Do i split the dataset into ...
2
votes
1answer
31 views

SVM Classifier: proper process

I'm working on a classification system of mine, but am needing help with the proper process order. Specifically, I'm using LibSVM and a range of feature sets extracted from my data. I'm wondering, ...
0
votes
1answer
48 views

determining how “important” a feature is in predicting a target in decision trees

Random forests allow us to compute a heuristic for determining how "important" a feature is in predicting a target. This heuristic measures the change in prediction accuracy if we take a given ...
1
vote
1answer
42 views

Do Neural Networks need “compound” features?

Apologies if I haven't got the terminology quite right. I have a question about Neural Networks, and I'm not sure exactly the best way to ask it! Hypothetically, let's say I have a dataset of houses ...
0
votes
1answer
52 views

What does the varImp function in the caret package actually compute for a glmnet (elastic net) object

I am fitting an elastic net model with glmnet via the caret package with 189 predictors and a binomial criteria (a,b) ...
0
votes
0answers
12 views

How to predict the predict upper bound of a learning algorithm?

I am selecting features for a Logistic regression classifier. I have tested a lot of feature selection algorithms. however, it seems that there exist a fixed upper bound AUC value for a fix feature ...
0
votes
0answers
19 views

What is the best practices as to cropping the positive examples for training hog classifier?

I've been reading up on the literature on HOG training, but I can't figure out what the best practices are for cropping the positive examples. Question: Do you crop tight or with margins on the ...
2
votes
1answer
60 views

K-means clustering feature selection

I have a set of English and foreign language documents that I would to perform k-means clustering on to find document groups by topic. These documents are concatenated social media comments for ...
0
votes
0answers
16 views

Bayesian network overfit - number of features and examples

For a dataset consisting of 150 examples (mostly binary features) what would be the number of features needed so that a Bayesian network doesn't overfit? I know there is no exact answer and I've ...
1
vote
0answers
26 views

Poisson model with fractions

I have a simple website with a home page that has 5 different images on it. All images have a fixed set of 'features' associated with them (size, color, position etc.,). When a visitor comes to the ...
0
votes
1answer
55 views

Selecting variables in multiple linear regression in R

Consider that we have a problem with 4 variables (y, x1, x2 and x3) and we want to do a multiple linear regression model. As we need to know which variables are the most important in the problem, we ...
1
vote
1answer
10 views

Multi Categorical Features vs multiple Features for categories

Say I am discretizing continuous data based on percentiles. (I realize this is generally frowned upon, but I am doing this for the sake of experiment) I am trying different percentiles, eg breaking ...
0
votes
1answer
57 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
0
votes
0answers
25 views

Collection of features that correlated with strength and elongation of nonwoven fibers

I'm working on a regression problem which estimates strength and elongation of a non woven fibers by using some features collected from processed images of the fibers. For this study I've collected 50 ...
0
votes
1answer
28 views

Using a set as a feature in decision tree classification

I'm faced with a data set where one of the features is a set of 4-5 categories (this number of categories isn't constant). I need to use this feature for building a decision tree. I searched online ...