Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
24 views

Data-driven, high-dimensional feature selection strategies

I am working on a biomedical/healthcare data science problem. I have a dataset of 600 samples, ~6000 variables and class label as "positive" or "negative". I want to perform feature selection on ...
0
votes
1answer
32 views

Classifier with variable number of features

I am trying to make a classifier when each sample has a variable number of features. An example of how this could occur is, for example, if the features are the purchases (type, dollar amount, etc) ...
1
vote
2answers
76 views

How To Better Represent A Problem To A Machine Learning Algorithm

I am familiar with the basics of how to present a problem to a machine learning algorithm using binary encodings. I am also familiar with, but still learning about, feature selection/extraction and ...
0
votes
0answers
11 views

Find the relative importance of features weights in multi-class SVM without PCA - plotting coef distribution?

I'm classifing users with a multiclass svm (one-against-on), 3 classes. In binary, I would be able to plot the distribution of the weight of each feature in the hyperplan equation for different ...
0
votes
0answers
20 views

feature slection in random forest in python

I have a dataset consisting of 24 numeric features and about 7000 rows, i am applying random forest to get the binary classification, So please tell me how to find only the relevant features to get ...
0
votes
0answers
11 views

What's the probability that there exists a hyperplane that can split a dataset which have random feature values ?

Given n data points, each with d features, n/2 are labeled as -1, the other n/2 are labeled as 1. Each feature takes a value from [0,1] randomly (uniform distribution). What's the probability that ...
2
votes
1answer
46 views

Will adding additional features hurt the performance of SVM ?

Just wondering the effects of additional features. Following are several thoughts: If the additional features are noisy (can not distinguish the two classes), then additional features won't hurt ...
1
vote
1answer
15 views

What kind of general strategy can you apply after selecting model and hyper parameter training?

As a rookie to machine learning area, I tried to play some Data Science tutorials and beginner competitions to gain some knowledge and experience. The problem I encountered in every scenarios is ...
1
vote
0answers
13 views

Relation between chi-squared statistic scores and classification accuracy

I am evaluating the utility of two distinct sets of features for solving a given supervised classification problem with two classes. I am using the chi-squared statistic as a feature selection ...
0
votes
0answers
9 views

Finding the features that have meaning in subset of data

I have a set of $N$ points $x_i=(x_i^1, x_i^2,...,x_i^{m+k})$ in $m+k$-dimensional space ($m$ continuous dimensions and $k$ discrete). Also I have a subset of these points that are marked as "bad". ...
0
votes
1answer
33 views

After adding additional features, same accuracy on test data, but higher accuracy on training data, how should I interpret ?

I've done 5-fold cross-validation and the model is SVM. 300 features: 0.53 on test, 0.55 on training; 700 featuers: 0.53 on test, 0.67 on training. Does this mean that the additional 400 features ...
1
vote
1answer
91 views

Feature selection for time series data

I am looking for methods for feature selection (or feature extraction) for time series data. Of course I did some research before, but it was not satisfying. I am aware of methods like PCA, ...
1
vote
0answers
27 views

High dimensional explanatory variable

I have a data set of 22 observations and 6931 variables. Data belongs to two classes, 0 and 1. I would like to know which features are important for each class (species) and which one contribute the ...
1
vote
0answers
24 views

Learning if instances from a dataset are part of the same subset

I was wondering if there are some well-known machine learning methodologies for subset learning. In other words, to learn if two instances are part of the same subset or not (boolean label?). One ...
2
votes
1answer
86 views

How to increase the performance of random forest classifier?

I have a text classification task. These are the metrics for different languages at present: class1: 0.6823 class2: 0.7450 class3: 0.66 class4: 0.6719 How can I ...
0
votes
1answer
154 views

R gbm package variable influence

I'm using the excellent gbm package in R to do multinomial classification, and my question is about feature selection. After deciding the number of iterations ...
0
votes
0answers
17 views

Feature Selection for Regression Models in R [duplicate]

I’m trying to find a feature selection package in R that can be used for regression. Most of the packages implement their methods for classification using a factor ...
0
votes
1answer
78 views

Feature selection in GBM

I am using gradient boosting (caret package in R). As far as I understand, the feature selection is already included in this package. However, I slightly ...
8
votes
4answers
195 views

How can top $k$ principal components retain the predictive power on a dependent variable?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from ...
1
vote
1answer
112 views

GBM: Predict the response variable measured in {0,20}

I need to predict the response that has values in {0,20}. Should it be used as a factor or as a numeric value? How does it influence on the prediction error? I am using GBM with the Gaussian ...
1
vote
0answers
11 views

Need a statistic for comparing “strength” of Markov blankets in a Bayesian network

Working with Bayesian networks. I take a given network structure and fit its parameters on data. I am looking for a statistic based on those parameter estimates that allows me to compare Markov ...
0
votes
0answers
28 views

How many samples are enough

I have objects with large number of attributes (about 60.000). Attributes are actually deviations of object part from model. I would like to cluster this objects, to get centroids that will represent ...
0
votes
1answer
45 views

Find entropy in WEKA

I am new in data mining so sorry for asking this kind of silly question. I am working on FAST feature selection algorithm and for that I need to find entropy of each attribute in dataset. But the ...
1
vote
0answers
18 views

feature selection in a small sample size

I need an advice. I have a dataset consisting of 108 observations (27 subjects * 4 time points) and ~10000 features. The data represents intensity values (comes from continuous domain). When I run ...
6
votes
4answers
156 views

How to prepare/construct features for anomaly detection (network security data)

My goal is to analyse network logs (e.g., Apache, syslog, Active Directory security audit and so on) using clustering / anomaly detection for intrusion detection purposes. From the logs I have a lot ...
3
votes
2answers
116 views

Appropriately selecting explanatory (independent) variables

My aim is to carry out a GLM. I have 400 sites where I have count data of animals (response variable) and environmental characteristics (explanatory variables). At the moment I have around 40 ...
0
votes
0answers
15 views

What must be the sample size for feature selection by coefficient correlation method?

I have eight features from which I want to select 2-3 significant features for classification. The method which I adopted for doing so is coefficient correlation. The problem I am facing is for ...
1
vote
0answers
12 views

Will the classification accuracy vary if we first classify based on a single variable and then use the rest?

Let's suppose I am doing classification and that I have 99 features and another feature that says if the person is male or female. I have two options viz to build one classifier using all the ...
4
votes
1answer
93 views

What are the disadvantages of using Lasso for feature selection?

As far as I understand, feature selection is difficult for classification problems because it's effectively impossible to identify an optimal subset of $k$ features in problems where the the total ...
1
vote
2answers
125 views

Feature selection before neural network classification

I have a training set of 87 samples and 9480 variables. My predictors are continuous and my response variable is binary. I'd like to use the caret package in R to tune a neural network classification ...
0
votes
2answers
47 views

Which variables to keep in my analysis based on loadings from PCA? [duplicate]

Could someone please explain me how I should decide which variables to keep in my analysis based on loadings from PCA. The output is: ...
1
vote
0answers
35 views

How to reduce the number of features for Gaussian Process regression?

Ridge regression reduces complexity of the model by scaling down the coefficient. Lasso reduces the complexity of the model by selecting the features used. For Gaussian Process, is there similar way ...
1
vote
1answer
69 views

Quantitative importance for interacting variables in Artificial Neural Networks?

Is there any common/sound method to quantify (similar to T-test or F-test in regression models) the measures of influence and significance of terms in Artificial Neural Networks? By terms I mean both ...
1
vote
0answers
30 views

Akaike information criterion for categorical and numerical data

How should I compute AIC for categorical and for numeric variables in classification problems? I see in Chapter 6 of Zumel and Mount that they use AIC before they train classification algorithms ...
1
vote
1answer
28 views

Alternative to AIC for feature selection in classification

I want to know what are the most common methods for feature selection in classification problems (binary and mutli-class). I see in Chapter 6 of Zumel and Mount that they use AIC before they train ...
0
votes
0answers
20 views

How to find original features corresponding to the first two principal components? [duplicate]

I have a set of data described by $n$ features. I do a principal component analysis (PCA) to reduce it to just 2 dimensions so I can make a 2D plot of the data, with the first two Principal ...
3
votes
1answer
99 views

When should I use feature selection and when should I use dimensionality reduction techniques?

When should I use feature selection and dimensionality reduction? I know that feature selection is different from dimensionality reduction. But I don't know under what circumstances should I use ...
0
votes
1answer
77 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
0
votes
0answers
42 views

In PCA, can the values in the principle component vectors which are close to zero be removed to see the important features? [duplicate]

In PCA, when I extract the principle component vectors, I am choosing the first vector with the largest corresponding eigenvalue. I notice that some of the values in this vector are close to zero. Can ...
0
votes
0answers
21 views

To be significant or to be stabile, what's more scientfically important?

Recently I discovered the techniques related to cross validation. Basically you can split up your data in n groups and then run your model on one part of the data and assess prediction reliability on ...
2
votes
1answer
47 views

How to handle changing input vector length with neural networks

I want to train a neural network with a sequence of character as an input vector. Learning examples have different length and for this reason I don't know how to represent them. Let's say I have two ...
1
vote
1answer
43 views

Proper variable selection: Use only training data or full data?

I'm going through the lab exercises in "Introduction to Statistical Learning" and am having difficulty understanding the proper way to do best subset selection. The book is available here ...
0
votes
0answers
11 views

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor?

Is there an algorithm to determine if too few features are selected for k-nearest-neighbor when no test set is available, --when the input vectors are unknowns? Here's my problem, I have a massive ...
0
votes
0answers
18 views

Elastic net is being used in genome wide analysis. Similar approach would work for survey analysis?

I'm approaching the elastic net procedure for genome wide analysis (GWAS) because it allows for feature selection, groups detection and improved validity. It's a powerful technique when you have many ...
2
votes
0answers
24 views

Can I reuse the dataset set aside for performing t-test based on the following condition?

I have a small number of samples and large number features. For doing the feature selection I'm going to divide my total set into a feature selection set and a test set.I run the t-test on the former ...
3
votes
1answer
84 views

What is the difference between feature selection and dimensionality reduction?

I know that both feature selection and dimensionality reduction aim towards reducing the number of features in the original set of features. What is the exact difference between the two if we are ...
0
votes
0answers
22 views

SVM In text classifcation

I am learning about SVM in text classification. However, here i am posed with a problem. I have a dataset of documents which have 3 class labels. First Question Do i split the dataset into ...
2
votes
1answer
46 views

SVM Classifier: proper process

I'm working on a classification system of mine, but am needing help with the proper process order. Specifically, I'm using LibSVM and a range of feature sets extracted from my data. I'm wondering, ...
0
votes
1answer
78 views

determining how “important” a feature is in predicting a target in decision trees

Random forests allow us to compute a heuristic for determining how "important" a feature is in predicting a target. This heuristic measures the change in prediction accuracy if we take a given ...
1
vote
1answer
45 views

Do Neural Networks need “compound” features?

Apologies if I haven't got the terminology quite right. I have a question about Neural Networks, and I'm not sure exactly the best way to ask it! Hypothetically, let's say I have a dataset of houses ...