Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
2answers
80 views

is linear regression/polynomial regression sensitive to irrelevant features/noise

is linear regression/polynomial regression sensitive to irrelevant features/noise will their respective weights/coefficients be automatically be tuned down? or is it a straight nail in the coffin? ...
0
votes
0answers
40 views

Analysis of wrapper feature selection ouptput in Weka

I am using Weka to select important features from a dataset. I am using the wrapper method in this application. I chose a decision tree (j.48) for my classifier and Genetic search for the search ...
3
votes
2answers
167 views

Cross validated penalized logistic regression - one standard deviation rule

I am new to this topic and would like to understand it better. I want to build a binary classifier based on penalized logistic regression. I have 10 features and 23 observations: 16 from class "0" and ...
1
vote
0answers
47 views
0
votes
1answer
15 views

Using SVM's to output binary 0 or 1 to data

I am using a very large ~700,000 sample training set and ~700,000 sample testing set and training an SVM with the training set. When I run the SVM (SciKit-Learn) on the testing set it outputs only 0's....
0
votes
2answers
131 views

Feature selection clustering customer segmentation

based on customer data I want to perform a clustering using different clustering algorithms (K-Means, Expectation Maximization, etc.) in R. The most attributes were engineered pursuing the goal to be ...
2
votes
2answers
105 views

Subset selection features acquired from randomized logistic regression

I learned about the concept of randomized logistic regression(or randomized lasso) recently. My data, biological data called Microarray, usually has large features but small samples - 10000 features ...
3
votes
1answer
47 views

restrict splitting variable number in random forest?

Background: I have a set of ~100 features (input) that predict 25 variables (output). My input variables are integers in {1,2,3,4,5,6,7}, my output is continuous. I have ~100K data rows available. I ...
0
votes
1answer
44 views

Appropriate model for feature subset selection

I am working with a feature selection problem. What I am trying to do is find optimal subset of features for classification. My data consist of 100 features and 300 instances, and class label is ...
0
votes
0answers
23 views

Doing backward feature elimination using a classifier different from the one used to classify test set: does it make sense?

I'll make an example: if I use Naive Bayes for backward elimination and then I use optimal attributes discovered from that procedure to classify test set using SVM, does it make sense?
4
votes
3answers
158 views

Can I use linear model on each variable to determine which variables are important?

Suppose we have a n*p matrix X and a n*1 matrix Y, where n is the number of samples and p is the number of variables. p>>n. Also suppose this data is from a biology field experiment. My goal is to ...
7
votes
3answers
187 views

Is it better to do exploratory data analysis on the training dataset only?

I'm doing exploratory data analysis (EDA) on a dataset. Then I will select some features to predict a dependent variable. The question is: Should I do the EDA on my training dataset only? Or ...
0
votes
0answers
44 views

Feature Selection Order

I am implementing univariate feature selection from feature selection!. I have several features among which I am intending to select some features and proceed. Should I scale my data before applying ...
1
vote
1answer
22 views

Can a value appearing more frequently for a class help predict it?

I am trying to analyse my data before doing multi-class classification with SVM. I have several variables. I pick one of them and study it. This is a categorical variable. It can have the value 0 or ...
5
votes
1answer
262 views

how does multicollinearity affect feature importances in random forest classifier?

I have a random forest binary classifier, but the results from the feature importances are somewhat erratic. Here's what I want to know: Does multicollinearity ...
1
vote
0answers
150 views

Remove features with high correlation

In a classification problem using Linear SVM, I am trying to remove variables which have a strong correlation (Pearson) between them from a dataset. What is the usual threshold recommended? I ...
1
vote
1answer
59 views

Positive and negative impact of predictors on responses in data mining models

My question is an extension to the question asked here. How does one identify the parity of predictor/feature/variable impact on response/outcome in a data mining model. Is there a standard procedure ...
4
votes
1answer
292 views

Use of Random Forests for variable importance as preprocess before another analysis

the question Demonstrate the speed and accuracy of properly applied 'Random Forest' as a variable importance selection tool especially in handling very large data against alternative approaches such ...
0
votes
1answer
33 views

Measure influence of attribues on clustering

I don't have a specific example for my problem and maybe this is trivial, but I want to know how to measure the influence of specific attributes (or dimensions) of a dataset for clustering, like there ...
0
votes
1answer
115 views

pickSizeBest() for recursive feature elimination

I'm struggling providing my recursive feature elimination (RFE) function with valid arguments. This question is technically pretty specific so I hope I've hit the right Forum to ask it. I want to ...
0
votes
1answer
57 views

Linear Regression Model in R - Which variables should I use?

I would like to fit a linear regression model in R for predicting motorbike prices. My dataset has 13 variables, including number of kilometers driven, colour, month of the first registration, etc. ...
1
vote
0answers
58 views

SVM predicts everything in one class

I'm running a basic language classification task. There are two classes (0/1), and they are roughly evenly balanced (689/776). Thus far, I've only created basic unigram language models and used these ...
0
votes
0answers
12 views

Can we derive features from the output variable?

I know this sound weird, but can we use the output variable of the training data set to derive the some new features for feeding the model. If yes then how can it be statistically significant?
0
votes
1answer
57 views

Variable selection for multiple regression from large number of predictors

I have 20 response variables $Y = (Y_1, \dots, Y_{20})$, and 1600 predictor variables $X = (X_1, \dots, Y_{1600})$. There are 128 observations. I wanted to know which pairs of $X$ can best predict ...
0
votes
1answer
221 views

MATLAB function TreeBagger() (Random Forest classification) and different number of variables

I am using MATLAB function TreeBagger() for Random Forest classification, for an assignment. It gives error when the number of variables of the Test data is different from the number of variables of ...
0
votes
0answers
17 views

feature selection for clustering: wrappers

I am trying to understand if it is correct to perform a feature selection process using a wrapper method (for example using algorithms such as random forest, linear regression etc.) and then to extend ...
0
votes
0answers
14 views

How do i classify the feature set obtained using tf*idf

i have obtained the feature set for text classificaion using tf*idf method.I want to classify this feature set using SVM. How do i go ahead?
4
votes
1answer
97 views

How to fit weights into Q-values with linear function approximation

In reinforcement learning, linear function approximation is often used when large state spaces are present. (When look up tables become unfeasible.) The form of the $Q-$value with linear function ...
0
votes
0answers
18 views

To what degree does Mahalanobis distance account for correlations of the data?

I understand that Mahalanobis distance is used when your data are correlated (e.g., as with many environmental variables), but just how correlated can the variables be? Should I be screening and ...
3
votes
1answer
142 views

Best approaches for feature engineering?

I have a regression problem. The aim is to estimate the best fitting curve from a set of features. Now I have extracted a set of features that are relevant based on the literatures found. Now the ...
2
votes
2answers
98 views

Choosing predictors in regression analysis and multicollinearity

I would like to run a linear regression analysis and I'm uncertain about including predictors. I have three predictor variables available. One is based on a lot of previous research. Therefore I am ...
0
votes
0answers
32 views

Determine which independent variables are related with a dependent variable

I have a list of independent variables, some of which might be related with dependent variable (linearly or non linearly). If I draw scatter plot then I might get some pattern but still I am not sure ...
0
votes
1answer
35 views

Variable selection for regression and classification

This might be noob question, because I've just started to learn data analytics. Why should we or should we not include a correlated predictor in our model? While selecting predictors, for a class ...
0
votes
0answers
14 views

what is the method in dictionary learning which does not have a overcomplete dictionary?

what is the method in dictionary learning which does not have a overcomplete dictionary? and what is the difference in minimization between these two methods (one using overcompelte dictionary and ...
1
vote
1answer
59 views

Subset selection / feature selection with a categorical response variable

There are a number of helpful posts on subset selection (or feature selection) with continuous or binary response variables (i.e., here). However, I've not been able to find any posts on subset ...
0
votes
1answer
50 views

What problems may be caused by redundant features?

Regarding feature selection in machine learning (e.g., classification or regression) tasks, what would be the biggest problems if two features are somewhat redundant? For example, we try to predict ...
0
votes
0answers
110 views

Feature selection in high dimension

Suppose I have a (labeled) date, where features have many categories. For example, one can take kaggle's Wallmart dataset https://www.kaggle.com/c/walmart-recruiting-trip-type-classification/data. ...
1
vote
0answers
18 views

the dangers of using too few/many features in reinforcement learning

In continuous space reinforcement learning, is there a rule of thumb that tells you how many features should you observe? What are the dangers of having too few or too many of the features?
0
votes
0answers
40 views

How does one reject regression models using Spjøtvoll's method?

How does one reject regression models using Spjøtvoll's method? I've been looking online for a solution but haven't found one. What is meant by saying one subset is better than another? There ...
0
votes
2answers
159 views

How can I select a model using subset selection and cross-validation?

I try to find a model using logistic regression. More precisely, what I did so far, is using stepwise regression and subset selection (although I know, it is often a bad idea) to find the "best" model....
0
votes
0answers
15 views

Classification problem, using time series features with different weights

In a classification problem, one of the features is the month, which is extracted from the timestamp of each observation. I have about four years of observations, but I wanted to use the most recent ...
0
votes
0answers
52 views

Accuracy decreased after feature selection

For my machine learning study, I tested different algorithms like SVM, SMO, Naive Bayes, Trees etc. All the algorithms resulted with low accuracy levels. In fact the highest accuracy I obtained was 46%...
3
votes
0answers
170 views

Why is lasso in matlab much slower than glmnet in R (10 min versus ~1 s)?

I observed that the function lasso in MATLAB is relatively slow. I run many regression problems, with typically 1 to 100 predictors and 200 to 500 observations. In some cases, lasso turned out to be ...
3
votes
1answer
58 views

How can I prove that the f-statistic does not follow an F distribution in the context of step-wise regression?

There is a good number of threads about the deficiencies of step-wise regression, and particularly on the shortcomings of the partial F test as a tool for step selection. However I find it difficult ...
3
votes
1answer
55 views

Variable selection question

In regression where there are multiple predictors $x_1, x_2, \dots$ let's say our feature selection algorithm (lasso, forward stepwise, etc.) returns that $x_1$ is an important predictor, but our ...
0
votes
0answers
13 views

When does fitting via intermediate submodels matter?

Suppose you have two features A and B and plenty of data. What happens when, instead of fitting a model to A and B simultaneously, you first fit a model to A and then fit a second model to the output ...
5
votes
0answers
62 views

Bayesian lasso vs spike and slab

Suppose I have the likelihood: $$y\sim\mathcal{N}(Xw,\sigma^2I)$$ where I can put either one of the priors: $$ w_i\sim \pi\delta_0+(1-\pi)\mathcal{N}(0,100)\\ \pi=0.9 $$ or $$ w_i\sim \exp(-\lambda|...
2
votes
2answers
398 views

Random Forests for predictor importance (Matlab)

I'm working with a dataset of approx 150,000 observations and 50 features, using SVM for the final model. To trim down the feature count, I decided to look into using RF so SVM optimization doesn't ...
1
vote
2answers
68 views

Feature selection with the help of Genetic algorithm in datasets with small number of features and samples

As a project, i should perform feature selection on small unbalanced datasets with at most 30 features and also at most 300 samples with the help of Genetic algorithm. So, the chromosomes in GA are ...
0
votes
1answer
68 views

Generalising correlation-based feature selection

One method to perform feature selection consists in calculating Pearson's correlation coefficient between each explanatory variable $X_i$ and the response variable $Y$. Then the absolute values of the ...