Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

3
votes
1answer
132 views

Best approaches for feature engineering?

I have a regression problem. The aim is to estimate the best fitting curve from a set of features. Now I have extracted a set of features that are relevant based on the literatures found. Now the ...
2
votes
2answers
97 views

Choosing predictors in regression analysis and multicollinearity

I would like to run a linear regression analysis and I'm uncertain about including predictors. I have three predictor variables available. One is based on a lot of previous research. Therefore I am ...
0
votes
0answers
32 views

Determine which independent variables are related with a dependent variable

I have a list of independent variables, some of which might be related with dependent variable (linearly or non linearly). If I draw scatter plot then I might get some pattern but still I am not sure ...
0
votes
1answer
32 views

Variable selection for regression and classification

This might be noob question, because I've just started to learn data analytics. Why should we or should we not include a correlated predictor in our model? While selecting predictors, for a class ...
0
votes
0answers
14 views

what is the method in dictionary learning which does not have a overcomplete dictionary?

what is the method in dictionary learning which does not have a overcomplete dictionary? and what is the difference in minimization between these two methods (one using overcompelte dictionary and ...
1
vote
1answer
55 views

Subset selection / feature selection with a categorical response variable

There are a number of helpful posts on subset selection (or feature selection) with continuous or binary response variables (i.e., here). However, I've not been able to find any posts on subset ...
0
votes
1answer
48 views

What problems may be caused by redundant features?

Regarding feature selection in machine learning (e.g., classification or regression) tasks, what would be the biggest problems if two features are somewhat redundant? For example, we try to predict ...
0
votes
0answers
100 views

Feature selection in high dimension

Suppose I have a (labeled) date, where features have many categories. For example, one can take kaggle's Wallmart dataset https://www.kaggle.com/c/walmart-recruiting-trip-type-classification/data. ...
1
vote
0answers
18 views

the dangers of using too few/many features in reinforcement learning

In continuous space reinforcement learning, is there a rule of thumb that tells you how many features should you observe? What are the dangers of having too few or too many of the features?
0
votes
0answers
40 views

How does one reject regression models using Spjøtvoll's method?

How does one reject regression models using Spjøtvoll's method? I've been looking online for a solution but haven't found one. What is meant by saying one subset is better than another? There ...
0
votes
2answers
141 views

How can I select a model using subset selection and cross-validation?

I try to find a model using logistic regression. More precisely, what I did so far, is using stepwise regression and subset selection (although I know, it is often a bad idea) to find the "best" ...
0
votes
0answers
15 views

Classification problem, using time series features with different weights

In a classification problem, one of the features is the month, which is extracted from the timestamp of each observation. I have about four years of observations, but I wanted to use the most recent ...
0
votes
0answers
44 views

Accuracy decreased after feature selection

For my machine learning study, I tested different algorithms like SVM, SMO, Naive Bayes, Trees etc. All the algorithms resulted with low accuracy levels. In fact the highest accuracy I obtained was ...
3
votes
0answers
153 views

Why is lasso in matlab much slower than glmnet in R (10 min versus ~1 s)?

I observed that the function lasso in MATLAB is relatively slow. I run many regression problems, with typically 1 to 100 predictors and 200 to 500 observations. In some cases, lasso turned out to be ...
3
votes
1answer
58 views

How can I prove that the f-statistic does not follow an F distribution in the context of step-wise regression?

There is a good number of threads about the deficiencies of step-wise regression, and particularly on the shortcomings of the partial F test as a tool for step selection. However I find it difficult ...
3
votes
1answer
54 views

Variable selection question

In regression where there are multiple predictors $x_1, x_2, \dots$ let's say our feature selection algorithm (lasso, forward stepwise, etc.) returns that $x_1$ is an important predictor, but our ...
0
votes
0answers
13 views

When does fitting via intermediate submodels matter?

Suppose you have two features A and B and plenty of data. What happens when, instead of fitting a model to A and B simultaneously, you first fit a model to A and then fit a second model to the output ...
5
votes
0answers
51 views

Bayesian lasso vs spike and slab

Suppose I have the likelihood: $$y\sim\mathcal{N}(Xw,\sigma^2I)$$ where I can put either one of the priors: $$ w_i\sim \pi\delta_0+(1-\pi)\mathcal{N}(0,100)\\ \pi=0.9 $$ or $$ w_i\sim ...
2
votes
2answers
342 views

Random Forests for predictor importance (Matlab)

I'm working with a dataset of approx 150,000 observations and 50 features, using SVM for the final model. To trim down the feature count, I decided to look into using RF so SVM optimization doesn't ...
0
votes
1answer
56 views

Feature selection with the help of Genetic algorithm in datasets with small number of features and samples

As a project, i should perform feature selection on small unbalanced datasets with at most 30 features and also at most 300 samples with the help of Genetic algorithm. So, the chromosomes in GA are ...
0
votes
1answer
67 views

Generalising correlation-based feature selection

One method to perform feature selection consists in calculating Pearson's correlation coefficient between each explanatory variable $X_i$ and the response variable $Y$. Then the absolute values of the ...
2
votes
1answer
174 views

Impute missing data before or after feature selection?

Will the results of the feature selection be biased if I perform the feature selection before imputing missing data? I have a large data set of 20000 samples and 130 variables. The data sets ...
0
votes
1answer
45 views

Selecting the best time lagged moving average for time series analysis

I am studying the effect of weather on agricultural outputs. I have yield data from one farm over 5 years and a number of weather inputs (rainfall, temperature, soil moisture, etc.) for the entire ...
1
vote
0answers
17 views

Finding correlations between large numbers of features

I have data which is composed of several thousand continuous features. What sorts of ML or statistical algorithms can efficiently find the rules for which features are most correlated with other ...
3
votes
1answer
352 views

Implicit feature space of Power Kernel

For the polynomial kernel, $K(x,y) = (x^Ty+c)^d$, the implicit feature space $\phi$ for which $K(x,y) = \phi(x)^T \phi(y)$ is of finite dimension and well known [1][2]. It is also well known that the ...
2
votes
1answer
96 views

CART and Clustered Data?

Just wonder if there is any caveat if one fits regular regression trees to clustered data but ignores the clustered structure of the data. More generally, how bad it would be if we fit regression ...
0
votes
0answers
35 views

Sample size for variable selection methods?

I have data in which the response variable is occupational injury rates ( positive continuous variable), and the predictors the job characteristics (standardized on a rating from 0 to 5) which might ...
1
vote
1answer
76 views

Principal Component Analysis Vs Feature Selection

I am doing a machine learning project using WEKA. It is a supervised classification and in my basic experiments, I achieved very poor level of accuracy. Then my intention was to do a feature ...
1
vote
1answer
42 views

How large of a percentage of my training set do I have to use to perform feature selection?

I have a data set that has 660,000 samples with 72 features and I'm trying to perform feature selection so that I can train a naive bayes classifier. The problem is that since the data set is so big, ...
0
votes
0answers
44 views

p-values for feature selection

I am doing multiple regression analysis,in which i want to eliminate some of the insignificant features. In most of the machine learning books subset selection,shrinkage methods or PCA is used for ...
1
vote
2answers
38 views

How can including an IV uncorrelated with the DV improve a multiple regression model?

Let's say $Y$ is my DV, and $X_1$,and $X_2$ are IVs: \begin{align} \newcommand{\Cor}{\rm Cor} \Cor(Y,X_1) &= 0.7994172 \\ \Cor(Y,X_2) &= -0.00041 \\ \Cor(X_1,X_2) &= 0.505 ...
0
votes
1answer
14 views

Similarities in ANOVA and subset selection method

I am addressing multiple regression problem which has 7 predictor variables, currently I am planning to use ANOVA fro reducing number of predictor variables. Which is the better method to use ANOVA or ...
1
vote
1answer
101 views

Classification: Calculating the significance of feature rankings?

I know that classification performance can be tested for significance by permutation tests which permute the class labels. I wonder now if for ranked features (ie a ranking according to the features' ...
0
votes
1answer
38 views

How to build classification model towards some rare response classes?

I was asked to build a predictive classification model that can predict some types of response. I am interested in 6 classes, however, the total occurence of these 6 classes (out of almost half a ...
3
votes
1answer
49 views

Match model selection strategies with modelling objectives

I am confused trying to match different model selection strategies with different modelling objectives. (Unfortunately, my confusion is reflected in the length of the post. Please be patient.) Model ...
2
votes
1answer
119 views

Recommend a method for variable selection (other than classification tree or random forest)?

Just wonder if you could recommend a few methods (other than tree-based methods) to analyze a dataset in which n= 350 and p = 35. The goal is not so much about prediction, but to find/select ...
0
votes
1answer
43 views

Methods for unsupervised subset selection on categorical data

I am new to this. I have a set of survey data with 18 questions (columns/features) with 165 observations. Responses are ternary: True, False, Don't Know. Each question has a correct response, which ...
0
votes
0answers
55 views

Interpreting Gradient Boosting Machine: Feature importance undervalues binary features, overvalues continuous features

I'm migrating this question over from stackoverflow because it has more to do with stats/ml than scikit learn itself. I'm using sci-kit learn's gradient boosting machine for a binary classification ...
2
votes
1answer
181 views

Building Cox Proportional Hazards Models and Getting Accurate Confidence Intervals

I have 4000 data records of survival data and 100 potential predictors/variables. My aim is to obtain the most important variables describing this data. Since complete data records are needed I ...
0
votes
2answers
49 views

Testing predictive power of a set of features

This is perhaps a typical setup in Bioinformatics: we need to build a model to predict a dependent biological variable (say $y$), given a large set of (usually genomic) features $X$ (which might not ...
0
votes
0answers
16 views

Properly calculate correlation for time series (feature selection)

I am using correlation to approximately evaluate quality of input signals $x_1$, $x_2$, ...$x_k$ for output signal $y$. All those signals have very low variance, i.e. the moves $x_i(t+1) - x_i(t)$ are ...
5
votes
1answer
161 views

Pattern recognition: Not possible with linear classifiers for local and distributed information?

I came across a paper with the following example to discuss the impossibility of localizing important features for certain patterns if linear classifiers are being used: As I understand it, we have ...
1
vote
0answers
45 views

Randomforest on Bag of words TFIDF/ LSI for Video Recommendation

I'm working on the Ted Dataset which has the Name, Description and Transcript of 1000 TED Talks. My task is to recommend 3 TED Talks to a given video. What I'm doing now is converting the Text to Bag ...
0
votes
0answers
35 views

Unsupervised feature selection for time series data

I have gathered a lot of time-series samples (>10000) in 5 sets (which don't need to be distinguishable) with multiple features (of which there are >500) over the time of 2 weeks. I can only record ...
0
votes
1answer
60 views

GLMNET small deviance explained; reuse of selected predictors in other model

I am trying to run glmnet for logistic regression (I have some continuos predictors which I have scaled with scale() and some categorical which I turned to dummy predictors, 27 predictors, 800 ...
0
votes
0answers
44 views

Variable Selection: Estimation versus Prediction

I have a pretty good grasp of the pros and cons of different methods of variable selection: LASSO and LARS, AIC, stepwise procedures, etc. But my question is: when modeling, should you conduct ...
0
votes
1answer
41 views

How to proceed when a linear model has a high p-value?

I am working with a data set that is 186x79. What I am interested in with this dataset is finding the features (predictors) that are the most important for ...
1
vote
1answer
73 views

correct feature selection for stable feature subset

I was just reading "Resampling Strategies for Model Assessment and Selection" by Richard Simon (Springer), and he says the following on page 183: "Data analysts are sometimes tempted to use the ...
0
votes
0answers
59 views

Using Shannon entropy for feature selection

Is it correct to use Shannon entropy for feature selection? And if so how can I set a meaningful threshold?
3
votes
2answers
96 views

Goodness of fit in logistic regression where features are not frequencies

I am fitting a logistic regression model with a set of features for predicting outcomes in football games (three outcomes available: home wins, away wins or draw). The features are such as difference ...