Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

2
votes
1answer
103 views

Are there any techniques that quantify the importance/signification of individual attribute values of a particular data point?

Are there any techniques that quantize the importance of individual attribute values in a particular data point, in terms of the attribute's overall importance/signification/contribution to the ...
0
votes
0answers
70 views

equivalent of PCA explained variance ratio for SVD ?

i am wondering if there is an equivalent of PCA explained variance ratio for SVD. What are the measures I can get to monitor the number of columns I keep after the SVD ? Are any of these metrics ...
0
votes
0answers
27 views

Investigating 'minor' effect variables?

Is there any way to investigate minor contributing $X$ variables in a model when there are one or two $X$ variables which contribute to the explanation of a majority of the variation in the $Y$ ...
5
votes
1answer
146 views

Explain steps of LLE (local linear embedding) algorithm?

I understand the basic principle behind the algorithm for LLE consists of three steps. Finding the neighborhood of each data point by some metric such as k-nn. Find weights for each neighbor which ...
0
votes
0answers
47 views

Feature Selection: markov blanket filter

I need to do a markov blanket filter for feature selection for highly unbalanced datasets. There are popular algorithms to do this? I need to understand the algorithm behind this. From what I ...
0
votes
0answers
25 views

Choice of 0 or -1 for failure in the independent variables of a logistic regression

I am performing some exploratory analysis on a dataset where the dependent variable is a dichotomous variable. I have ~10 explanatory variables, some of which are dichotomous observations. I am ...
0
votes
2answers
70 views

Choosing the best featureset for prediction

I have this N sets of features F each with $F_i$ number of features. All the feature sets have 20000 examples and we have 20,000 labels for them. Lets say feature set $F_1$ has 10 features and ...
2
votes
1answer
99 views

How to calculate number of features based on image resolution?

Just covered Andrew Ng's Non-linear Hypothesis of Neural Netowrks, and we had a multiple choice question for determining number of features for an image of resolution 100x100 of grescale intensities. ...
2
votes
2answers
117 views

improve precision in text classification

I am working on binary text classification using sklearn: The length of each sample is not high (~ 200-500 characters) I use TF-IDF to get important words as TfidfVectorizer(sublinear_tf=False, ...
2
votes
1answer
77 views

Purposeful selection and confounding

I conducted purposeful selection as outlined in Jewell's Statistics for Epidemiology. The log likelihood tests showed covariates, which I considered to be confounding though not significant in the ...
1
vote
0answers
48 views

Inferring dimension weight in a mapping from a triangle to a distribution over its vertices

I have a dataset $(y_i, \mathbf{X}_i)$, where $\mathbf{X}_i$ is a $3 \times n$ matrix of reals and $y_i$ takes a value in $\{1, 2, 3\}$. Essentially, $y_i$ represents a "selection" of the row vector ...
0
votes
3answers
92 views

Classification performance and the feature set selection

I am now working on a classification problem. The generated feature set can be separated into two group. I did a comparison study: use all of the features; use the features of group 1 only; and use ...
2
votes
1answer
101 views

penalized regression applications in epidemiology

I am seeking advice on penalized regression models for selecting covariates in epidemiological studies. A difficult tasks I face is feature selection while still attempting to account for confounding ...
2
votes
1answer
52 views

Should I exclude predictor variables if used to create a new one?

I have a dataset that includes race, gender, income, and family size. In addition, a variable for "sliding fee scale" tier is included, which is determined by income and family size. Should income and ...
0
votes
0answers
14 views

How to find entropy of vocabulary terms in multilabel document classification problem?

I have 5 million of document s with varying number of labels for each. I intent to find entropy value for selecting discriminative terms to degrade the size of vocab. However, having that multiple ...
0
votes
0answers
22 views

ARIMAX feature selection

I am implementing the ARIMAX model and need to implement feature selection. I have >100 features and a lot of data, so I need a method that isn't too computationally expensive. I tried a wrapper, ...
0
votes
1answer
139 views

Steps followed when Binary logistic regression when both dependent and independent variables are binary

I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which ...
3
votes
3answers
1k views

Term frequency/inverse document frequency (TF/IDF): weighting

I've got a dataset which represents 1000 documents and all the words that appear in it. So the rows represent the documents and the columns represent the words. So for example, the value in cell ...
0
votes
0answers
62 views

Random Forests and Feature selection [duplicate]

First, I split my training set into 10 parts. 9 parts of it, I use as LS and the other one as TS. I now want to do feature selection, so I do feature selection on 9 parts. I use Random Forest to do ...
16
votes
4answers
551 views

When wouldn't I use LASSO for model selection?

Assume that you need to build a linear model to make predictions for new observations, and that there is uncertainty about which subset of variables should be included in the model. You are only ...
1
vote
2answers
144 views

CV on training set with feature selection

I've got a problem with CV on feature selection. I've used a method, but I don't know it's correct... I split my data into 70% training set and 30% test set I work now with my training set. I do on ...
0
votes
1answer
156 views

Feature selection and cross validation

I'm working on a project and I would like to know if the following strategy is good/correct. Sorry if this is a basic/stupid idea (I'm new to this). The input is a dataset with 2.500 features and ...
0
votes
0answers
19 views

Comparing 2 different sets of features

So let's say I have two different sets of features A and B. I'm trying to determine which set of features is the best. I'm using leave-one-out cross validation as the final metric as my data set is ...
1
vote
0answers
72 views

How to explain difference of importance between feature selection and model quality?

I have a data collection with a mixed feature set consisting of both numerical features and text features. The number of numerical features is quite small, i.e., 6, comparing to the number of text ...
0
votes
1answer
80 views

Random forest importance differs between rf$importance and importance()

My model is working ok (the AUC is 0.7) but the importances from a randomForest run for my binary classification problem differ depending on how I retrieve them. Is ...
0
votes
1answer
38 views

Predictive features with high presence in one class

I am doing a logistic regression to predict the outcome of a binary variable, say whether a journal paper gets accepted or not. The independent variable or predictors are all the phrases used in these ...
1
vote
1answer
77 views

Numeric example of data for special case of stepwise linear regression

Stepwise Regression works as follows if I'm correct: fit the initial model add the variable which has its f-stat larger than a in-threshold and repeat step 2. if there are no candidates to enter - ...
0
votes
0answers
16 views

Why the maxStages argument in biglars.fit does not work

Why doesn't the biglars.fit function work when maxStages is specified? I've tried multiple values and multiple ways of casting $y$ but it doesn't work. ...
2
votes
3answers
435 views

How to divide feature set for selection and training

I have training data with 260 observations that have a total of 7 classes. Each observation has 120 features. I applied feature selection based on the Bhattacharyya Algorithm and got the top 40 ...
1
vote
1answer
56 views

finding an optimal subgroup of binary indicators

My dependent variable is continuous variable that measures the (potential) success of a person in some activity. I have hundreds of binary indicators, each indicates about the existence of a specific ...
1
vote
0answers
147 views

Cross validation for variable selection and coefficient shrinkage?

Is cross validation an appropriate technique for variable selection and regression coefficient shrinkage? A former colleague of mine used 10-fold CV to compare the regression coefficients from the ...
0
votes
0answers
178 views

Metric warning using caret's rfe

I am using the caret package to do feature selection with rfe while training a knn ...
0
votes
0answers
42 views

Variable selection (automated)

I was wondering whether the following mechanical selection procedure will result in a possible bias. First let me introduce the first procedure, we start with a model and only look at the t-value and ...
8
votes
4answers
180 views

Lasso-ing the order of a lag?

Suppose I have longitudinal data of the form $\mathbf Y = (Y_1, \ldots, Y_J) \sim \mathcal N(\mu, \Sigma)$ (I have multiple observations, this is just the form of a single one). I'm interested in ...
0
votes
0answers
12 views

Sparse variable selection algorithms that account for parameter changes

I am variable selecting for a time-series forecasting model that has parameters sampled from a high variance sampling distribution centred near zero and that undergo changes over time. Each predictor ...
13
votes
2answers
589 views

Why does the Lasso provide Variable Selection?

I've been reading Elements of Statistical Learning, and I would like to know why the Lasso provides variable selection and ridge regression doesn't. Both methods minimize the residual sum of squares ...
3
votes
0answers
157 views

When would I choose Lasso over Elastic Net

What are the scenarios where Lasso is likely to perform better than Elastic Net (out of sample prediction)?
2
votes
2answers
90 views

What if Lasso selects transformed terms but not untransformed terms

Suppose I have standard normal features $X_i \in \{X_i : i \in \{1,...,1000\}\}$. I extend this set of predictors with transformations as follows: $\{X_i,X_i^2,X_iI(X_i > 0) : i \in ...
0
votes
1answer
142 views

Best subset selection

My statistical learning text claims that for best subset selection, 2^p total models must be fit through regression if for p covariates, we fit p choose k models at each k, k = 1,...,p. I interpret ...
0
votes
0answers
29 views

Recursive Feature Elimination Fails to Output as Expected

Currently I am using rfe function in the "caret" package to do feature selection. There are 380 variables as input candidates. I have done many trials and I noticed that something weird always ...
0
votes
0answers
126 views

Find weight of features for feature selection

I have a data set of videos from which I need to recognize the emotion of the speaker. For that reason I have some markers on the face of the speaker. I detect their movement as the speaker speaks and ...
2
votes
2answers
185 views

Variable selection with groups of predictors that are highly correlated

What variable selection approach should I consider if I have thousands of predictors with clusters that are extremely correlated? For example I might have a predictor set $X:= ...
1
vote
1answer
83 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
1
vote
1answer
226 views

Decision trees variable (feature) scaling and variable (feature) normalization (tuning) required in which implementations?

In many machine learning algorithms, feature scaling (aka variable scaling, normalization) is a common prepocessing step Wikipedia - Feature Scaling -- this question was close Question#41704 - How and ...
0
votes
0answers
31 views

Confusion related to feature engineering

I was reading this tutorial where they mentioned ...
0
votes
4answers
130 views

Machine learning input relationships

After learning about a few machine-learning models (NN, SVM, decision trees), I was wondering if these models are able to find inherent relationships when learning. For example, if I feed it two ...
0
votes
1answer
123 views

How Can I use some variables selected by LASSO?

I am very new about statistics. So, please understand if my question is somewhat awkward, and please give me related any advice. I have some data set. X = 500 x 100 (500 observations x 100 ...
2
votes
2answers
203 views

Variable selection in time-series forecasting

I have a time-series forecasting task and would like some input on variable selection and regularisation. My problem has the following characteristics: 2,000,000 sample size. Most of the time, no ...
5
votes
3answers
802 views

Selecting the best subset of variables for parsimonious binary logistic regression models

In addition to PROC VARCLUS, randomForest, glmnet, and assessing multicollinearity among potential predictor variables (without regards to the outcome of interest), I am seeking other methods of ...
0
votes
0answers
45 views

Network/structure learning

Given a data set $\mathbf{X}\in\mathbb{R}^{n\times p}$, where $n$ is the number of samples (observations) and $p$ is the number of features, I would like to know what kind of methods exist for ...