Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
1answer
45 views

GLMNET small deviance explained; reuse of selected predictors in other model

I am trying to run glmnet for logistic regression (I have some continuos predictors which I have scaled with scale() and some categorical which I turned to dummy predictors, 27 predictors, 800 ...
0
votes
0answers
33 views

Variable Selection: Estimation versus Prediction

I have a pretty good grasp of the pros and cons of different methods of variable selection: LASSO and LARS, AIC, stepwise procedures, etc. But my question is: when modeling, should you conduct ...
0
votes
1answer
37 views

How to proceed when a linear model has a high p-value?

I am working with a data set that is 186x79. What I am interested in with this dataset is finding the features (predictors) that are the most important for ...
0
votes
1answer
53 views

correct feature selection for stable feature subset

I was just reading "Resampling Strategies for Model Assessment and Selection" by Richard Simon (Springer), and he says the following on page 183: "Data analysts are sometimes tempted to use the ...
0
votes
0answers
54 views

Using Shannon entropy for feature selection

Is it correct to use Shannon entropy for feature selection? And if so how can I set a meaningful threshold?
3
votes
2answers
86 views

Goodness of fit in logistic regression where features are not frequencies

I am fitting a logistic regression model with a set of features for predicting outcomes in football games (three outcomes available: home wins, away wins or draw). The features are such as difference ...
0
votes
1answer
75 views

Python Text Classification Features Engineering

I am trying to train a model on text classification. I have a large labeled dataset. Documents are set of comments, notes on a incident. Labels are high level categories for the incidents. As ...
4
votes
1answer
180 views

Why can't ridge regression provide better interpretability than LASSO?

I already have an idea about pros and cons of ridge regression and the LASSO. For the LASSO, L1 penalty term will yield a sparse coefficient vector, which can be viewed as a feature selection method. ...
0
votes
1answer
46 views

What is the best choice of algorithms for good supervised classification in our data?

I'm a novice in machine learning-based classification techniques. Please do help 1) What is the difference between SMO (weka) and the LibSVM algorithms? Which is the best? Because the parameter ...
0
votes
1answer
59 views

What methodology does proc varclus use to reduce the number of variables

In statistics, we can use methods like principal component analysis, linear discriminant analysis for variable reduction. In SAS, there is a proc called VARCLUS which is used for variable reduction. ...
0
votes
2answers
45 views

Advice on feature selection

I have a large universe of features, and potentially a large universe of targets that I want to use them for. I need to construct some kind of summary stats that ranks the features by their relevance ...
1
vote
0answers
30 views

What does error mean in this context?

In the paper: Avrim L. Blum and Pat Langley. 1997. Selection of relevant features and examples in machine learning. Artif. Intell. 97, 1-2 (December 1997), 245-271. are several definitions of the ...
2
votes
0answers
85 views

How to use LDA results for feature selection?

I am working on the Forest type mapping dataset which is available in the UCI machine learning repository. I have 27 features to predict the 4 types of forest. I am performing a Linear Discriminant ...
2
votes
3answers
134 views

Mixing unsupervised and supervised learning

Is it a good idea to mix supervised and unsupervised learning together? I'm trying to predict some sales data given some data on hand, so I think a regression is the best way to go. However, I'm also ...
2
votes
0answers
50 views

Selecting multiple hyper-parameters via successive nested cross-validation

Selecting multiple hyper-parameters via successive nested cross-validation I am currently working in a classification task on motion data. Each sample to classify is represented by a set of features ...
0
votes
1answer
50 views

Oja rule incorrect pcs

I am trying to get sequentially pc from oja's rule. But the error seems to increase with pcs, which shouldn't happen. Please let me know where the error, I can't seem to get it. I am intialising ...
2
votes
1answer
21 views

Do features remain informative across domains and predicted variables?

For example, consider a feature like "wait_time_in_queue" is important (informative) in predicting whether a customer will return to a restaurant. Will the same feature be important for another ...
3
votes
1answer
128 views

What does it mean that stepwise, backward and forward selection methods are “path dependent”?

In many papers I read that stepwise, backward and forward selection methods are "path dependent". What does it mean? Could anyone give me some practical example to understand the underlying concept? ...
1
vote
1answer
65 views

Feature selection and PCA in logistic regression with rare events data

I am working on a census-related project where I am interested in assigning to everyone a score that estimates the probability they will have a demographic characteristic of interest. In this case, ...
0
votes
0answers
63 views

Feature selection using p-values

I am working on a regression model and I have 26 features. I want to select a subset of important/significant features, not necessarily the ones that help prediction. In other words I want to find a ...
3
votes
2answers
159 views

Feature selection for clustering problems

I am trying to make group together different datasets using unsupervised algorithms (clustering). The problem is that I have many features (~500) and a small amount of cases (200-300). So far I used ...
0
votes
0answers
44 views

Regression problem with observations and uninformative features

You have a regression problem with $n = 100$ observations and $p = 2000$ features, and you are told by your collaborator that most of the features are likely to be uninformative (but he/she ...
7
votes
0answers
93 views

Speed, computational expenses of PCA, LASSO, elastic net

I am trying to compare computational complexity / estimation speed of three groups of methods for linear regression as distinguished in Hastie et al. "Elements of Statistical Learning" (2nd ed.), ...
1
vote
2answers
36 views

Visualizing Nodes in a Neural Network (Dimensionality Reduction?)

Imagine we are working with the MNIST dataset and creating a neural network with 1 hidden layer. So we have a vector of 784 inputs, 100 hidden nodes, and 10 outputs. If we were to visualize each ...
0
votes
1answer
31 views

Selecting features to find predefined groups in clustering

I have a dataset I believe is easily clustered into a few groups, however, a bunch of junk features interfere with clustering. Is there a method of eliminating these bad features within this dataset? ...
2
votes
0answers
152 views

Paradox in model selection (AIC, BIC, to explain or to predict?)

Having read Galit Shmueli's "To Explain or to Predict" (2010) I am puzzled by an apparent contradiction. There are three starting points, AIC- versus BIC-based model choice (end of p. 300 - start of ...
0
votes
1answer
46 views

stepwise, forward and backward selection when the regressors are too much correlated

Why automated variable selection methods like stepwise regression, backward elimination and forward stepwise regression are not suitable when the regressors suffer from multicollinearity? Could anyone ...
0
votes
0answers
30 views

Classifier puts everything into the same class, inspite of reasonable distribution?

First of all I'd like to describe the characteristics of the problem I'm working on, the things I've tried, and the problem I'm running into. I'm attempting to assess a client's propensity to pay for ...
1
vote
0answers
26 views

Multiclass classification problem: how to analyze data to determine affect on precision?

I have a multiclass text classification problem being solved by SVM. There are $K$ mutually exclusive classes but my task is to improve precision while maintaining reasonable recall of only $k \ll K$ ...
0
votes
1answer
91 views

Standard logistic regression post-Lasso

The situation I'm interested in is logistic regression for a binary response variable with lots of predictors (500 to 1000), lots of which are correlated. I would like to use a logistic LASSO approach ...
0
votes
1answer
86 views

Feature selection on large file with missing categorical and numerical data in R

I know that there have been similar questions but most of them have not worked, this is why I start this question. I have a very large dataset (around 2,500,000 records) with approximately 100 ...
1
vote
1answer
48 views

Combining several features into one single feature

I would like to know what might be the side effects of combining several features into one single feature for classification tasks. Imagine I have two variables with the following domains: A is ...
0
votes
0answers
26 views

Which classifiers need feature scaling? [duplicate]

Is feature scaling (for example mean normalization) always preferred or can it sometimes lead to poor accuracy? How do I know whether I should perform feature scaling or not? Which tasks/classifiers ...
1
vote
1answer
52 views

how to find contribution of feature in accuracy

Given a set of features, how do I find what all features are contributing how much to the accuracy / prediction ?
0
votes
0answers
18 views

Use of Correlation

How do we use a correlation score between two variables for analysing data? I have a set of 20 features and need to predict 21st feature. Now is it necessary that correlation between any two features ...
0
votes
0answers
17 views

Why sparse regression model fail to select informative features?

I have data set with few thousands data points and around 800 features. My features basically belong to different sets (for example 300 of them belongs to S1 and the rest is S2). All values of my ...
0
votes
0answers
21 views

What is the simplest way to find out if one feature can be replaced by another one?

I use a set of features as an input to a machine learning algorithms. Now, I have a question if a feature A can be replaced by feature B (because it is easy to measure feature B). The direct way to ...
0
votes
0answers
17 views

How to set bin size for computing mutual information from continuous variable?

I want to compute the mutual information between features and my output variable. I was wondering, what is the best way to set the number of bins for each feature? How should be bins interval ? Does ...
0
votes
0answers
21 views

Column space of within class scatter matrix $S_W$

Having a data set that contains $c$ classes where each class contains $n_i$ $d$-dimensional data samples with a total number of samples $M$. Let $x_{m}^{i}$ be a the m-th $d$-dimensional data ...
4
votes
1answer
77 views

Alternatives to stepwise regression for generalized linear mixed models

Are there any easy to use alternatives to stepwise variable selection for GLMMs? I have seen implementations of e.g. LASSO for linear regression, but so far not seen anything for mixed models. Mixed ...
2
votes
1answer
38 views

Feature normalization independent from test data

I know that it is good practice to perform normalization (subtracting the data by its mean and dividing it by its standard deviation) first on the training data, and in a later step to use the mean ...
0
votes
0answers
9 views

Determine the attribute type

I have an attribute that represents a proportion expressed in percentage. According to Stevens' rules the variable would not be ratio - instead interval. But since interval scales do not have the ...
0
votes
0answers
14 views

Ordinal target feature selection

I am looking for a way to do feature selection for ordinal target (in R language: where the target variable is ...
1
vote
0answers
18 views

Dimensionality reduction for narrow, tall matrices

I have a matrix with three columns and a lot of rows. The first two columns contain integers N1 and N2. N1 is always {0,1,2,3}, but N2 can vary from let's say -50 to 50. It's always the same N1s, but ...
0
votes
0answers
25 views

Determining attribute type and level of measurement

I know that we when we discuss attribute types, we distinguish between nominal, ordinal, interval and ratio. I have a data set with these attributes: price: median housing price ...
0
votes
1answer
30 views

Advice on predicting continuous dependent variable

My challenge: maximize $R^2$ on an out of sample data set. Constraints: Continuous dependent variable with negative values Over 150 variables with no information about them Some of these ...
3
votes
0answers
48 views

How can I estimate the influence/significance of the every observation on classification?

There are many ways to estimate the significance of the features on the classification model. But how I can estimate the influence of the every observation on the classification quality? My thinking ...
0
votes
2answers
57 views

Determine the attribute type

I need to determine which type of attribute the median of housing price in USD is among the attribute types nominal, ordinal, interval, and ratio. I would assume, since it is a continuous variable, ...
0
votes
1answer
86 views

How to perform binning in order to discretize continuous features for feature selection in R?

If I wanted to use uncertainty measures e.g. information gain for feature selection continuous features need to be discretized. How can I do this in R? Thank you for your help.
1
vote
0answers
24 views

Accounting for variation between populations when comparing failure rates

Here's my situation. The big picture is that I need to figure out how to compare the failure rates of two groups of widgets, Group A (defect) and Group B (control), in such a way that answers the ...