Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

9
votes
1answer
2k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
0
votes
0answers
18 views

Bayesian Variable Selection with NMIG

I have a Bayesian linear model like this: $Y_i = X_i*\beta + \epsilon_i$ . Just for completion: ($\epsilon_i \sim N(0,\sigma^2)$ $\beta \sim N_p(b_0,B_0)$, $\sigma^2 \sim Inv-Gamma (a,b)$) I would ...
0
votes
0answers
17 views

how to preprocess/feature stacle multimodal input data?

I am wondering how to normalize data for the use of SVMs etc. that has a clear non Gaussian, i.e. non unimodal distribution. I wrongfully scaled the data by subtracting the mean and dividing the std ...
0
votes
2answers
173 views

Feature selection clustering customer segmentation

based on customer data I want to perform a clustering using different clustering algorithms (K-Means, Expectation Maximization, etc.) in R. The most attributes were engineered pursuing the goal to be ...
6
votes
2answers
2k views
1
vote
0answers
44 views

Assessing feature importance without random forests

What ways are there to assess variable (feature, covariate) importance in regression models, except for using random forests? (For instance, using OLS regression, Bayesian parametric regression, etc.?...
0
votes
1answer
197 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
0
votes
1answer
20 views

Advance Methods of Understanding Significance of Customer Behaviors

I currently own a couple of websites and lately I've been implementing some feature changes - I've noticed some changes in website traffic and I was wondering what some of the more sophisticated ways ...
28
votes
2answers
655 views

A more definitive discussion of variable selection

Background I'm doing clinical research in medicine and have taken several statistics courses. I've never published a paper using linear/logistic regression and would like to do variable selection ...
0
votes
0answers
24 views

How to deal with categorical target variable that has more categories in prediction than training?

I'm building a logistic regression model and found out that with my categorial target variable there are more categories in my prediction set than my training set. To be clearer: In e.g. my training ...
1
vote
1answer
17 views

what is the method in dictionary learning which does not have a overcomplete dictionary?

what is the method in dictionary learning which does not have a overcomplete dictionary? and what is the difference in minimization between these two methods (one using overcompelte dictionary and ...
0
votes
0answers
11 views

Correlation-based feature selection for regression problem

I am working on a regression problem in which each data sample has a covariate vector $x_i$ and a response variable $t_i$. Intuitively, each feature value $x_{i,j}$ in my problem should have a ...
1
vote
1answer
35 views

Nested cross-validation and feature selection: when to perform the feature selection?

I am trying to predict a behavioral variable using neuroimaging data using supporting vector regression. Since there are ~ 400.000 voxels (=features) in an image and I have a limited sample size I ...
0
votes
0answers
11 views

How to group/cluster variables/features using Python? [on hold]

I have 200 variables and 1 million records. I want 20 clusters, so that I can pick out top variable (based on Information Values of the variables) in each cluster.
9
votes
1answer
812 views
+50

What to conclude from this lasso plot (glmnet)

Following is the plot of glmnet with default alpha(1, hence lasso) using mtcars data set in R with mpg as the DV and others as ...
2
votes
1answer
208 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
3
votes
1answer
116 views

What to do when lasso does not remove correlated variables?

The very essence of lasso is that it is supposed to select only one of two correlated variables. However, when I include two highly correlated predictors (they are correlated with each other at level ...
0
votes
0answers
3 views

RFE (Recursive Feature Elimination) for Poisson Regression with offset [migrated]

It's my first post so I hope I don't make any editing mistakes. Here's my issue : I'm working on count data and am implementing a Poisson Regression with an exposure factor (that needs to go in the "...
1
vote
0answers
19 views

Which method for variable selection for multivariate data?

I have a dataset with 299 observations, 35 independent and 141 dependent variables. This is a vegetation dataset, the IVs are environment variables, the DVs are coverage of 141 species (of course many ...
2
votes
2answers
668 views

Event Prediction through Machine Learning

I have a large data set consisting of ca. 40 categorical data items and a few interval data items (real numbers, less than 5 such items). Most categories should have a lot of values that repeat ...
0
votes
3answers
48 views

Standardizing some features in K-Means

I have 21 features in my dataset, some features are more important than others. As a fact I know, if I don't standardize (mean=0, SD=1) any features, then features with low variance will have slightly ...
1
vote
1answer
239 views

Python Text Classification Features Engineering

I am trying to train a model on text classification. I have a large labeled dataset. Documents are set of comments, notes on a incident. Labels are high level categories for the incidents. As ...
0
votes
0answers
14 views

How to use same metric in rfe and train?

I'm running a feature selection together with a model tuning using caret's rfe and train methods on a multi-class problem. I would like to select my features in rfe, as well tune my model parameters ...
0
votes
1answer
194 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
1
vote
2answers
65 views

Am I performing feature selection correctly?

I'd like to design a feature extraction, selection, and classification scheme to use on novel data sets. For each row in a table I calculate 10 features. I then select which features are relevant (...
0
votes
1answer
46 views

Identical variable importance values for different model types

I trained two different caret models on the same multi-class training data using repeated cross validation and computed the variable importance. What strikes me, is that for both models varImp returns ...
1
vote
0answers
49 views

how to assess importance of each predictor in robust linear regression

I have been using rlm() in the MASS library in R with the redescending weights (using MM or Tukey's biweight function). I wanted to find the importance of each predictor in the fitted model. Can ...
0
votes
1answer
245 views

MATLAB function TreeBagger() (Random Forest classification) and different number of variables

I am using MATLAB function TreeBagger() for Random Forest classification, for an assignment. It gives error when the number of variables of the Test data is different from the number of variables of ...
0
votes
0answers
8 views

Suggested methods when logistic regression outperforms with boosted outliers

I am using logistic regression to predict binary outcomes with 5 features. When putting 20x weight on the 0.001% outliers the peformance gets a lot better. It seems that some really high/low values ...
0
votes
1answer
20 views

Chossing an algorithm when there is only one feature

I am combining multiple base classifiers for an ensemble classifier. Different sensors, such as an accelerometer, gyroscope and altimeter are classified individually, and their outputs are then fed ...
3
votes
2answers
2k views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation coefficient,...
3
votes
1answer
281 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
3
votes
3answers
2k views

The disadvantage of using F-score in feature selection

F-score can be used to measure the discrimination of two sets of real-numbers and can be used for feature selection. However, I once read that A disadvantage of F-score is that it does not reveal ...
0
votes
0answers
19 views

Number of Sinusoids Feature Extraction for MUSIC Algorithm

I am a complete novice to machine learning, but I wanted to ask this question to get me started on a path to solving this problem. I have been working on a radar project where I am receiving a signal ...
2
votes
0answers
37 views

How to improve the sensitivity of minority class on imbalanced datasets

I am working on a classifier which stratifies a population of samples into different classes. The class distribution (ground truth) is imbalanced, and the prevalence of each class is: $$\begin{...
0
votes
0answers
28 views

Appropriate procedure regarding the preprocessing of datasets as an input of testing a classifier in R

I would like to test a 39 gene signature that i have identified, through a feature selection procedure in R-based on a training microarray dataset-, in some independent datasets, regarding its ...
1
vote
1answer
42 views

covariate selection in inference problems in logistic regression

For my specific problem, but a common situation in the medical field, I have several hundred patients, and about 10-20 exaplnatory variables. the goal is to examine a specific predictor("treatment") ...
1
vote
3answers
325 views

Evaluating features and similarity measures

I am currently developing a classifier, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
0
votes
0answers
21 views

Feature map of kernel

Let $K_1,K_2$ be valid kernels. Kernel $K_1$ has a feature map $Θ(x)∈R^{50}$ while Kernel $K_2$ corresponds to feature map $ψ(x)∈R^{10}$, satisfying: $∀i=1..10,∀x:ψ_i (x)=0.2 Θ_i (x)$ What is the ...
1
vote
0answers
56 views

Using PCA to model highly correlated variables

Specifically, Andrew Ng states that PCA should be used to speed up algorithms or to visualize data. He also states that using PCA as a way to prevent overfitting is an incorrect application of PCA. ...
3
votes
0answers
63 views

Is there an appropriate order to apply bagging and filter feature selection?

I'm training a (regression) learner on a $p \gg n$ problem, including bagging and filter feature selection (information gain). I'm in doubt though regarding the order of the procedures: Apply the ...
2
votes
0answers
30 views

Is stagewised feature engineering/ selection an invalid approach?

Suppose we want to build a regression or classification model. However, the features (independent variables used) are not all ready at one time. This is very realistic in business, because the data ...
0
votes
1answer
136 views

Feature selection with a binary dependent variable

Given we have a binary dependent variable and 100s of features and ~50k observations, is there a generally accepted way to trim the features via some type of machine learning concept? I was trying a ...
2
votes
0answers
58 views

PCA for questionnaire reduction

I'd like to have some opinion regarding if I'm in the right way with my questionnaire reduction. I have a questionnaire with 275 questions and 34 issues (so a couple of questions are related to each ...
0
votes
1answer
36 views

Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
0
votes
0answers
12 views

CNN extract information out of several feature maps?

I've trained a weakly supervised convolutional neural network. It's convolutional layers are initialized with the weights of a fully-supervised CNN and the weakly supervised network performance is ...
2
votes
1answer
152 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
3
votes
4answers
353 views

Logistic Regression: Does my model selection process make sense?

This is kind of a broad question and so I am okay with broad or general answers. In fact, each of these could be their own individual questions, but I think it makes sense to ask them all. Even if you ...
2
votes
1answer
175 views

Machine learning tutorials / examples on data sets larger than a terabyte

I am trying to gather a list of practical ML examples / tutorials on more than a terabyte of data. I'm particularly interested in feature extraction from large data sets that involves aggregation (the ...
2
votes
0answers
33 views

Feature Selection with Categorical Variables: Multicollinearity and Statistical Significance

Building a logistic regression model with three categorical features and one continuous. For simplicity, let's say I have the following features and variables: ...