Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
7 views

Scikit Learn Python PCA and Feature Agglomeration feature indices?

I am using Scikit learn to perform PCA and Feature Agglomeration of my survey data. I have done the following : ...
0
votes
0answers
27 views

Feature scaling of non-Gaussian data before SVM

I have data for a binary classification problem and was wondering generally what to do if the different dimensions/features of your input training data display vastly different distributions in ...
0
votes
0answers
17 views

Best deep learning method of feature extraction for brain fmri region of interest data

I am trying to build a binary classfier(healthy or not healthy) for the ADHD nneurological disorder. I have a small dataset known as the ADHD-200 consortium which has a total of 750 mri scans(...
0
votes
0answers
40 views

how to preprocess/feature stacle multimodal input data?

I am wondering how to normalize data for the use of SVMs etc. that has a clear non Gaussian, i.e. non unimodal distribution. I wrongfully scaled the data by subtracting the mean and dividing the std ...
0
votes
0answers
20 views

Bayesian Variable Selection with NMIG

I have a Bayesian linear model like this: $Y_i = X_i*\beta + \epsilon_i$ . Just for completion: ($\epsilon_i \sim N(0,\sigma^2)$ $\beta \sim N_p(b_0,B_0)$, $\sigma^2 \sim Inv-Gamma (a,b)$) I would ...
1
vote
0answers
49 views

Assessing feature importance without random forests

What ways are there to assess variable (feature, covariate) importance in regression models, except for using random forests? (For instance, using OLS regression, Bayesian parametric regression, etc.?...
0
votes
1answer
22 views

Advance Methods of Understanding Significance of Customer Behaviors

I currently own a couple of websites and lately I've been implementing some feature changes - I've noticed some changes in website traffic and I was wondering what some of the more sophisticated ways ...
0
votes
0answers
27 views

How to deal with categorical target variable that has more categories in prediction than training?

I'm building a logistic regression model and found out that with my categorial target variable there are more categories in my prediction set than my training set. To be clearer: In e.g. my training ...
0
votes
0answers
13 views

Correlation-based feature selection for regression problem

I am working on a regression problem in which each data sample has a covariate vector $x_i$ and a response variable $t_i$. Intuitively, each feature value $x_{i,j}$ in my problem should have a ...
0
votes
0answers
13 views

How to group/cluster variables/features using Python? [closed]

I have 200 variables and 1 million records. I want 20 clusters, so that I can pick out top variable (based on Information Values of the variables) in each cluster.
0
votes
0answers
3 views

RFE (Recursive Feature Elimination) for Poisson Regression with offset [migrated]

It's my first post so I hope I don't make any editing mistakes. Here's my issue : I'm working on count data and am implementing a Poisson Regression with an exposure factor (that needs to go in the "...
1
vote
0answers
21 views

Which method for variable selection for multivariate data?

I have a dataset with 299 observations, 35 independent and 141 dependent variables. This is a vegetation dataset, the IVs are environment variables, the DVs are coverage of 141 species (of course many ...
28
votes
2answers
677 views

A more definitive discussion of variable selection

Background I'm doing clinical research in medicine and have taken several statistics courses. I've never published a paper using linear/logistic regression and would like to do variable selection ...
0
votes
0answers
16 views

How to use same metric in rfe and train?

I'm running a feature selection together with a model tuning using caret's rfe and train methods on a multi-class problem. I would like to select my features in rfe, as well tune my model parameters ...
1
vote
1answer
36 views

Nested cross-validation and feature selection: when to perform the feature selection?

I am trying to predict a behavioral variable using neuroimaging data using supporting vector regression. Since there are ~ 400.000 voxels (=features) in an image and I have a limited sample size I ...
0
votes
3answers
48 views

Standardizing some features in K-Means

I have 21 features in my dataset, some features are more important than others. As a fact I know, if I don't standardize (mean=0, SD=1) any features, then features with low variance will have slightly ...
1
vote
2answers
68 views

Am I performing feature selection correctly?

I'd like to design a feature extraction, selection, and classification scheme to use on novel data sets. For each row in a table I calculate 10 features. I then select which features are relevant (...
0
votes
0answers
8 views

Suggested methods when logistic regression outperforms with boosted outliers

I am using logistic regression to predict binary outcomes with 5 features. When putting 20x weight on the 0.001% outliers the peformance gets a lot better. It seems that some really high/low values ...
0
votes
1answer
21 views

Chossing an algorithm when there is only one feature

I am combining multiple base classifiers for an ensemble classifier. Different sensors, such as an accelerometer, gyroscope and altimeter are classified individually, and their outputs are then fed ...
1
vote
0answers
50 views

how to assess importance of each predictor in robust linear regression

I have been using rlm() in the MASS library in R with the redescending weights (using MM or Tukey's biweight function). I wanted to find the importance of each predictor in the fitted model. Can ...
0
votes
0answers
21 views

Number of Sinusoids Feature Extraction for MUSIC Algorithm

I am a complete novice to machine learning, but I wanted to ask this question to get me started on a path to solving this problem. I have been working on a radar project where I am receiving a signal ...
0
votes
0answers
28 views

Appropriate procedure regarding the preprocessing of datasets as an input of testing a classifier in R

I would like to test a 39 gene signature that i have identified, through a feature selection procedure in R-based on a training microarray dataset-, in some independent datasets, regarding its ...
0
votes
0answers
21 views

Feature map of kernel

Let $K_1,K_2$ be valid kernels. Kernel $K_1$ has a feature map $Θ(x)∈R^{50}$ while Kernel $K_2$ corresponds to feature map $ψ(x)∈R^{10}$, satisfying: $∀i=1..10,∀x:ψ_i (x)=0.2 Θ_i (x)$ What is the ...
1
vote
0answers
61 views

Using PCA to model highly correlated variables

Specifically, Andrew Ng states that PCA should be used to speed up algorithms or to visualize data. He also states that using PCA as a way to prevent overfitting is an incorrect application of PCA. ...
3
votes
0answers
65 views

Is there an appropriate order to apply bagging and filter feature selection?

I'm training a (regression) learner on a $p \gg n$ problem, including bagging and filter feature selection (information gain). I'm in doubt though regarding the order of the procedures: Apply the ...
0
votes
0answers
12 views

CNN extract information out of several feature maps?

I've trained a weakly supervised convolutional neural network. It's convolutional layers are initialized with the weights of a fully-supervised CNN and the weakly supervised network performance is ...
1
vote
1answer
43 views

covariate selection in inference problems in logistic regression

For my specific problem, but a common situation in the medical field, I have several hundred patients, and about 10-20 exaplnatory variables. the goal is to examine a specific predictor("treatment") ...
2
votes
0answers
59 views

PCA for questionnaire reduction

I'd like to have some opinion regarding if I'm in the right way with my questionnaire reduction. I have a questionnaire with 275 questions and 34 issues (so a couple of questions are related to each ...
2
votes
0answers
30 views

Is stagewised feature engineering/ selection an invalid approach?

Suppose we want to build a regression or classification model. However, the features (independent variables used) are not all ready at one time. This is very realistic in business, because the data ...
2
votes
0answers
33 views

Feature Selection with Categorical Variables: Multicollinearity and Statistical Significance

Building a logistic regression model with three categorical features and one continuous. For simplicity, let's say I have the following features and variables: ...
3
votes
4answers
357 views

Logistic Regression: Does my model selection process make sense?

This is kind of a broad question and so I am okay with broad or general answers. In fact, each of these could be their own individual questions, but I think it makes sense to ask them all. Even if you ...
2
votes
0answers
39 views

How to improve the sensitivity of minority class on imbalanced datasets

I am working on a classifier which stratifies a population of samples into different classes. The class distribution (ground truth) is imbalanced, and the prevalence of each class is: $$\begin{...
1
vote
1answer
25 views

Feature varying in orders of magnitude

I have learned that you are supposed to scale features as a preprocessing step for most ML algorithms (so that all features are of the same magnitude and no direction is preferred, for example in ...
0
votes
1answer
46 views

Identical variable importance values for different model types

I trained two different caret models on the same multi-class training data using repeated cross validation and computed the variable importance. What strikes me, is that for both models varImp returns ...
0
votes
0answers
22 views

How to select important features with multiple datasets?

There are several feature selection methods for dataset, but it is difficult for me to find methods for multiple datasets. The example of my datasets is like below. Sensors are features, and they are ...
1
vote
0answers
40 views

Dimensionality reduction for multivariate time series

I have a data set including 25 variables $(x_{1,t},\dotsc,x_{25,t})$ at each time $t$ and all of this group is repeated through time. I want to explore the relationship between these variables through ...
0
votes
2answers
25 views

Correlation based feature selection(CFS) tool

Is there any tool or script that was implemented for correlation based feature selection? My feature vector data is in a large-scaled data file, so if I use tools like Weka for feature selection, I ...
1
vote
0answers
24 views

Why do loadings of princomp in R report identical proportion of variance for all principal components? [duplicate]

I'm trying to run a few tests using princomp in R. In princomp there is a value called <...
0
votes
1answer
32 views

Consequences of overlap between training, validation and test data

If I'm splitting my data in training, validation and test data to assess different (sub)sets of features for my task. What are the consequences if I (by mistake) split my data incorrectly? In the ...
4
votes
4answers
139 views

Can PCA allow to identify redundant variables that can be removed before doing cluster analysis?

I hope this is suitable for this forum: I am new to PCA and what I ultimately want to do is perform cluster analysis on my dataset. I have 20 physical descriptor variables for organisms, each with ...
1
vote
0answers
36 views

How the Correlation Matrix is built for PCA in Weka?

Just to give a context, I want to use PCA (Principal Component Analysis) to identify which attributes are similar to others, so I can use just one (or a subset) of them. The correlation matrix of n ...
0
votes
0answers
22 views

Correlation for feature selection in multi-dimensional time series

I have multi-dimensional time series data with seven dimensions. The correlation coefficient between two of these dimensions is about 0.65. Can those two variables be said to be/treated like being ...
0
votes
0answers
16 views

How to deal with missing data when calculating Information Gain

While working on a neural network for classification problem I'm dealing with huge number of possible features and information gain seems like a good way to narrow them down (there are hundreds of ...
0
votes
1answer
46 views

Feature importance in gradient boosted trees

I am tuning the parameters of a gradient boosting regression tree algorithm and find it hard to understand the importance of some variables. Here is the case.. when the number of estimators is ...
2
votes
1answer
67 views

Challenges in interpretation of variable selection from LASSO and OLS [duplicate]

I work as a consultant and I am often faced with variable selection and prediction problems. For my clients, I run OLS and I am recently pushing for penalized methods which can handle variable ...
0
votes
0answers
18 views

Can LARS or Coordinate Descent select features that are marginally uncorrelated with the response?

I could construct a response Y the following way: Given $\left\lbrace X_k \right\rbrace_{k=1}^p$, and the regression model $$ Y = \sum_{i = 1}^p X_i - \rho p \beta_{p+1}X_{p+1} + \varepsilon,$$ if ...
0
votes
1answer
21 views

Testing for feature importance with missing values

I'm looking for an appropriate model to do the following analysis: I'd like to test which courses are the most important in determining if a student stays in or leaves a university program. Imagine I ...
1
vote
0answers
18 views

Covariance-residual technique for linear regression feature selection

When doing forward feature selection for linear regression, it is a well known trick that to select the next feature to add, we can compute the covariance of each candidate feature against the current ...
0
votes
1answer
27 views

Selecting Subsets with Linear Correlation

I'm looking for a method of grouping 200+ samples with 30+ features into groups which share linear correlations among a subset of the features. I've found Ransac to sometimes return a good ...
2
votes
1answer
54 views

Can I use output of classifier A as feature for classifier B?

This is likely to be a confused question, but I'm curious if this is a valid way to combine classifiers. I have a classification data set, i.e. column of labels and N columns of features, and I use a ...