Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

0
votes
0answers
6 views

Can normalized values and original values be combined in a feature vector classification?

I standardized feature vector before SVM classification in MATLAB. The feature vector consists of time domain signal features and its normalizations i.e., feature vector is a combination of actual ...
1
vote
1answer
11 views

Feature extraction vs Fine tuning with Restricted Boltmann Machines

I am reading a paper which uses a Restricted Boltzmann Machine to extract features from a dataset in an unsupervised way and then use those features to train a classifier (they use SVM but it could be ...
0
votes
0answers
12 views

Haul's Correlation-based Feature Selection (CFS) formula spread

I want to use the Correlation-based Feature Selection (CFS) proposed by Haul. I found this formula where $r_{zc}$ is the correlation between the summed components and the outside variable, $k$ is ...
0
votes
0answers
9 views

How to do variable selection for Gradient boosting models like Xgboost and LightGBM

I am building a classification model with about ~110 variables and that gave me an AUC of about 71.96 on validation. I added about 10 more features and my AUC value decreased to 71.56 (which led to ...
0
votes
0answers
19 views

How to add new features to already trained model without training again on whole dataset?

Suppose, we have following features on which a classification model (Neural Network) is trained to predict whether a customer will buy Milk or not (0 :Will not buy, 1:Will buy) each week(n): ...
0
votes
0answers
42 views

Feature Selection For Random Forest

Random Forest aims to combine many decision trees to make good predictions for testing data in regression and classification. It is an ensemble learning method. I have a dataset with 100 samples, ...
0
votes
1answer
23 views

What is meant by Low-Order combination of features?

I came across a Machine Learning paper that talks about input with low-order combination of features. A statement says: The initial feature is used as the input of the model, and the non-linear ...
0
votes
0answers
13 views

Chi Square Test for Dimensionality Reduction

According to many resources, we should have categorical variable to be able to apply chi square test. ...
0
votes
1answer
31 views

Dimensionality Reduction - Feature Selection

For example, we have a dataset in which the samples contain 400 features. In this case, if we try to perform classification, we get very low accuracy because our learning model will become very ...
0
votes
0answers
12 views

Feature analyzing methods

I am a little confiused how to interpret following situation: I am trying to implement a image classification task using hog+SVM. For that i tried to analyze and understand the properties of the ...
0
votes
2answers
61 views

Choosing model for more predictors than observations

I'm working with a data consisting of 1000 observations of 2000 predictors and one variable we wish to predict. There are couple of problems I can't get around. I am aware that such setting has been ...
0
votes
0answers
7 views

Avoid learning certain known features / selecting alternative features in neural network training

In an application of neural network to a classification problem, often time one trains the network to pick out different features in the input data set (learnt by the hidden units) and classify the ...
0
votes
0answers
5 views

Partial Least squares regression - Variable Importance on Projection (VIP) method of selecting variables

I understand that partial least squares regression produces VIP scores for each predictor variable enabling variable selection (using a VIP threshold of >1). Does this method account for collinearity ...
1
vote
0answers
14 views

In Regression Analysis, should variable transformations occur before or after subset selection?

I'm looking at fitting a model that has many parameters. In order to simplify the model and prevent overfitting, I am planning to use the best subset selection for variable selection. My question is, ...
0
votes
0answers
10 views

Filter-based feature selection for binary classification with unbalanced classes

I have a data set with ~10k observations and ~50 features. Each observation is assigned one of two classes (labeled 0 and 1, say). Approximately 98% of the observations are class 0, and the remaining ...
0
votes
0answers
10 views

How can Item Response Theory be used to remove questions asked in a customer satisfaction survey?

I have results from a survey of around 30 Likert-style questions that are asked of customers on their opinion about company X. Each of the 30 questions belongs to a certain category. For example, ...
1
vote
0answers
7 views

factor of safety

I need to validate a specification on a part. The requirement is to withstand a minimum 1 kg of force (i.e. not fail at less than 1 kg). I was told that if I measure, on 6 specimens, a force of 2.5 ...
0
votes
0answers
32 views

Application of GAM on large dataset

I was suggested that my questions were too broad. As I commented below, I have nearly a million data points and perhaps a hundred variables. This may be a very basic modeling question: I am curious to ...
0
votes
0answers
8 views

When doing pca on a specific cnn feature map, is the pca substance always the same regardless of the input?

Lets say I have a pretrained CNN, and I extract the feature map from one of the layers for some input data x. I do the same thing for for a second set of input data y, which is very different from ...
0
votes
0answers
6 views

In R, why do they have so different performance between RF models with reduced feature vs ones built manually?

For a regression problem, I'm comparing two randomforest (RF) models: (a) A resulted model after doing feature selection by caret::rfe(). As I know, the ...
0
votes
0answers
28 views

Will LASSO choose variables that are highly correlated with the outcome variable?

Suppose we have access to an outcome variable $Y_i$ and a $p$-dimensional vector $X_i$ for $i=1,\ldots,N$. We run a LASSO regression of $Y$ on $X$ for every penalty/shrinkage parameter $\lambda$ in an ...
0
votes
0answers
12 views

Calculating Variable Importance for Feature Selection - PLSR

I have used the plsr() function in R (from the pls package) to predict a Y variable using many X variables (spectral bands) - and am wanting to calculate variable importance (ViP) to begin to reduce ...
0
votes
0answers
18 views

Cross Validation and Feature Selection with Chronological Split and Feature Preprocessing

I have a task with daily entries for which I need to do binary classification. Suppose I have 18 months of data and the model is refit every month. In addition I've got about 150 one-hot encoded ...
0
votes
0answers
10 views

Variable selection with standardised variables

Recently I performed a lasso regression on a set of 1000 standardised time series variables to select variables to use in a linear regression model. I used the non-standardised original form of the ...
0
votes
0answers
26 views

Feature engineering suggestion

I've to forecast the revenue generated by a company on a monthly basis. The dataset looks like this: ...
0
votes
0answers
13 views

In the book “Applied Predictive Modeling”, why it repeats over the same fold when doing recursive feature elimination?

I'm trying to conduct recursive feature elimination (RFE) referring to Ch19.7 of the book "Applied Predictive Modeling" by Max Kuhn. In the book, it uses 5 repeated 10-fold cv for RFE. To do this, it ...
0
votes
0answers
8 views

Finding most related variables independent of variable type

Problem I am analysing a dataset containing variables of different types: continuous, ordinal and categorical. To prioritise in which order to analyse the variables, I would like to evaluate the ...
3
votes
1answer
212 views

Using Random Forest variable importance for feature selection

I'm currently trying to convince my colleague that his method of doing feature selection is causing data leakage and I need help doing so. The method they are using is as follows: They first run a ...
0
votes
0answers
9 views

How to infer results from tree-based feature selection and chi squared?

I have a data set with 12 continuous features, with 3 discrete output labels. I want to determine the two best features. My research thus far has led me to use chi^2 tests and extra tree classifiers ...
0
votes
0answers
31 views

How to identify and reduce question overlap and redundancies in a survey? (remove questions asked for a more concise survey, w/o losing information)

Suppose I have a survey that contains 30 items. The items ask about the relationship between the respondent and their family, in many different realms. For example, the strength of the connection ...
0
votes
0answers
13 views

cross-validation: feature selection and hyperparameter tuning. Is nesting necessary?

I am a little bit confused by the use of feature selection inside a K-fold CV together with hyperparameter tuning. So I have my dataset. I split in training & test as usual, and work on training ...
1
vote
1answer
22 views

Forward Model Selection Using p-value

I know that it is not advised to use the p-value as the criterion in practice, but I am not asking about that. I am wondering how this p-value would actually be calculated. In other words: Forward ...
2
votes
1answer
38 views

Bivariate analysis as a basis for a subsequent analysis?

I have run across many research articles which used bivariate analysis, whose results become the basis for a subsequent analysis. For example, a Chi-squared test was used as a preliminary analysis to ...
0
votes
0answers
18 views

Dealing with variable length time-series data for linear models

I am working with variable length time-series signals. I want to use a sliding window to extract features, things like mean, standard deviation, kurtosis, skewness. The length varies pretty ...
0
votes
0answers
14 views

Feature selection in parameter optimization

Let't say that $\theta$ is a vector or real numbers of the form $(\theta_1, \theta_2,\theta_3, ...,\theta_n)$ and $Obj(\theta)$ is a continuous function (objective function). Let's further say that I ...
1
vote
1answer
23 views

Model that can make a prediction (classification) based on a sub set of features

Model that can make a prediction (classification) based on a sub set of features What is the recommended approach, best model or algorithm that handles use-case where we want to predict based on sub-...
0
votes
0answers
9 views

Automating Feature Selection

Currently trying to use Monte-Carlo Tree Search to automate the process of feature selection for SVM (which I'm using to evaluate my features). Although successful so far, It tightly depends on the ...
0
votes
0answers
27 views

Z score standardisation vs min-max scaling for feature selection

I am applying l1 norm on the input weights of a single layer MLP. I wanted to know if I should standardize or min-max scale ([0 1] feature scaling) my input data?
1
vote
1answer
23 views

How does feature selection work for non linear models?

A model like a neural network or an SVM is called for only if the interactions between the features and the target is non-linear, otherwise we're better off using linear or logistic regression. But ...
0
votes
0answers
33 views

Feature Selection using Monte Carlo Tree Search

Trying to tackle the problem of feature selection as an RL problem inspired by this paper here: https://hal.inria.fr/inria-00484049/document So I used Monte-Carlo Tree search for this problem, where ...
1
vote
1answer
43 views

Do decision tree's perform variable selection?

I'm a bit confused how decision tree's select the variables to split. I know they splitt the data set through variable to get a more pure data set. But can it happend that some explenatory variables ...
1
vote
3answers
59 views

LASSO or random forest (RF) to use for variable selection when having highly correlated features in a relatively small dataset with many features?

I have a cross sectional data-set with around 1000 features and 5000 observations. There are many features (no categorical features) which are highly correlated (higher than 0.85). I want to decrease ...
3
votes
1answer
79 views

Which machine learning algorithms get affected by feature scaling?

Which of the following machine learning algorithms will be affected if we apply feature scaling? Naïve-Bayes k-Nearest Neighbor (KNN) Support Vector Machine (SVM) Decision Trees Neural Network (...
1
vote
0answers
16 views

How to use Machine Learning to discover important biomarkers in an unbalanced small data set

I have a project which I am just starting out, I am only just learning machine learning and statistics so I am somewhat unsure as to what approaches will be good to start off with, and I am sorry if ...
0
votes
1answer
46 views

Are legacy values useful for regression models?

I'm building a model that predicts house prices in order to learn some regression techniques. Currently I'm trying to engineer features that might be significant when predicting prices. I got a hold ...
0
votes
1answer
31 views

On starting feature engineering

I would like to start my feature engineering process by first selecting a subset of features that are highly correlated with the target feature. However, if I do select let’s say the top k in terms ...
0
votes
1answer
14 views

Correlation Analysis and Data Leakage

In machine learning, we perform feature engineering and selections in pipelines and crossvalidate to obtain results in order to avoid data leakage and avoid introducing prior knowledge into the ...
1
vote
0answers
15 views

How to calculate OOB error vs. features plot

I have dataset (numeric values from 30 th number raster brick and some classes for every pixel. (class value was taken from RF classification result)). Well I know how to calculate importance of ...
0
votes
0answers
11 views

Feature Selection in Machine Learning [duplicate]

Is it appropriate to conduct feature selection (e.g., Recursive Feature Selection) on a data set IN ADVANCE of model fitting to scale down features for more expedient machine learning model fitting? ...
0
votes
2answers
64 views

Feature selection using PCA for linear regression

I am using PCA to the training data set to do feature selection before applying linear regression to build a classifier model. In this scenario, would it be useful to use ridge regression to ensure ...