Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

Which are well-known/popular feature selection methods? [duplicate]

I am trying to test the performance of first reducing the number of features before applying methods such as neural networks for prediction. Due to the fact that the number of observations is not ...
user avatar
1 vote
0 answers
16 views

features from random forest classifier vs regressor

I have built random forest classifier and regressor models on same data, target and independent variables. For classifier, I am giving a classification parameter that divides the data into more or ...
user avatar
0 votes
0 answers
6 views

Automatic Feature Selection for Regression or Classification with a mix of numerical & categorical inputs

I am new to ML. I read about some feature selection method at https://machinelearningmastery.com/feature-selection-with-real-and-categorical-data/ If my input variables consist of BOTH continuous and ...
user avatar
  • 1
0 votes
1 answer
28 views

Separating datasets vs one dataset with extra categorical feature

I have regression/classification problem. Dataset contains data from 4 sensors on 4 positions (1,2,3,4). Processes measured on all 4 positions are equivalent and same label and features describe all 4 ...
user avatar
  • 3
1 vote
1 answer
57 views

Best practice for Post-Double Selection LASSO (pdslasso)

I'd like to have a clearer idea of the optimal approach to the post-double selection LASSO (paper, webpage). Take data on an RCT with 2 treatment arm dummies $D_1, D_2$ and a potential driver of ...
user avatar
  • 15
1 vote
1 answer
30 views

How can a feature that when removed, does not affect the model's performance not be declared unimportant?

In the paper on the Boruta algorithm, there is a statement that is unclear to me (highlighted in black). The all-relevant problem of feature selection is more difficult than usual minimal-optimal one....
user avatar
  • 190
-1 votes
2 answers
44 views

Don't understand why SelectKBest with Chi Square does not involve p-value

According to SelectKBest's documentation page, it 'select features according to the k highest scores', which in this case would be the Chi Square score. https://scikit-learn.org/stable/modules/...
user avatar
0 votes
0 answers
23 views

In Variable selection method, should we include all the variables?

I've been working on a class project. While building a model, We've selected a few initial predictors (8 predictors out of 20) based on Business Knowledge. Next, we wanted to choose predictors based ...
user avatar
1 vote
1 answer
30 views

How to tell if my features improve model performance?

Setup Task: binary classification Models: logistic regression, SVM, ELM, neural networks - anything that can do classification Dataset: 10 basic features + 6 my own features Question How do I see ...
user avatar
  • 148
0 votes
0 answers
20 views

Feature selection from large time series data

I have a dataset containing thousands of economic variables (FRED data) and I am looking to algorithmic-ally extract a set of leading indicators to be used in a forecasting framework. Granger ...
user avatar
0 votes
0 answers
10 views

Can you class averaging 3D data across the third axis as feature selection / dimensionality reduction?

I am currently doing some research and as part of one of my projects, I had a dataset comprising of approx 200x200x200 data. The research is partly to do with ...
user avatar
0 votes
0 answers
9 views

Searching for combination of feature-comparisons to optimize a metric on a subset of the data?

Suppose that I have a dataset with n features X1, ..., Xn, and label Y (consider it binary for now). Features can be constructed with "meta"-features by comparisons (>,<, >=, <=),...
user avatar
0 votes
0 answers
11 views

Can a basis expansion guarantee no worse performance than original features?

Consider the typical learning problem where given inputs $x_i \in \mathbb{R}^p$ and targets $y_i \in \mathbb{R}$ for $i = 1, \dots, n$ we would like to learn some function $f$ such that $L(f(x_i), y_i)...
user avatar
  • 436
0 votes
0 answers
20 views

Sequential feature selection stopping condition

When using sequential feature selection approach, say forward feature selection, I want to stop adding new features when the improvement in model scores is smaller than a certain level. Is there a ...
user avatar
1 vote
2 answers
23 views

Methods for selecting the best variable into the regression model

I have constructed a continuous variable by using two different methods. Now I want to know the variable created under which way is the best and should be included in the model. Some preliminary ...
user avatar
0 votes
0 answers
16 views

CCA in feature selection [R]

I'm trying to conduct feature selection based on the Canonical Correlation Analysis (CCA). As far as I know, for two given datasets $X$ and $Y$ CCA looks for the linear projections of $X$ and $Y$ such ...
user avatar
  • 249
0 votes
0 answers
22 views

Understanding tf.feature_column and dimensionality input

I hope this is the correct forum. I am going over the feature_column package of the TensorFlow [1] and have checked the code that generates a DNN using the feature_column. Assume that there is a ...
user avatar
  • 101
0 votes
1 answer
41 views

Feature selection with RandomForest and then retrain RandomForest using the selected features

I am trying to classify patients into 2 different groups using a random forest. The features correspond to the gene expression of individual patients. This means, that I have around 20.000 features (...
user avatar
  • 101
0 votes
0 answers
17 views

Discarding highly correlated variables

I have got a dataset of $N$ observations with $k$ predictors $x_1, ..., x_k$ and a vector $y$ of binary responses, where $y \in \{0,1\}$. I assume that given a class my data comes from a $k-$...
user avatar
  • 249
5 votes
2 answers
907 views

Can dropping an insignificant factor from a model make the model worse?

I constructed a negative binomial model for examining the relationship of 1 count variable="carid_den" on another "juv_cneb_den" (with an offset="Area_towed"), along with ...
user avatar
  • 681
0 votes
0 answers
11 views

Can adding features hurt a Reinforcement Learning Algorithm?

I want to use a reinforcement learning algorithm to maximize the reward in a given environment. The problem is that the environment is so large that there are literally hundreds of potential data ...
user avatar
  • 1
0 votes
0 answers
11 views

Right term for "Leave-Multiple-Feature-Out" feature importance?

I am working on a ML problem incorporating four data streams, each producing multiple features. We would like to know if each of the data streams provides a significant addition to the model ...
user avatar
0 votes
0 answers
43 views

Use LASSO & bootstrapping for inference stats - no machine learning

I have a simple question: it is valid to use a LASSO model for variable selection in a small dataset? I won't do any machine learning. The goal is to use LASSO for variable selection instead of ...
user avatar
  • 1
0 votes
0 answers
18 views

Validating features that were already selected by train-test of a LASSO model

I have 50 tagged samples. I have selected features out of lots of possible features (10,000 maybe) I would like to test their ability to predict the tags. I tried to train lasso/ridge models on a ...
user avatar
2 votes
2 answers
81 views

Why use regularization instead of feature selection for logistic regression? [duplicate]

For a non-linearly separable problem, when there are enough features, we can make the data linearly separable. It seems to me that for logistic regression, the reason of overfitting is always ...
user avatar
1 vote
0 answers
40 views

XGBoost Feature Importance Changes with Random Seed

Analysis Goal: Identify features that provide an accurate prediction of a binary outcome and also explain how the features are related to the output Data: 72 features and 200 instances. Process: ...
user avatar
1 vote
0 answers
24 views

Chi squared test used for feature selection on numerical response

I leant that we can use chi-squared test for feature selection, especially this is useful on categorial features. But as we need to create the contingency table and so forth. I am not sure if I can ...
user avatar
  • 11
0 votes
0 answers
23 views

Combining frequentist and bayesian model

It has been said that combining frequentist model with a bayesian model is mathematically incorrect. Is this true? Can there be genuine cases where these two types of equations or algorithms can be ...
user avatar
1 vote
1 answer
77 views

Discriminatory model but no discriminatory features?

I am working on a binary classification problem using random forests, neural networks etc with dataset size of 977 records (class proportion of 77:23) I used ...
user avatar
  • 1,726
1 vote
1 answer
76 views

LASSO vs. Standard Variable Selection via p-value

How can I reason about compare/contrast variable selection between LASSO running a standard multi-variate regression and setting betas to zero if the p-value is > 0.05 ?
user avatar
  • 411
1 vote
1 answer
16 views

Feature with multiple attributes/properties

I want to build a machine learning model where each feature has further multiple attributes. Apologies for the lame example, but this will convey my doubt: Predict the animal on the basis of its ...
user avatar
  • 113
0 votes
0 answers
10 views

Is this a case of a confounding variable and how can I handle this?

Say I want to predict the speed of an airplane based on the engine power and the weather conditions using a simple linear model. To model the weather conditions, we have measurements of the amount of ...
user avatar
  • 145
1 vote
1 answer
55 views

Are important features or noise model agnostic?

I want to select important features of a given dataset which contains a lot of noisy features. My question is general: If I select features, by let's say Recursive feature elimination or L1 penalty ...
user avatar
1 vote
0 answers
15 views

PCA to identify patterns in the data, forced to a particular variable?

Dataset: I have a hyperspectral dataset that consists 250 wavelength bands (x1,...x250) and corresponding reflectance measurements (y) for each band. Plotting X vs Y yields a spectral profile. I have ...
user avatar
2 votes
0 answers
33 views

What is the scores calculated from the score function in SelectKBest algorithm? Do they return P value, F score, or others?

I am wondering that how the scores in Select K Best be calculated? I applied the SelectKBest with f_regression as score function. According to the documentation of SelectKBest(https://scikit-learn.org/...
user avatar
  • 21
0 votes
0 answers
15 views

Does boosting help select better features compared to bagging?

I am working on a binary classification problem using traditional algorithm and neural networks. with 977 records, my class proportion is 77:23 Currently, I am doing the below steps a) Identify the ...
user avatar
  • 1,726
0 votes
0 answers
129 views

Feature selection - Before vs After oversampling

I understand that class imbalance is not considered as a problem by some of experts here. But for the sake of experimentation, I would like to try out the oversampling for my data (which is of 77:23 ...
user avatar
  • 1,726
1 vote
0 answers
15 views

using predictors and responses from one random forest as predictors for a subsequent random forest

I am planning to do a model (stage 1) for imputing missing data in a variable that will be used as a predictor in a subsequent model (stage 2). The overall goal of this project is having the best ...
user avatar
4 votes
1 answer
135 views

Applying PCA to each group of features separately to use it in a predictive model

I have a large data-set with >100 features. Can I use PCA to reduce the dimension of each group of features instead of t applying PCA to the whole data-set? For example, if I have 10 demographic ...
user avatar
  • 63
0 votes
0 answers
23 views

Choosing the right score_func for SelectKBest

I am building a classification model with a dichotomous dependent variable. I want to use scikit-learn’s SelectKBest to select my top 10 features, but I’m not sure which score function to use. I ...
user avatar
  • 1
1 vote
0 answers
13 views

How does Sequential feature selection work

I recently watched a Youtube tutorial on Sequential feature selection and ran the below code for identifying best features in my dataset (of 977 rows, 77:23 class proportion, binary classification and ...
user avatar
  • 1,726
1 vote
1 answer
42 views

Can nested cross-validation include feature selection

The interaction between model complexity and it's ability to fit a given dataset is crucial for model selection. This thread discusses this for model selection and the same answer is suggested to ...
user avatar
  • 474
1 vote
0 answers
47 views

Why the grid scores from RFECV using ROC AUC is different from ROC AUC obtained by the same selected features?

I'm using RFECV with the scoring ROC AUC for feature selection and the model selected 3 features. However, when use these 3 features with the same estimator and same cross validation when I run the ...
user avatar
0 votes
0 answers
10 views

How to perform feature selection and nested cross validation with a high number of features?

I have a huge set of features in my dataset and I want to apply nested cross validation to estimate the generalization error and feature selection to select the best features. I'm not sure about the ...
user avatar
0 votes
0 answers
47 views

bootstrapped l2,1 least square does not produce sparse solution

I am trying to model an autoregressive model with $\ell$-2,1 regularization, where $\|X\|_{2,1}=\sum_i|\sum_jX_{ij}|$: $$y_{t+1} = wx_{t} + \lambda_1\|w\|_{2,1}, y\in \mathbb{R}^{n_1}, x\in \mathbb{R}^...
user avatar
  • 231
1 vote
0 answers
24 views

Cross validation with brute force feature selection

I've got data from 1000 people on a score, and a few outcome variables. Each score is formed of 10 components; to get the total score you sum the score for each component. What I'm trying to do is ...
user avatar
  • 11
0 votes
0 answers
18 views

Effect of apparent correlation due to clustered data on performance of binary classifier

I am exploring possible features for a binary classifier, probably using a SVM, and have encountered an issue with correlation of features. I have chosen a number of features that may be of interest ...
user avatar
1 vote
1 answer
19 views

Why sequential forward doesn't select same feature as sequential backward?

I am working on a binary classification with imbalanced dataset of 77:23 proportion. class 1 is the minority class. Currently, am exploring different feature selection techniques and that's when I ...
user avatar
  • 1,726
1 vote
1 answer
59 views

Hyperparameter tuning required for feature selection using wrapper methods?

I am working on binary classification with class proportion of 77:23 (977 records) Currently, I am exploring the feature selection approaches and came across methods like below a) Featurewiz b) ...
user avatar
  • 1,726
0 votes
0 answers
53 views

SelectKbest feature selection - non normal

I am using SelectKbest for my feature selection process. https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectKBest.html My data is non normal and actually skewed. I don't ...
user avatar
  • 299

1
2 3 4 5
45