Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

0
votes
1answer
13 views

Feature selection by lasso in two samples compared to one joint sample

Let's say you have two sets of features $X_1$ and $X_2$ together with a response variable $Y$. I wonder whether the two following procedures are identical asymptotically (or in finite samples) in ...
0
votes
0answers
27 views

Combining independent variables in linear regression - does it make sense?

I try to model energy consumption for a set of about 50 relatively similar production facilities. I have annual data of energy consumption and 3 independent variables that - from a technical point of ...
0
votes
1answer
18 views

Discard features with small variance, how to do in practice? [duplicate]

I'm training a neural network for regression. The input vectors consist of $92$ different features, I want to discard features with small variance (standard deviation). There are two ways that came ...
0
votes
0answers
22 views

How to extract static program features automatically?

I did want to know how to extract statistical features from program. Like supposing I wanna do an extractor for loops programs so features in this case could be The loop nest level. Is the loop ...
0
votes
1answer
16 views

Recursive Feature Elimination returns suspect results regarding accuracy

I am doing RFE with 18 features. My understanding of RFE is, it starts by trying N - 1 models, and then removing the feature that causes the N - 1 models to perform worst. Then it tries N - 2 models, ...
0
votes
0answers
7 views

Do I need to treat class imbalance before preforming RFE (Recursive Feature Elimination)? [duplicate]

I am trying to perform RFE on an imbalanced data set that will later be used for binary classification. I have chosen to use the caret::rfe package for feature selection. I am using random forests ...
1
vote
1answer
42 views

Should I discard a feature whose max - min is smaller than 1e-5?

I'm training a neural network for regression. The input vector consists of $140$ entries. For each feature vector entry, I calculate both min and ...
0
votes
0answers
45 views

Random Forest Model gives accuracy of 99.9% and AUC of 1 [duplicate]

I am trying to build a random forest model and have been getting a test accuracy of 0.998 and and AUC score of perfect 1. Now I know from intuition that this should not be happening but I am not able ...
0
votes
0answers
7 views

What is the best input for denoise autoencoder for sound/audio data?

I am currently trying to build an autoencoder to de-noise audio data. However I have not found any good articles explaining about the input to the autoencoder, i.e. feature vector. As in speech ...
0
votes
0answers
11 views

Should Feature Selection be Done Before or After Feature Scaling?

I read feature selection should be done before scaling but did not find the source to be too reputable. Should Feature Selection be Done Before or After Feature Scaling? Why?
0
votes
1answer
13 views

Adding Bad Features to Decrease Model Performance

I have a dataset on which some researchers have already performed some definitive data analysis and feature selection. Fitting a model to this dataset returns pretty good accuracy. In order to ...
0
votes
0answers
4 views

Univariate Feature Selection KBest Test Score Function with Binary Target

What is/are good score functions for univariate feature selection tests when the target variable is binary? Are any of the available scoring functions bad, aside for the regression functions of course....
0
votes
0answers
7 views

Predicting Daily Binary Variable with Mostly Monthly Variables

I have a binary classification task with a dataset composed of a time series of independent events. As part of my data augmentation efforts, I have found a lot of economic indicators that however ...
1
vote
1answer
27 views

feature importance using forward selection

In the following article the author has correctly mentioned that the "petal" is more important than "sepal" in case of iris data: https://towardsdatascience.com/feature-importance-and-forward-...
0
votes
1answer
16 views

Automated variable selection with unique variables

I have a dataset which contains areas covered by different landuse variables such as agriculture, forest, grassland etc for different spatial scales. The spatial scales that I have used are 30 m ...
0
votes
1answer
26 views

Is it normal for the intercept to have a 1600 Variance Inflation Factor (VIF)?

I'm using Python's module to calculate the VIF for my variables to be used in a binary logistic regression. I'm completely following this post to do this: https://etav.github.io/python/...
1
vote
1answer
101 views

Can t-SNE help feature selection?

I'm training a fully connected feed forward neural network for regression. Given one training example $(x_i, y_i)$, I need to convert the raw representation $x_i$ into an invariant representation $...
0
votes
1answer
22 views

Application of LASSO , Ridge, PLS in feature selection of spectral data

The meatspec data in faraway package is spectral data with 100 features .(215 *101). Use of LASSO over ridge and PLS gives better performance (RMSE based) But none of the features are removed ( no ...
0
votes
1answer
53 views

Random forest permutation test: Is permutation of the training set appropriate?

I have a rather high-dimensional data set (p > 1000) with several variables ranking significantly higher than the rest in terms of variable importance (measured by Gini impurity). However, these ...
0
votes
0answers
10 views

Does it make sense to apply recursive feature elimination on one-hot encoded features?

Does it make sense to apply recursive feature elimination on a feature set pre-processed with One-Hot Encoding? This is my code for feature selection: ...
0
votes
0answers
9 views

How to correctly plot the outputs of a recursive feature elimination algorithm?

I am a bit confused with understanding the parameter step of RFE and RFECV algorithms. This is how I run RFECV for multi-class classification problem (3 classes): <...
0
votes
0answers
8 views

all-relevant feature selection vs minimum optimum feature selection

There are many different ways to selection features in modeling process. One way is to first select all-relevant features (like Boruta algorithm). And then develop model upon those those selected ...
1
vote
0answers
26 views

Reducing Bias from a Random Forest - Feature Importance

I'm currently looking to show which of three variables is more important in classifying something as True or False. Everyone agrees that all three variables are important, but not all agreeing on what ...
0
votes
0answers
22 views

Will Boruta Analysis work on unbalanced dataset? [closed]

I have over 50 variables and would liked to use Boruta Algorithm to perform feature selection based on my target variable (which is binary). My dataset is unablanced. My results with the first run ...
1
vote
1answer
21 views

Can I view the features of my clusters without doing it by hand?

I have performed hierarchical clustering on a data set with 186 participants and 94 variables for each participant. What I want to know is if there is a way to see which features are "driving" my ...
1
vote
1answer
29 views

Choosing model for feature selection on categorial data

I have a dataset composed 30 features and 1 response. My response is 0 or 1 and, all of my features composed three status includes = -1,0,1. I wanted to do features selections in R, firstly I want to ...
0
votes
0answers
14 views

Feature reduction of Biological time series signals

I have a data set of biological signals (PSG signals); the dimension of the signals is high (850 features for each sample). I am looking for the best way to reduce the dimensionality of the signals. ...
0
votes
1answer
38 views

Is there an efficient approach in machine learning when I have the confidence (uncertainty) values for the input features?

Could you give me some comments? I'm looking for a better approach when I have confidence (uncertainty) values for each input feature. For example, let's say each class has 3 features. ...
0
votes
0answers
17 views

Feature selection using information gain numeric features

I am trying to perform feature selection using the information gain criteria i.e. with the information.gain function in the FSelector R package and I am at a loss to what to do with my features that ...
1
vote
1answer
34 views

Can I use Lasso directly in classification for feature selection?

In the scikit-learn package, Lasso is a linear regression model while it can be used for feature selection. However, is it reasonable if I use it directly in ...
0
votes
2answers
29 views

Use PCA to discover the most impactful variables on the original data set?

I am trying to find a way to statistically show that some variables in my data set are more important than others to determine its classification. I have an example data set with three variables from ...
0
votes
0answers
24 views

RFECV + GRIDSEARCHCV on entire dataset (?)

I have 30,000 samples with 150 features for a binary classification problem, now I plan to follow: https://stackoverflow.com/questions/23815938/recursive-feature-elimination-and-grid-search-using-...
2
votes
1answer
61 views

Feature Engineering: Should I drop features that can be calculated using other features?

In feature engineering, should I drop all features that can be calculated using other features? For example, let us say that we have this dataset: ...
1
vote
0answers
22 views

What is the most common KIC? How does it work?

Information Criterions are methods of assessing model fit penalized for the number of estimated parameters. Another question on the site asked for a comparison between the KIC and two other common ...
0
votes
0answers
28 views

Best feature selection model for large number of predictors?

I've been working with text data (26000 different tokens over 38000 rows) and I want to extract around 100 tokens to use in my final model. My first approach was to fit a Lasso model, using all my ...
0
votes
1answer
28 views

SVM and correlation

Can anyone guide me about the feature selection based on correlation using SVM? RBF kernel check the correlation too or not? I am using weka and matlab. Any help would be appreciated.
0
votes
0answers
14 views

Incorporating prior knowledge into feature selection in the setting of multicollinearity?

Background: I'm trying to find the optimal combination of two parameters for finding the first peak meeting some criteria in a signal. The filtering is a bit simplistic, there's a threshold (...
1
vote
1answer
19 views

For a specific dataset do all the features have the same importance across different algorithms?

I wonder if by implementing a feature selection technic using training with a specific algorithm you can select the feature you need to use with other algorithms also. To be more specific after I ...
0
votes
0answers
18 views

Can we use non-Invertibility property of a matrix to detect linearly dependent features?

In order to find whether two features (such as the size of a house in feet^2 and metert^2 ) are linearly dependent or not? One way of finding it is! you take the transpose of the feature vector and ...
0
votes
1answer
31 views

how to use elastic net to select a set of features

I have a dataset with 500 samples and 100 features. I need to come up with a set of features. The management prefer a model with a smaller set of features. How exactly should I use elastic net to do ...
4
votes
3answers
95 views

Almost reverse feature importances by Extratrees vs RandomForest

I am using scikit-learn to find feature importances using ExtraTreesClassifier and RandomForestClassifier, both of which have feature_importances_ attribute. The data has 4 numeric predictors, 2 ...
0
votes
0answers
13 views

Why would the mean and standard deviation of the first and second derivatives of a signal (e.g. EDA) be useful?

When analysing a signal, e.g. EDA, I intuitively understand why one would want to determine the mean and standard deviation of the signal. The mean would tell us the average value of the EDA signal, ...
0
votes
0answers
17 views

How do deep CNNs handle problem of “perceptual aliasing” while visual place classification?

Deep CNNs have achieved state-of-the-art performance in image classification. The underlying concept of place recognition is similar to image classification where the task is to classify or recognize ...
0
votes
0answers
26 views

Trend of 1 feature in a regression model

Let's say I have a regression model that is trying predict sales of a product in 1 year. I have about 10,000 data points with features like: sales in first 4 months product type ... The first ...
0
votes
1answer
38 views

Before running a ridge regression model, do I need to preform variable selection?

I am currently constructing a model that uses last year's departmental information to predict employee churn for the current year. I have 55 features and 318 departments in my data set. A good ...
1
vote
1answer
41 views

Using SVM with only one feature

I am doing a parametric study on the performance of SVM w.r.t various input feature sets. In one case, I analyze the performance of SVM using a only one feature at a time (similar to One-at-a-time ...
1
vote
0answers
11 views

When is feature selection sensible?

I'm using multiple linear regression to identify what affects discounts, customers receive. Therefore, I interviewed salesmen and collected data based on their assumptions (e.g. weather data, pricees ...
0
votes
1answer
35 views

Can exogenous variables in ARIMAX be time series variables?

I am trying to do a multivariate forecasting with ARIMAX. And I am not sure if ARIMAX can handle time series features as exogenous variables or it can only handle none time series features or both.
0
votes
0answers
18 views

GBDT- randomized repetition feature selection

Consider the following approach for feature selection in the specific case of gradient boosting decision trees: Randomly pick X% of features Run algorithm Record importance of each feature Repeat ...
0
votes
0answers
15 views

Variable selection with nested data

I am interested in creating a risk score for a binary event. We have large pool of patient-level candidate predictors -- about 30 plus some expected interactions. The data set is large in terms of ...