Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

How to develop intuition to reason about models/predictions

I am trying to answer a few questions that I was asked about data science and was wondering what the best way is (from your experience) to develop the qualitative/quantitative intuition behind ...
0
votes
0answers
15 views

How to balance transformation decisions, feature selection, and model tuning vs time in text analytics?

Being to new text analytics, I haven't gotten the hang of my typical ML workflow given how long processes take to run in the commonly large feature space of text analytics. I would like to know what ...
0
votes
0answers
17 views

When to perform feature selection?

I'm making my undergraduate thesis that proposed K-Nearest Neighbor and Chi-Square feature selection to do sentiment analysis. I also using TF-IDF as term weighting. My question: is feature selection ...
1
vote
0answers
23 views

Finding the “most representative” variables of a population

Most polls use the so-called quotas method to obtain their sample. They draw candidates upon a large panel of people and add one-by-one each responding candidate to the sample unless a certain quota ...
0
votes
0answers
8 views

Are there defined categories of features in images? [closed]

I have been looking into deep anomaly detection and I am currently wondering about what kind of features can be extracted from an image. I have seen papers about edge features and texture features, ...
-1
votes
0answers
10 views

How to use PCA for feature extraction?

So what I have to do is given n images for training and m images for testing, I have to build a model for classification. To do that I need to extract features from the images. Now the person under ...
1
vote
1answer
23 views

Do typical NNs generate new features by applying some function to the input?

The toy network on playground.tensorflow.org has the option to generate new features by applying some function based on the input, e.g. with the inputs $X_1, X_2$ ...
1
vote
0answers
20 views

Computation efficient time series causal graph representation?

I am trying to select the best features to train a Deep Learning model from a big database, consisting of stock/cryptocurrencies prices (and possibly technical indicators). I have found this algorithm,...
0
votes
0answers
13 views

Random Forest_feature selection

I am working with Random Forest classifier and after performing a feature selection, I want to keep let's say top 15 features out of 50 features according to the Gini index. So after learning with all ...
0
votes
1answer
24 views

Joint optimization - Feature extraction and a classifier

I am dealing with a classification problem and high dimensional data. I am using a feature extraction method ( PCA - Principle Component Analysis) followed by a Support Vector Machine (SVM). I just ...
2
votes
0answers
19 views

Variable criterion for inclusion in principal component analysis

I have a set of 9 variables that I consider for inclusion in Principal Component Analysis. I've read that in order to decide whether a variable enters PCA, it needs to have at least one correlation ...
2
votes
1answer
33 views

Feature selection: nested cross validation

I'd like to select features, and evaluate model performance using nested cross validation. My question is that I have to split data in order to select features or not. Additionally, is the following ...
1
vote
1answer
16 views

RFE on small dataset: What kind of cross validation should I use?

I’m using recursive feature elimination (RFE) to rank features. My dataset contains 50 observations and 20 predictors. Is there a specific type of cross-validation method I should be using to estimate ...
0
votes
2answers
24 views

Feature importance changes drastically when adding other features

I have a model (GBDT) where adding a feature X is not important (according to SHAP), but when I add other features, and add X again, now feature X is the second most important! What could explain that?...
0
votes
0answers
18 views

How to properly measure accuracy with feature selection?

I applied a feature selector (with this great python package) in my dataset. This package uses the wrapper approach, where you define a classification model that runs on your data and find the best $k$...
2
votes
1answer
31 views

Feature extraction with TSFEL giving inconsistent sizes

Looks like this is TSFEL's first appearance on Stack Overflow. Time Series Feature Extraction Library (TSFEL for short) is a Python package for feature extraction on time series data. Here goes. I'm ...
0
votes
1answer
15 views

RFE, feature selection for Churn/Credit Risk modelling

I am currently working on a redemption model for a financial company, using time series data and Logistic Regression. Currently we have a few features that are time dependant (I know, logistic is not ...
0
votes
0answers
5 views

Model optimization: feature slection in artificial neural network

Since the neural network algorithms work as a black box, how can I select the features that are irrelevant to oppress them in order to reduce overfitting? I don't know if that changes anything, but my ...
0
votes
1answer
19 views

If a variable has a weak correlation can you take the inverse of that variable to mean strong correlation?

I'm doing a feature selection to determine what variable correlate most with NFL teams covering the spread. If I have a variable that represents the number of people who have bet on one side of the ...
1
vote
0answers
34 views

ELI5 Why do we drop highly correlated features?

Let's say we have a classification task with two features, such that classes "can't" be separated because the Pearson correlation equals 1. If we train decision tree on that, will both of ...
0
votes
0answers
21 views

Interpreting multiple logistic regression result - which features are most important?

I'm doing a multiple logistic regression analysis on my dataset to explain what predictors are most important for predicting outcomes. I think I'm confused with interpreting the results independently ...
4
votes
2answers
55 views

In feature selection, what is the reason for considering removing low variance features?

I've overheard a few times that when doing feature selection, one should look at features with low variance and consider removing them. (My guess is that if we have a dataset of 100 observations and ...
0
votes
0answers
11 views

Stepwise feature selection for a classifier: use accuracy or loss? [duplicate]

When using stepwise feature selection for a classifier, is it typical to measure progress by using the loss (e.g., cross-entropy loss) or the accuracy? Is there any basis for choosing one or the ...
0
votes
0answers
11 views

Ranking importance of input parameters in NN

While it is rather straightforward to rank the importance of input parameters in Decision-Tree-like models... are there any efficient technics to do that with regard to neural networks that are ...
0
votes
0answers
11 views

Loss low but extremely low feature importance score

I'm running permutation importance on DNN and for some reason the numbers seem suspiciously low, highest scores are around 0.015 with explained variability score and pretty much the same with r2 ...
0
votes
0answers
8 views

Is it possible to extract the coefficients from a model built by VSRUF?

I'm using the VSURF package in R to perform a random forests analysis. However I would like to actually see the coefficients of the selected variables (at least their signs) and can't figure out how ...
0
votes
0answers
6 views

how to do the feature_selection in testing data

I have a simple RF classifier model trained with a sample dataset and it works fine. So, I use some test data to predict the target class and let's say it find the target class as ...
0
votes
0answers
28 views

Box-Cox formation with model selection, regularization, etc

As my data is not normally distributed, I performed the Box-Cox Transformation on the response. ...
2
votes
1answer
51 views

Forward and backward stepwise regression (AIC) for negative binomial regression (with real data)

I am doing some count data analysis. The data is in this link. Column A is the count data, and other columns are the independent variables. At first I used Poisson regression to analyze it: ...
1
vote
1answer
14 views

why max value of n_component depends upon n_classes in LDA()

According to sklearn LDA() documentation, max n_components value depend upon the classes the data have. but I am unable to understand why it can't be more than no. of classes when features are more. ...
4
votes
2answers
38 views

What is the ideal approach to determine relationship between candidate predictors and a dependent variable in a data driven way?

I have asked several related questions (1, 2, 3), but now I would like to ask the most basic questions and hope to get a very solid answer. I have 40 treatment variables, and I am interested to find ...
0
votes
0answers
10 views

How to do data mining that consider all possible variables specification?

First of all, I know the drawback associated with datamining in modelling, but this case is very specific, and my model don't need any replication. I just need to overfit the results of my database. ...
1
vote
1answer
24 views

What do eigenvectors of a data matrix consisting of house features/prices tell us? [duplicate]

I know this is one of the most repetitive question but bear with me please. I am trying to gain an intuitive understanding of eigenvectors. I had this example in my mind where there is a matrix A, the ...
0
votes
0answers
10 views

PLS regression - VIP treshold to exclude variables

I have been developing PLS models in the software SIMCA. To optimize the model and decide which variables to exclude, I use the VIP (Variable Importance in Projection [1,2]) and in the software ...
1
vote
1answer
52 views

What analysis should I run if I have 40 predictors and I want to know which are related to the dependent variable?

I have forty candidate predictors. They are no colinear. I want to know which ones are related to the DV. Prediction isn't important to me. I want to do this in an exploratory and data-driven way. ...
0
votes
0answers
9 views

Performing a regression (post-hoc) on PCA output

I have one dependent variable (y) and 7 independent variables (x1,x2,x3,x4,x5,x6,x7). I'm performing a PCA on the dataset. Once I get the Principle components, I wanted to be able to interpret the ...
0
votes
0answers
4 views

Alternative of Sequential Feed Forward or Backward Feature Selection

Is there some alternative of Sequential or Backward feed forward selection according to following scenario? I have a Matrix of features and subjects where columns corresponds to features. Vector of ...
1
vote
1answer
17 views

Is there a benefit to splitting the data by gender or age range when building predictive models?

Assume we had a set of data that contained thousands of samples with the following information: gender, age, height, weight, country. Now, suppose we wanted to build a model for predicting people's ...
0
votes
1answer
30 views

Decision Tree uses too few Variables

The Decision Tree algorith used only 3 variables ("lstat" "rm" "dis") to make a regression. The Dataset is Boston Housing which has 13 predictor variables and 1 ...
0
votes
0answers
8 views

two ways of features selection

There are two ways in the features selection: Remove the features with low variance; Remove the features with the small coefficient in regression; However we ...
0
votes
0answers
16 views

Feature importance across time

I performed an experiment that generated 40 input variables (or features) and one output variable for 150 samples (150 trees). I replicated the experiments 5 times across the growing season for all ...
0
votes
1answer
25 views

Can I use decision tree to create/combine attributes into a new attribute?

I might have a silly question - I am building a linear model with many many attributes. I have narrow down to a few - I do have a group of 3 attributes that are highly correlated (for example sales ...
0
votes
1answer
24 views

Not sure if Cox or Gaussian glmnet regression is appropriate

I am working with data from an outpatient physical rehab clinic and am trying to figure out what variables can predict the number of hours of service provided (DV). We have two years of data with a ...
2
votes
1answer
126 views

When in Adaptive LASSO process does it make sense to constrain control variable lambdas to 0?

Lets take this example for how to conduct an adaptive LASSO. Essentially an initial model is fit using ridge regression. Then, a LASSO is fit, in which the value of lambda is tuned individually for ...
0
votes
2answers
30 views

Feature importance understanding

I would like to ask if I understand correctly the feature importance in random forest. I am examine random forest by selecting 4 or 6 features and also with different number of trees. I would like to ...
0
votes
1answer
32 views

Improve XGBoost performance in a huge dataset with a lot of missing values

I have a dataset with around 250 features and 4 Millions samples and we obtained a model with Xgboost that has acceptable performance. The dataset has a high percentage of missing value, for an ...
4
votes
3answers
113 views

If there are 40 candidate predictors, and I want to know which ones predict the dependent variable and in what way, is LASSO a good option?

I have about 40 candidate dichotomous predictors. I want to know which ones predict a DV, and in what way. Is an adaptive LASSO regression a good way to do this? If not, could you explain why not, and ...
0
votes
2answers
46 views

Feature Selection in Multivariate Linear Regression

I have $m$ time series and I partition the time series into two sets: set X and set Y. I want to predict set Y using set X. The time series of set Y are ${\bf y}_t$ and those of the set X are ${\bf x}...
0
votes
0answers
19 views

Autoencoder Incorrect Output/Predict - Model Built and Trained. *Please Assist*

Background Let me preface, I am new to python and machine learning. I have been tasked with creating an autoencoder to reduce dimensionality on a made-up dataset (proof of concept). I am working in ...
1
vote
2answers
29 views

L1 feature selection followed by exhaustive search

I'm working with a group on an ML project and one of the team members has proposed using L1 to reduce the feature space and then apply an exhaustive search with the reduced feature set. In each step, ...

1
2 3 4 5
40