Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

How to compare two sets of data?

I have a data set that describes the behavior of a set of devices (a1, a2, b1, b2, ..., f1, f2) using several features (features1-features14). Of these features, I want to choose a subset of features. ...
1
vote
0answers
20 views

Using on-demand features in machine learning

I have 6 input features $[m1,m2,m3,m4,m5,m6]$. I am trying to build a model that can predict the value of all 6 of these values using $[m1,m2,m3]$. However, I have the option of asking for another ...
1
vote
1answer
17 views

Does it make sense to use Gaussian Naive Bayes for a single feature?

I understand that 'Naive' Bayes refers to the approach where all the features are assumed to be independent. But I want to evaluate the performance of each feature individually before I combine all of ...
0
votes
0answers
11 views

Does Sample Size affects Mutual Information for Feature Selection?

There is a dataset with n rows (samples) and p columns (variables/features), the objective is to predict a certain target variable (y). Should n (sample size) matter to the results of pairwise mutual ...
0
votes
0answers
20 views

Can Mutual Information based feature selection be used when the input variables are numerical and the output is categorical?

I am working on a machine learning project for a classification problem. In the dataset the input variables are numerical and the output is categorical. Is it appropriate to apply the Mutual ...
0
votes
0answers
15 views

Classification Problem using multiclass features input and ensemble methods

I am working on a classification problem. I am applying tree-ensemble methods (Histogram-Based Gradient Boosting and Random Forest) and evaluating premutation importance in order to understand ...
0
votes
0answers
19 views

Quadratic programming for EM algorithm

I am currently working my way through the following paper and I am stuck. Please can somebody explain to me how on earth the quadratic programming would work for the following equation? (Eq (15)-(18) ...
0
votes
0answers
11 views

Feature selection before hyperparameter selection & cross-validation

I'm trying to train a model to predict water solubility and the dataset has 200 features, with just a few of them being informative and interpretable. My plan is to validate the estimator using 5-fold ...
0
votes
1answer
33 views

Can ML or DL model automatically pick up “difference” feature?

ML requires manual feature extraction whereas DL doesn't necessarily require feature engineering, since recent advanced models like transformers learn necessary features automatically during training -...
1
vote
0answers
29 views

choosing most important predictors for logistic regression

I have a dataset of cars with price label as binary outcome including "affordable" and "costly". I aim to ...
0
votes
1answer
33 views

Naive-elastic net and elastic net variable selection comparison

The elastic net paper (here) introduced the naive-elastic net and elastic net. The coefficient estimates of naive-elastic net are obtained by solving the problem $$\hat\beta_{naive-enet}=\text{argmin}...
1
vote
1answer
17 views

Impact of feature selection

I am trying to understand the impact of feature selection on neural networks. If I have n features and n' number of features are redundant, and I eliminate those n'. Do the remaining number of ...
0
votes
0answers
8 views

Ways to handle features that are not applicable for all records [duplicate]

I have a data set that contains records of distinct groups identified by a 'type' variable. Depending on value of this 'type' variable certain other variables are either applicable or not. Effectively ...
3
votes
0answers
17 views

Should an ordinal variable in an interaction be treated as categorical or continuous?

I have an ordinal categorical variable (A, with 3 categories). There are 2 ways to include it in a regression model: 1) as a factor or as 2) a continuous variable. ...
0
votes
0answers
25 views

Does my model add any value?

I have 4 features in my dataset. let's say one feature is currently used by doctors in hospital (along with their domain expertise) to make informed decisions. So if I build a binary classification ...
1
vote
1answer
31 views

Does comparing Shannon Entropy between categorical variables makes any sense?

I want to present a report that highlights the variables with lowest information, to emphasize that some action must be taken by the department that controls our data source. I've applied Shannon ...
0
votes
0answers
16 views

How to encode high dimensional dynamic categorical features

Example: Facebook newsfeed ranking. This is typically done by relevance scoring each post using a regression model based on a number of features. Typical features might include information from the ...
0
votes
0answers
16 views

Separate Hyperparameter/feature selection and Model Selection cv vs nested cross validation (cv)

I realise that nested cross validation can be used to reduce bias when hyper-parameters tuning is combined with model selection. However, I wonder if it is possible to perform hyper-parameter tuning ...
1
vote
1answer
339 views

Which covariates to use in order to answer a specific research question in longitudinal model?

I have a longitudinal dataset of 2 time points where Body mass index of patients has been recorded in 2018 and 2020. the research question is to investigate the evolution over time ( from 2018 to 2020)...
0
votes
0answers
12 views

How can I recognizing the reduced features after applying PCA?

I am working on the "global index of economic freedom" data which has 26 exploratory variables. I am trying to reduce the features using PCA and I am able to recognize that the first 5 PCs ...
0
votes
0answers
15 views

Can Transformer neural networks be used for botnet attack classification?

I would like to ask for some advice/guidance regarding a deep learning project we're working on, we're trying to do Feature analysis of IoT botnet attacks using Deep Learning we're working with the Nb-...
2
votes
1answer
85 views

What is the criteria for including and excluding variables in longitudinal models?

I am dealing with longitudinal data with two times points 2018 and 2020, I am modeling two variables BMI (continuous) and anxiety disorder ( binary), my main interest is to investigate the evolution ...
0
votes
1answer
19 views

What does the prefit parameter in sklearn's SelectFromModel do?

I'd like to use sklearn's SelectFromModel to do feature selection. However, I'm not quite sure I understand the difference between prefit=True and ...
0
votes
0answers
7 views

How to get features importance for different classifiers?

I am currently using different classifiers (Naive Bayes, Random Forest, SVM, Logistic Regression) and for some of them (e.g., MultiNaive Bayes) I cannot run some built-in function for feature ...
0
votes
0answers
30 views

Should I use Mann Whitney U or Wilcoxon Rank for Feature Selection Effect?

I implemented an application to see how 9 different algorithms are affected by the feature selection in a dataset. As a result, I have accuracy values obtained before and after feature selection for ...
0
votes
1answer
13 views

Removing a low predictive column before or after train/test split

Based on what I found from the other posts I should always first split the data into train/test set and then perform feature selection to prevent information leakage. Here's the part that I don't ...
1
vote
0answers
21 views

Feature selection based on correlation test between quantitative discrete and dichotomous variable

I want to perform a binary classification on CLAIM_FLAG where one of my explanatory variable is YOJ (Year of Joining) and the range is from 0 to 23. I plotted out the relation between these two and it ...
0
votes
0answers
23 views

Tweedie distributions for large dataset, power parameter for different density peaks and random effects

I have a large dataset (343750 variables and 151 observations) and I want to model each variable as a response one in order to know if it can be explained by the group of patients and the gene ...
0
votes
0answers
6 views

Using other predictions on same subject in data set as a feature to improve accuracy

I have a problem I'm working on which is similar in structure to this example. Say I'm predicting the score a student received on an exam based on a collection of their social media activity and the ...
0
votes
0answers
7 views

Units of measurement for different feature selection techniques

I am trying to use multiple methods to select the best features, more especifically, ANOVA, Mutual information selection, Chi-2 and the internal feature selection method of Random Forest. I came ...
0
votes
1answer
26 views

Why is AdaBoost with short decision trees a form of feature selection?

Why is AdaBoost with short decision trees a form of feature selection? What is so special with short decision trees?
0
votes
0answers
68 views

RFECV (Recursive Feature Elimination with Cross Validation) grid scores discrepancies

I would like to know why the grid scores obtained by RFECV (Recursive Feature Elimination with Cross Validation) for nth features do not match the scores when I run RFE and train a model with same ...
1
vote
1answer
46 views

Does it make sense to engineer new variables and keep the original variables?

While doing some feature engineering on a dataset, I recently thought: When I create new features, should I keep the original ones? Let me specify my questions a little bit more and give you an ...
0
votes
0answers
11 views

Feature Selection based on single-feature classifier score - sound feature selection method?

To gain reliable estimates of model performance, feature selection typically should be performed in a nested CV pipeline. However, in a current project with a small amount of data, the nesting I tried ...
0
votes
1answer
16 views

Should we use the same fitting method in model selection and prediction?

I am curious whether we have to use the same fitting method in model selection and prediction. For example, suppose that we are going to use the logistic regression in prediction. Then, we may select ...
0
votes
0answers
19 views

Choosing feature selection method

I have a dataset that contains input features of type numerical as well as in the form of Boolean (0 and 1). My ouput is either 0 or 1 (classification). I want to ask what feature selection techniques ...
0
votes
0answers
11 views

Selecting correlated variables for time-series analysis

I am using time-series analysis to observe the spending patterns of individuals and to detect any possible anomalies, which could be fraudulent transactions. For the time-series model, I am ...
0
votes
0answers
7 views

Chosing data for cluster analysis in a patient database

I am planning to perform cluster analysis on a patient database with several different variables such as prescriptions, conditions, labs, etc. Reading similar works I conclude that not all authors ...
1
vote
1answer
42 views

How to combine stability selection and model selection with LASSO?

I was reading about stability selection applied to LASSO. My understanding is that stability selection (Meinshausen & Buhlman, 2010) helps in finding stable variables, with error control provided ...
1
vote
0answers
88 views

time series forecast by extracting descriptive stats features

I am relatively new to time series data and I wanted to know whether forecasting a value into y(t+1) can be achieved by training a model using descriptive statistics (mean, standard deviation,max-min ...
0
votes
0answers
14 views

Feature Selection with Potentially “Useless” Features

Suppose in a regression setting, we have many features and potentially many of them have nothing to do with the target. We use Lasso and various regularization parameters and perform cross validation ...
0
votes
1answer
25 views

How to select a set of independent features predictive of all other features without a target variable

Problem I have a dataset NxD, where N - number of observation (~100k) and D - number of features (~10k) (More specifically it is a single-cell RNAseq data, so each observation is a single cell and ...
0
votes
0answers
26 views

How to select variables in R from NHANES dataset to fit a logistic regression?

I need to fit a logistic regression of the NHANES dataset available in R. It has 76 variables in it but I have been able to filter it to 40 variables by removing NA columns. I still need to select the ...
0
votes
0answers
63 views

Lasso cross-validation plot looks weird

My dataset is about the Airbnb room rental price, after encoding dummy variables, data cleaning etc., left 30 features, 7770 rows. Price is the response. The link to download the dataset is here, http:...
1
vote
0answers
28 views

How to determine if 2 specific variables should be in the regression at the same time [closed]

I am trying to run a regression and I have two variables; income and gender. How can I determine if these 2 regressors should be in the model at the same time? Is there some sort of a test?
0
votes
1answer
24 views

Classifier performance decreased after feature selection

Im working on stock price direction prediction project and have tried out some models (SVMs and Random Forests). I used Ransom Forests for feature selection and it actually decreased the performance ...
1
vote
0answers
24 views

Feature selection with PCA and CA?

I am studying some factorial methods, namely, PCA and Correspondence Analysis and I have a few questions for you. It is clear that the principal axes in PCA are linear combinations of the original ...
1
vote
0answers
11 views

How to find best features using statistical tests?

I am new to Machine Learning and the first part where I am stuck is identifying which features are best for the model. I understand that usually correlation heat map shows the input and output ...
1
vote
1answer
40 views

Are the important factors given by gradient boost more arbitrary compared to random forest

I compared the output from the two approaches using rfsrc() and gbm() in R respectively. The important factors given in the output from the two approaches are totally different. Since the importance ...
0
votes
1answer
44 views

LASSO/Ridge regression with adjustment for a covariate

I'm trying to address the following analysis problem in high-dimensional biological data. The setup is bulk gene expression data where multiple cell types (tumor and immune cells) can contribute to ...

1
2 3 4 5
43