A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0
votes
0answers
22 views

How is domain knowledge developed?

Most sources I've read state that domain knowledge is crucial for making good inferences. As an example, if I'm conducting a study to assess the importance of a new biomarker for heart attacks, the ...
0
votes
0answers
18 views

Feature selection to calculate AUC using the LOOCV approach

I came across this paper: https://www.ncbi.nlm.nih.gov/pubmed/29355115 where the authors use random forests for feature selection in the following way: "..we performed the RFE procedure 100 times ...
0
votes
0answers
7 views

Has anybody tried MIRL package? [closed]

I am thinking of using the method mentioned in the following paper. I have installed MIRL package, but ran into issues determining the threshold with the error message "Error in x[rand != i, ] : (...
1
vote
1answer
28 views

How should I perform feature selection?

I am trying to know the right way of doing feature selection. I am kind of mixed up with feature selection and cross-validation. I am not even sure if I should perform feature selection inside cross-...
1
vote
0answers
20 views

Different seed gives different LASSO output

So I am using LASSO in order to reduce the amount of independent variables (~150) I have for my logistic model (n=1200). However, when doing so, the end result (i.e. number of predictors) it chooses ...
0
votes
0answers
12 views

The most basic question about Feature Important or Permutation_Importance

Consider the XOR gate with three inputs. The truth table will be: Now all the variables on their own are near random as far as the model is concerned. Each input 1 or 0 has a 50% chance of being ...
13
votes
3answers
416 views

Variable selection in logistic regression model

I'm working on a logistic regression model, I have 417 independent variables to try, but running a model with all of them is too much (for Rstudio), what would be a good criteria to discard some ...
0
votes
0answers
14 views

Reduction in number of observation by extracting piecemeal signal features,while keeping the no of features same. Can it be called feature extraction?

I have a dataset generated from 9 sensors in an E-nose system for a binary class classification problem. The system provides a response for 240 seconds for each sample. i.e. I have a data set of 240 * ...
5
votes
2answers
99 views

Does too many variables in a regression model affect inference?

Regression models can be used for inference on the coefficients to describe predictor relationships or for prediction about an outcome. I'm aware of the bias-variance tradeoff and know that including ...
7
votes
3answers
116 views

How to test for the best parameters for transformed independent variable in linear model

Let's assume that I have a linear model with $k$ variables: $y = \beta_0 + \beta_1\cdot x_1 + \dots + \beta_k \cdot x_k$. Now, I want to add variable $x_{k+1}$, but, according to domain knowledge, ...
1
vote
0answers
25 views

Assumptions for Univariate Analysis

I am working a binary classification problem. To identify the risk factors, I would first like to do univariate analysis and then do ...
0
votes
0answers
11 views

Hypothesis tests via permutation importance on validation data

A model agnostic way to measure variable importance of a variable $x$ in a statistical model (originately proposed by Breiman) is as follows: Select a scoring function $\mu$ to measure model ...
0
votes
0answers
10 views

Identify time series contributing to another time series

I have a time-series of returns on an unknown portfolio. I do not know the assets within this portfolio or their weights. I now have around 30 time series with asset prices. I need to identify a ...
0
votes
0answers
23 views

Best Feature selection algorithm Boruta, Step, Information Values(WoE) or RFE

I have landing data with 103 columns Would like to understand which algorithm tis best for feature selection and what may be the logic to call any feature as best. I have landing data with 103 ...
0
votes
1answer
25 views

Variable selection for Cox regression repeated for multiple covariates of interest

I am doing a retrospective analysis of the effect of various measures of haemodynamics in sepsis on mortality. I will separately look at the effect of 5 independent variables: 1) shock index 2) blood ...
3
votes
3answers
192 views

May using more features decrease the accuracy of a classifier?

I'm testing a project. I have train and test data. There are 182 features and 1000 samples for train and 3500 for test. If I select certain columns of data and apply naive Bayes classifier to them, I ...
0
votes
0answers
9 views

Using scagnostics for feature selection

I want to ask, is it make sense to use Scagnostics for feature selection? I plan to use this method to subset feature.
3
votes
1answer
18 views

In linear regression, what is the difference between performing variable selection before assessing multicollinearity or vice versa?

If you have a number of variables you're interested in and want to perform linear regression, is there a clear preference between: Method A. Perform variable selection techniques (e.g. using ...
0
votes
0answers
11 views

Feature importance with Monte-Carlo iterations - mlr

I need some assistance with the statistical interpretation of the output of the function generateFeatureImportanceData() from mlr...
0
votes
0answers
19 views

Feature selection using Fisher score

I am trying to perform Image Quality Assessment on Retinal images. I have extracted certain features using my algorithms. Now I want to perform 'feature selection' and classify the image using ...
0
votes
1answer
23 views

are Dimensionality reduction techniques performed before or after feature selection?

What would be a sensible approach to get the best features to improve accuracy? DImensionality Reduction -> Feature Selection -> Reduction -> Selection ...... Selection-> Reduction Reduction -> ...
1
vote
0answers
18 views

Is it good idea to generate features from data points similarity comparison?

I know about polynomial features in machine learning, which can introduce nonlinearity to original dataset. I also heard about binning, which also allows us to create new features from existing ones. ...
1
vote
1answer
32 views

False positive/negative rate in ridge and lasso regressions

I have a confusion matrix of true and estimated $\boldsymbol{\beta}$ vectors of lasso and ridge models from a replicate of a simulation study, say. The following tables illustrate the scenario. $$\...
1
vote
0answers
14 views

Choice of the learning algorithm for Recursive feature elimination

I have a dataset that I divide into 80% for training+test and 20% for validation I have been using Recursive feature elimination for feature selection with SVM on the 80% partition...
0
votes
0answers
10 views

What is the difference between Feature selection techniques for Classification versus Regression?

Is there any difference between feature selection techniques and methods for Classification, clustering, regression? For example, features with high colinearity are never preferred in Regression. Is ...
0
votes
0answers
6 views

Is the process of variable selection not testing of a disguised statistical hypothesis?

Please refer to https://robjhyndman.com/hyndsight/tests2/ This link discusses the incorrect use of statistical tests for variable selection, and gives examples of situations in which two predictors ...
0
votes
0answers
24 views

The utter futility of bivariate analysis

Consider the following quote from here: Moreover, univariable prefiltering, sometimes also referred to as “bivariable analysis,” does not add stability to the selection process as it is based ...
0
votes
1answer
25 views

Feature Selection Before or after Encoding?

Should I apply feature Scaling and Selection before or after the One Hot Encoding/Label Encoding? Please Correct me if I'm Wrong- Deal with Outliers Impute ...
0
votes
0answers
9 views

Identify Influential predictors

I have a dataset with binary class as outcome. I was exploring the data through plotting the variables for both the classes. For example, something like below ...
1
vote
1answer
14 views

What are the important Factors for Feature Selection in Classification Problems? [closed]

While doing a classification I have to choose from the ocean of choices at every step like model selection, performance criteria selection and all. But the important two things I get confused most of ...
0
votes
0answers
16 views

Model Examples always with 1 or 2 features

Why are all the model examples that I see on sklearn (e.g., https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.LocalOutlierFactor.html or https://scikit-learn.org/stable/auto_examples/...
0
votes
0answers
9 views

Order of discretization and variable selection

I want to test the effects of discretization in a dataset. The problem is that it has too many variables, so I have to do feature selection. My question is if I should do feature selection before or ...
1
vote
1answer
26 views

How to find significant predictors that can differentiate case and control without ML approach?

I have a dataset with more than 70 columns and I have an binary output column. What I did currently was to explore the dataset by plotting the bar and line graphs for the input variables vs output ...
0
votes
0answers
56 views

Best way to remove multicollinearity and feature selection for binary classification problem?

I am having around 1200 features 20k observations. Objective is to get the not highly correlated best 100-130 features to build binary classification models such as LR, hypertunned ML trees etc. ...
0
votes
0answers
5 views

Using supply as feature in price predictor model

On a machine learning model that outputs the optimum price of a product (ex: cars listed on some website), would it make sense to use the number of instances of that product as a feature? In the case ...
0
votes
1answer
15 views

How to manage a variable collected at various level in a machine learning model based on nested hierarchical data?

I'm trying to use machine learning to model the risk of healthcare-associated infections (HAI) for patients in a number of hospitals. I have variables both at the patient, ward and hospital level. ...
0
votes
0answers
3 views

How to build standard error and confidence intervals from Buckland variance estimator after variable selection

In this paper https://www.sciencedirect.com/science/article/pii/S0167947307003957 it is shown how to build an estimator of the variance of estimates computed after variable selection. It evaluates ...
0
votes
0answers
16 views

Bootstrap on LASSO regression

I am doing LASSO regression in order to reduce the amount of independent variables a bit. In this case, it's not the only thing I am doing, but the LASSO is the last step. However, I have read that ...
0
votes
0answers
7 views

In statistics, do variable screening and variable selection have same meaning?

I thought they had the same meaning.But some paper related to screening would propose new screening method,then it says ...
0
votes
0answers
49 views

Extracting AlexNet features and using them for CIFAR10

I have been given an assignment of using the pre-trained model of AlexNet (PyTorch, ImageNet) and use its FC8 feature (1000 dimensional feature output) to train a linear SVM for the CIFAR data. ...
0
votes
0answers
21 views

Using MCA/PCA together?

If I have a large dataset with continuous, discrete, and categorical data, is it appropriate to use MCA on the categorical features and PCA on the continuous, separately? I'm preprocessing my data ...
2
votes
1answer
35 views

Feature selection by lasso and cross validation of model with low sample number

I have a set of RNA-Seq data with 20 samples (so ~14000 features and 20 observations), of which I have 3 groups with 3,3 and 4 samples respectively and other just scattered around which I will group ...
0
votes
0answers
16 views

Should I use something which is just a relative feature of my original features for classification?

I am trying to adapt recency, frequency & monetary value (RFM) modeling into a phone usage dataset I have to classify users' emotional status. Here is a short explanation of RFM: basically its a ...
0
votes
0answers
14 views

Clustering binary data : feature selection vs Apriori

My data set is a 999 rows x 964 columns Panda DataFrame. Each row is a user. My data is binary : 0 for absent and 1 for present. I would like to fit the users ...
1
vote
1answer
26 views

Correlation vs Lasso for Feature Selection

I read from literature that the following two methods can be used for feature selection prior to model development: 1. Correlation factor between target and feature variables (select those features ...
0
votes
1answer
23 views

What are the steps and correct order of the operations in Machine Learning? [from Getting data to optimising models]

I've followed lots of tutorials on Machine Learning but in each of these, they go for a different strategy so it's quite confusing for me. I want to Know that what are the operations involved and what ...
0
votes
1answer
22 views

Survival analysis cox model and accelerated failure time (AFT) variable selection

I'm reading a paper about survival analysis. After doing the univariate analysis, the paper only put covariates which have significant results in the univariate analysis into the multivairate ...
1
vote
1answer
19 views

Variable Selection on a imbalanced data set

Suppose I want to perform variable selection on a highly imbalanced data set. Do I have to balance the data set either by downsampling the majority class or upsample the minority class before I ...
0
votes
0answers
9 views

Parameter significance analysis

I have 2 years of customer loan request data, which has basic customer details, loan amount requested, loan approved/rejected and 10 parameters on which loan approval decision is made. The process of ...
0
votes
2answers
22 views

The sufficiency of univariate selection

Why is feature selection such a common topic, when one could simply use SelectKBest to find the optimal set of features? The only trouble would be to find the best ...