Skip to main content

Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

How can I use a neural network to choose a subset of features from a dataset? [duplicate]

Suppose my dataset has 256 features. Right now, what I can think of is this: Create an NN model like this: a. create a sequential model b. add a Conv1D layer c. add a flatten layer d. add 1024 dense ...
user366312's user avatar
  • 2,195
1 vote
0 answers
11 views

Assessing Random Search Cross Validation: Tuning in ElasticNet with Large Feature Sets

I'm working on estimating an ElasticNet model for a large dataframe with over 100,000 variables, resulting in a well overidentified scenario. To tune my model, I've set up a grid of hyperparameters (...
george1994's user avatar
0 votes
0 answers
19 views

What kind of classifiers we shouldn't use for feature selection?

Generally, I see that, for feature-selection, people use PSO as optimizer and inside the cost function, they use less powerful classifiers like SVC, Logistic regression, KNN, etc. Is there a reason ...
user366312's user avatar
  • 2,195
0 votes
0 answers
15 views

Feature selection for logistic regression and random forest (using Orange - no coding)

I’m using Orange to create a prediction model for the Indian liver patient dataset (binary target variable – either has or does not have liver disease – with 580 instances and 10 features). I’m using ...
Jess's user avatar
  • 21
2 votes
2 answers
59 views

Looking for a modification of variable importance in ANCOVA-type glmm

This question is about a statistical concept I think should exist. I would like to know if it has a name and hopefully an R package that will implement it. It is related to variable importance/...
IMH's user avatar
  • 41
0 votes
0 answers
28 views

PLS Regression on data with high number of zeros in dependent variable

I want to perform a PLS regression on a data set coming from spectral images (NIRS). My goal is to relate the different spectra to the total amount of a compound. To do this, I have a dataset ...
anttphy's user avatar
  • 31
3 votes
1 answer
35 views

What is the boundary curve for $λ_1$ and $λ_2$ that give at least a 0 component in elastic net?

Define the elastic net estimate: $ \hat{\beta}^{\lambda_1, \lambda_2} = \arg \min_{\beta \in \mathbb{R}^p} \left( \frac{1}{2n} \| y - X\beta \|_2^2 + \lambda_1 \ \frac{1}{2} \|\beta \|_2^2 + \lambda_2 ...
george1994's user avatar
1 vote
0 answers
56 views

Least-bad stepwise procedure for a simulation that shows issues with stepwise regression

I am well-aware of the issues that stepwise regression causes. I want to demonstrate some of them via simulation in a particular situation. I am thinking of a regression where I have some categorical ...
Dave's user avatar
  • 64.7k
4 votes
1 answer
173 views

Motivation for automated variable selection in case of p>n

I have written the following text as a motivation for using automated variable selection in cases where the number of variables (p) is greater than the number of observations (n). However, I am not ...
george1994's user avatar
0 votes
0 answers
38 views

Evaluating Lasso's Unique Solution and its consequences in applications?

I've grasped from a paper (https://www.stat.cmu.edu/%7Eryantibs/papers/lassounique.pdf) that Lasso may not yield a unique solution when the number of variables (p) exceeds the number of observations (...
george1994's user avatar
0 votes
0 answers
12 views

Log-likelihood calculation for unigrams

I am calculating the log-likelihood for each unigram that I generated by using the CountVectorizer to see each unigram's importance. However, I got all the positive value after calculating the log-...
Nick's user avatar
  • 1
0 votes
0 answers
48 views

Behavior of Lasso Estimator with More Predictors Than Observations (p > n) and Identical Correlations?

What is the behavior of a Lasso estimator if it is used in a dataset with more predictors (p) than observations (n), where all predictors are uncorrelated but highly relevant to 𝑦 y with exactly the ...
george1994's user avatar
2 votes
3 answers
90 views

Is feature-extraction and dimensionality-reduction a kind of compression?

I'm struggling to understand what these terms have in common: Feature extraction Feature selection Compression Dimensionality reduction Relatedly, the information / entropy in our data should always ...
sueszli's user avatar
  • 23
0 votes
0 answers
41 views

Variable importance in cluster analysis

I'm new to the cluster analysis, read lots of things but I'm not able to understand how to variables are ordered into cluster. I mean, I find that my data are clustered into 3 different cluster, but ...
Riccardo's user avatar
  • 101
0 votes
0 answers
47 views

How to deal with possibly important predictors omitted during the building of an OLS multivariate linear regression model?

I am building a descriptive model using OLS multivariate linear regression. I have a couple dozen candidate predictors, but only around 200 cases. Since I wanted at least 10 cases / variable for the ...
jorvaor's user avatar
1 vote
1 answer
48 views

Questions on Basic Data Cleansing For Linear Regression

I'm following some tutorials on doing some linear regression and as I was building my notebook, I'm working on outlier detection and amongst the techniques described for doing outlier detection, one ...
joesan's user avatar
  • 163
4 votes
5 answers
150 views

Is it valid to exhaustively test all possible combinations of features to find the best combination?

I have about 1000 labelled observations from about 50 subjects responding physiologically under different situations and am trying to classify the situation (usually into three classes of roughly ...
user1596274's user avatar
0 votes
0 answers
29 views

Can a Catboost oblivious tree split on the same feature more than once

When training an oblivious decision tree in Catboost, can it use the same feature more than once for splitting? Let's say there is a feature age. If the first split ...
NotProbable's user avatar
2 votes
2 answers
68 views

Is feature importance given by decision tree universal?

I'm wondering that if I have a set of features on a fitted classification decision tree with relative low feature importance, would it mean that these features would also be negligible when fitted ...
bachts's user avatar
  • 21
4 votes
1 answer
106 views

Variable selection strategies

I'm wanting to learn more about current best-practice approaches to variable selection, in the context of the reason for creating the model. I don't have much experience with modern methods like lasso,...
LucaS's user avatar
  • 771
1 vote
2 answers
31 views

PSPP multiple variable linear regression analysis

I'm just starting with linear regression, and I'm having trouble understanding it. It doesn't seem to make any sense to me. Yes, this is school work, but instead of asking for direct answers, I need ...
Juster's user avatar
  • 111
4 votes
2 answers
150 views

Detecting interactions in large logistic regression models

I have a dataset of a few million observations of a binary response with a low "Success"-probability of on average 1% to 2%. The dataset encompasses several categorical (~20 some with up to ...
g g's user avatar
  • 2,708
0 votes
0 answers
24 views

Improving a logistic regression where multiple signals separately yield the same accuracy, and combining them does not improve the model

I have a logistic regression that estimates the probability of an event occurring. There are roughly 10,000 data-points, and I have roughly 20 model features. The model features are each quite ...
olives's user avatar
  • 31
1 vote
1 answer
27 views

Estimating Confidence in Feature Rankings from Multiple Experiments with Non-Normal Data

Hello dear Cross Validated Community, I am a new doctoral student in bioinformatics, and I am working on a project involving multiple experiments, each generating a single numerical result for each of ...
Thomas Rauter's user avatar
1 vote
0 answers
24 views

Determining significant predictors for a dataset with the outcome variable [closed]

I am working on a dataset and trying to get the significant predictors? The dataset contains numeric and categorical variables and there is an outcome variable. What would be the best approach? I am ...
EngineerP's user avatar
1 vote
0 answers
29 views

Logistic regression with a lot of predictors/ general understanding

i'm currently planning to write my bachelor thesis but it's been a while since my last statistics seminar, i'm extremely rusty and so any guidance here would be appreciated. I have a relatively small ...
statbear's user avatar
0 votes
0 answers
43 views

Permutation Feature Importance in the Context of Cross Validation

I am considering two apporaches to calculate the mean, std and standard error (se) for ...
Kevin Li's user avatar
  • 103
1 vote
1 answer
33 views

How can I find what is driving the change between my updated model and the original model?

I'm working on updating a model of rents (which is currently simple OLS) for my employer who has a large national portfolio. By tweaking here and there and drawing in a large amount of exogenous data ...
wjb_hwe's user avatar
  • 11
1 vote
1 answer
59 views

Manual variable selection in GAM model based in deviance explained

I'm fitting a generalized additive model (GAM) to predict bottlenecks in a manufacturing process of a company. They have data of the bottlenecks that occurs in their process of making hot steel rolls. ...
Raheshi Knuwga's user avatar
0 votes
0 answers
14 views

Feature Selection Before or After kNN Imputation?

From my understanding, kNN imputation is dependent on the variables where any two cases do not have missing values. Thus, would it be ideal to do feature selection before or after imputation?
aj12302's user avatar
2 votes
1 answer
41 views

Questions about the process of feature selection through feature importance

'Shap feature importance' was obtained through xgboost, and variables with the lowest feature importance were removed one by one from 50 variables until only 1 variable remained. As a result of ...
JAE's user avatar
  • 89
6 votes
1 answer
181 views

How to adjust p-values in a multiple regression when variables have been a selected by Lasso?

I have designed an analysis where I am testing a lot of variables together. So I first apply a Lasso regression to select the top variables, and then I run a standard (unregularized) multiple ...
PlasmaBinturong's user avatar
0 votes
0 answers
31 views

Could the community provide critiques on my method for picking features for modeling?

I just started a new job that is a bit more stats heavy than my old DS job. In my old gig I was mainly in the domain of scraping, descriptive statistics, visualizations, dashboards, etc. So I'm having ...
Nye307's user avatar
  • 11
1 vote
1 answer
62 views

What does it mean for a variable to be statistically significant but not selected as important by classification methods?

I'm basically playing around with some lipidomics data to practice, so my question is purely theoretical. I wanted to see if I could find lipid classes that differ between two groups and I was ...
maglorismyspiritanimal's user avatar
0 votes
0 answers
18 views

Stock clustering based on fundamental reporting

I want to made a stock clusterization, based on their fundamental features from companies quarter reports. I collected quarter reports from 2018 to 2022. Some companies have reports for all quarters ...
TImur Nazarov's user avatar
0 votes
0 answers
9 views

Selection of important features through cross validation and shape value importance

To extract important features for the binary classification problem, recursive feature elimniation was performed based on the importance value of the shap value through nested cv. The first thing I am ...
JAE's user avatar
  • 89
0 votes
1 answer
46 views

Analysis of interactions in quantile regression models

I possess a large dataset with n ~10,000. My goal is to develop a quantile regression model using rq() from the ...
Mikołaj's user avatar
  • 141
0 votes
1 answer
24 views

can i have multiple similar features derived from same properties in a dataset?

Say I am fitting a (linear) regression model to a hundred-row dataset, whose data records a series of experiments that I can hardly reproduce or further conduct to get more records, I notice that ...
Yuuya's user avatar
  • 9
1 vote
1 answer
27 views

Determine best model for variable selection process

When we do variable selection, theoretically we fit a model for each selected subset of variables. We can then compare all models and select the best. There are various statistic to help selecting ...
amineh's user avatar
  • 87
6 votes
2 answers
119 views

Comparison of roc-auc values ​through cross-validation for feature selection

Through 10 cv, the roc-auc value was obtained as follows. At first, I tried to select the feature with the highest average roc-auc value, but I had doubts about whether the difference in these scores ...
JAE's user avatar
  • 89
3 votes
1 answer
64 views

Can I aggregate several continuous variables into percentages and then compare those percentages between groups?

I have a dataset with the concentrations of several lipids. I'm interested in finding lipids that are altered between two conditions, but the lipids are not indepentent from each other and the ...
maglorismyspiritanimal's user avatar
3 votes
2 answers
129 views

How many zeroes from lasso linear regression?

Given a dataset $X$ with $d$-dimensional features $x \in R^d$, and a response variable $y$ you can perform a lasso regression, ie linear regression with L1 regularization, as $$ \min_{\beta} (X\beta - ...
alexmolas's user avatar
  • 288
0 votes
1 answer
61 views

How to handle correlated variables before using Recursive Feature Elimination?

I have seen a few Kaggle notebooks that list without reason that RFE works better when removing correlated variables. I struggle to see the reason why so I conducted some of my own research and would ...
AvanishM's user avatar
0 votes
0 answers
19 views

Is the random forest classfier affected by related samples or biological replicates?

Correlation or collinearity between features can impact the results of random forest. So can having unbalanced data. However, I have not found a clear answer on whether having related samples can ...
Tal's user avatar
  • 11
1 vote
0 answers
89 views

Why does feature importance decrease for highly correlated variables?

I am investigating the relationship between correlation between features and its impact on their feature importances using sklearn's DecisionTreeClassifier algorithm. I manipulated the correlation of ...
AvanishM's user avatar
0 votes
0 answers
14 views

Model Significance for Svy logistic regression

I am working on the survey logistic regression. More than ten predictor variables are identified based on a P-value of < 0.25 on bivariate analysis. However, when I try multivariate analysis the ...
Melese Siyoum's user avatar
0 votes
1 answer
51 views

Feature selection using backward feature selection in scikit-learn and PCA

I have calculated the scores of all the columns in my dataframe, which has 312 columns and 650 rows, using PCA. I used the following code: ...
Mostafa Bouzari's user avatar
0 votes
1 answer
58 views

Is it necessary to remove redundant variables from a random forest classification model?

I am running a random forest model in the following variables(attached): Is it necessary with Random Forests Classifier to remove highly correlated variables or should I leave the model as is? If I ...
John Gallop's user avatar
0 votes
0 answers
43 views

Variable selection based on PLS

What is the logical way to select the variables from PLS? Does choosing the feature from loadings and loading weights make sense? Loadings... Loading weights... Regression coefficients... Variable ...
vdu16's user avatar
  • 13
0 votes
1 answer
54 views

Comparing two groups by the counts of their features

Imagine there are two different groups of individual samples. I know they are different but I don't know why. For example in biology a group of sick individuals and a group of healthy ones. Now for ...
lesolorzanov's user avatar

1
2 3 4 5
49