Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Approximate ratio of no. features to no. samples to force a feature selection

I know that feature selection is adopted by many machine learning scientists with the hope of reducing model complexity, avoiding curse of dimensionality, avoiding overfitting lowering training times ...
0
votes
1answer
36 views

Variable selection/ Classification time series binary outcome

I have a dataset of subjects with a binary outcome (1=disease, 0=healthy), and a column with the time in which the levels of several biomarkers (continuous values) were measured at each time point for ...
0
votes
0answers
8 views

Define lambda.1se in cv.glmnet [closed]

I came across this topic (Variablity in cv.glmnet results) when trying to find the best solution in defining my coefficients while using the cv.glmnet function. ...
1
vote
1answer
39 views

Variable selection without strong theory: Can we do better than LASSO for prediction?

When a variable of interest has many plausible explanatory variables, and one lakes strong theoretical or subject-matter grounds for selecting among them, it is tempting to build a “kitchen sink” ...
0
votes
0answers
9 views

Suitability of Random survival forest (RSF) as variable selection method for Cox proportional hazard model

I am trying to find ways to validate predictors selected by random survival forest (RSF) for survival analyses. a) Would it be appropriate to use random survival forest as a variable selection ...
0
votes
0answers
7 views

What are the advantages and disadvantages of including a baseline measurement variable in longitudinal data analysis?

What are the advantages and disadvantages of considering a baseline measurement variable in longitudinal data analysis? A baseline variable could be the number of seizures each patient had per ...
0
votes
0answers
18 views

Deep learning's auto extraction of representation

I am following up on a paper that demonstrates using deep learning (CNN) for classification. Specifically, their approach transformed the spatial data into fixed-length segments appropriate for CNN ...
1
vote
0answers
14 views

Get best performing feature

A data set that looks like this ...
0
votes
0answers
7 views

Statistical Significance with Multivariate Logistic Regression and Feature Selection

Background: Given a dataset with output y (Bernoulli distribution) and features x_0, x_1, x_2, ..., x_n I want to test which ...
0
votes
0answers
11 views

Consecutive Feature Selection-CV and Model Selection-CV [closed]

I want to ask a question about general workflow of algorithm development. I want to include a "feature selection with Random Forest" step into my workflow but I have doubts about data leakage. It ...
0
votes
1answer
35 views

Feature selection for time-series

I am new to ML and was exploring a time-series dataset for the very first time. The aim was to predict the volume of vehicles passing one of the 4 junctions given some historical data. After dividing ...
0
votes
0answers
23 views

Mutual information estimated on subsets of data

I am estimating the mutual information for a continuous data set using the kNN-based mutual information estimator proposed by Kraskov et al [1]. Lets consider two features $X$ and $Y$, and the ...
0
votes
0answers
21 views

Find Correlation between Grouped and Ungrouped Data

I have 2 datasets. First is a Quarter-yearly data that has PTR (Pay-through-rate) value associated with an Agent for every Quarter. The second dataset has the detailed data for Sales related to those ...
0
votes
1answer
24 views

Two ML models use different features. Does knowing the features of one model help improve the accuracy of the other model?

Suppose two firms are operating in the same field (e.g. insurance). If firm 1 knows which features firm 2 is using in their model, can firm 1 improve its model using that information? What if firm 1 ...
3
votes
1answer
276 views

In variable selection, Lasso over-penalizes coefficients. How to prevent?

I was doing some simulation on the Lasso. Particularly, I set p=200 variables, where only the first 5 have non-zero coefficients. I generated a training sample of size n=100. Whatever I do to tune the ...
0
votes
0answers
23 views

How to handle empty feature values in Random Forest

I have a question about Random Forest and features. I have put down a table below with an example that should describe the problem I wonder about. I have 2 Features(SUN and CLOUDS) that are always ...
0
votes
1answer
18 views

What are these features called (LOG and C3)?

I used a feature extraction code, where two of the features are unknown to me. They work well for my model but I don't know the formal names for them. The first one has the following python ...
1
vote
0answers
8 views

Positional features and feedback loops in ranking

I read the following in the Google "rules for machine learning": Rule #36: Avoid feedback loops with positional features. The position of content dramatically affects how likely the user is ...
0
votes
0answers
6 views

Ranking: Only allowing features to have positive weights

I read the following in the Google "rules for machine learning": Rule #35: Beware of the inherent skew in ranking problems. Only allow features to have positive weights. Thus, any good ...
0
votes
0answers
17 views

Why is the use of the F-Statistic / p-value as a criteria in Stepwise Model Selection outdated? [duplicate]

I am coming from the field of psychology and in most publications Model Selection (OLS, Regression) is done via Forward/Backward Selection using the F-Static/p-value of the regression coefficients to ...
0
votes
0answers
7 views

Removing features in a training set with no analysis?

The following question is concerning a project I am working on. Lets say I have a large dataset with 30 features (columns). I would like to build a Binary Classifier. Say that 10 of those features ...
0
votes
0answers
17 views

Do I need to select features for XGBoost [duplicate]

I have a regression task and I have around 79 numeric features that predict a numeric target value between 0 and 1, I used gradient boosting trees as there are several relationships both linear and ...
0
votes
0answers
11 views

Are there assumptions for using the regsubsets function in R?

I'm doing a multiple linear regression with 24 independent variables. I don't need to check the OLS assumptions because I'm only interested in R^2 that each variable explains. To get a smaller model I ...
0
votes
0answers
11 views

Feature encoding with non-fixed feature variables?

Lets say i have a feature matrix like this with Y being the label column. {A:x, B:y, Y:1} {A:p, C:q, Y:0} x,y,p,q are all distances. I can modify this matrix to have all feature columns ...
1
vote
0answers
13 views

How to (easily) calculate confidence & prediction intervals AFTER a model selection procedure?

First of all, this question refers mostly to linear regression. When someone uses a model selection procedure, for example, choosing the optimal dimensionality of the predictor space via best subset ...
1
vote
0answers
34 views

Machine Learning Feature Importance Method Disagreement (SHAP)

I am interested in reasons as to why different feature importance methods might give different feature rankings. In particular, Shapley values vs other methods such as weight/gain from OOB score. ...
2
votes
1answer
14 views

Maximum accuracy of a set of features, can a small number of relevant features may impact negatively on model performance?

I'm working on a project to predict students at risk of dropping out, my dataset currently consists only of the students academic records (their grades and what courses they took). I'm hoping to get ...
0
votes
0answers
17 views

Interpreting Variable Importance of XGBoost based decision tree models

While building an XGBoost based decision tree model, working with a set of say 100 variables gives a particular variable importance. Now if I want to improve performance through an additional set of ...
1
vote
1answer
17 views

Use text as feature in classification algorithm

I'm developing a classification model and for each sample I have a mix of numeric and categorical features. I also have a paragraph of text describing each sample. I'm looking for ways to incorporate ...
1
vote
3answers
48 views

What is the purpose of dimensionality reduction?

Does dimensionality reduction using PCA or UMAP or others try to preserve the most important features of the data so you can see it in a 2D or 3D space?
0
votes
1answer
14 views

Does it make sense to do feature selection after PCA?

I have a dataset of 50 features that resulted after PCA was employed (originally, the dataset had 343 features. The 50 features are the principal components obtained with PCA). Does it make sense to ...
0
votes
0answers
21 views

How Helmert Encoding is done here?

I was going through an article on Helmert Coding there I encountered an example in which I was not able to understand how Helmert coding is done. I went through How to calculate Helmert Coding this ...
1
vote
1answer
36 views

Understanding feature selection in Lasso regression

Given a selection of features, how can I get some insight about why have those features (and not others) been selected? Is there a standard approach?
0
votes
0answers
10 views

Feature Engineering: How to deal with imbalanced numerical/categorical features

I'm analyzing a data set and solving a classification problem and find that values concentrate on one number in many features. For example, a categorical feature 'loan' indicating a person having loan ...
0
votes
1answer
23 views

What is more meaningful in causal inference - magnitude of coefs or information criterion?

I have 100+ variables with 2000+ observations. I am trying to pick most important variables to put it into casusal inference models and get insight into the "true" process. I am aware of the ...
1
vote
1answer
38 views

Variable importance logistic and random forest

I have created variable importance plots using varImp in R for both a logistic and random forest model. I want to compare how the logistic and random forest differ in the variables they find important....
1
vote
1answer
28 views

how to combine recursive feature elimination and grid/random search inside one CV loop?

I've seen taught several places that feature selection needs to be inside the CV training loop. Here are three examples where I have seen this: Feature selection and cross-validation Nested cross-...
1
vote
0answers
13 views

Classification: Random Forests and Variable Importance

In classification, when we want to get the importance of each variable in the random forest algorithm we usually use Mean Decrease in Gini or Mean Decrease in Accuracy metrics. Now is there a metric ...
2
votes
1answer
27 views

feature selection after extraction process

I read that it is recommended to use feature selection after feature extraction process. But there is something missing in all ...
0
votes
0answers
11 views

Recursive Feature Elimination with a maximum number of features?

I want to find the optimal number of features and feature names in a dataset with 500 features. Let’s say that I ran recursive feature elimination with cross validation (RFECV) and found 25 optimal ...
0
votes
0answers
13 views

How smart is Random Forest with the Features

I have a question about Random Forest. The question is how smart Random Forest is in this scenario when it comes to Features. I will take the example. 1. We have Feature A that has a Gini Index of 0....
1
vote
1answer
34 views

Is it ok to keep a very strong predictor and other weak predictors in the model? The model built is GBM

Age is coming out as a really strong predictor compared to other variables. This is a classification problem, the dependent variable is a (0/1)
0
votes
1answer
9 views

How to apply imputation when creating an analytical base table?

I am asked to read up on how to deal with missing values. From what I read I can use imputation with a package like MICE (for R) to automate this process. However I also read that when I am missing ...
1
vote
0answers
23 views

Check which parameter causes less dispersion in a sample set - When to use CV (Coefficient of Variation) or CD (Coefficient of Dispersion) etc

The overall goal is to extract/engineer features from approximately 100 segments (various lengths but always more than 80 data points) that are as similar as possible to each other and have a very low ...
0
votes
0answers
26 views

Can we average the coefficients from bootstrapped samples for Logistic Regression with L1 regularization?

I have a model built using logistic regression with L1 regularization (glmnet package). I built this model using 1% of total data available to me. To ensure that ...
0
votes
0answers
21 views

What is the best way of reducing the number of independent variables of a GLMM?

I am trying to model the abundance of beach litter across different beaches by using (currently) 13 independent variables. I have a very big dataset and therefore think that it is acceptable to use ...
0
votes
0answers
12 views

VarianceThreshold by sklearn for feature selection in regression

For my assignment I am working with a data set that has only about 300 data samples but over 5000 features which makes me wonder if p >> N is already given. If it is given and I was to solve this ...
0
votes
0answers
8 views

I want to replace XGBRegressor with a simple model to make feature selection

I will make some for loop on to select the best features by my Data frame is big 10M row and about 50 columns so if i replaced xgb with a single Decision tree would it be the same results for the best ...
2
votes
0answers
25 views

Using LASSO for Only Variable Selection [duplicate]

How valid is an approach of using LASSO to determine appropriate values to then use in a logistic regression that does not use any penalties?
0
votes
2answers
58 views

Checking Normality of Numerical, and Categorical Data

I have come across 3 questions on the title subject. Why is it necessary to do a normality test? To check if data is imbalanced or not? Are these 4 methods of checking if the data follows normal ...

1
2 3 4 5
39