Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

0
votes
0answers
8 views

Using mutual information for feature selection between feature maps

I want to do feature selection between 512 3X3 feature maps from convolutional layers. I want to calculate a 512X512 MI matrix and choose 256 feature maps with the lowest MI values between themselves (...
0
votes
0answers
10 views

Debiased regularised regression (elastic net)

I have ~55,000 binary observations and 12 explanatory variables (EVs). I am looking to perform variable selection, followed by inference on the effect of the retained EVs on the binary outcome. Since ...
1
vote
0answers
20 views

Feature selection and interpretation for black box classification systems

I just did terribly in an interview. One of the questions that came up was (paraphrased): Say you have a binary classifier, a neural net or a random forest, and a dataset with huge dimensions. ...
0
votes
1answer
22 views

Feature selection using Restricted Boltzmann Machine

I am new in the field of RBMs, DBMs and I cannot understand some things. I came across the idea of feature selection using RBMs (or Deep Belief Networks). Although the Hidden nodes which make new ...
-1
votes
2answers
15 views

What does this output from SelectKBest mean? [on hold]

I am using selectKBest from ScikitLearn. I am getting this kind of output with mutual_info_regression. Y Column: someYColumn, Predictors: ['predi.1', 'predj', 'predk', 'predk.1', 'predk.2'] ...
0
votes
0answers
15 views

SVM on small dataset: no difference in loss value after feature selection and small lambda

I am running an SVM on a very small dataset of 56 items, 20 features and 4 classes. I want to know which features are the most important for classification, and how they interact (no future real ...
0
votes
0answers
8 views

ensemble learning with conv net

I extract feature with a conv net at last fully connected layer got around 85% performances on training set and 80% on test set. I use feature from CNN TEST set and I train multiple classifier (svc, ...
0
votes
0answers
25 views

Do I discard all my dependent variables as proved by chi-squared test of independence?

I have 134 categorical columns in my data. 7 of which are categorical variables [ one variable is highly unbalanced and has 34 classes while all other variables just has 3-5 classes in each variable ...
0
votes
0answers
3 views

What are the time complexity of image feature extraction algorithms, including HS, HOG, MSER and SIFT?

Can somebody help me by writing me a time-complexity of each image feature extraction algorithms. Especially I am interested in Harris-Stephens(HS) corner detection, Maximally Stable Extremal Regions (...
0
votes
0answers
9 views

How important are hyper-parameters in SVM based RFE feature selection?

How important are hyper-parameters in svm based RFE feature selection? For feature selection using RFE (recursive feature elimination / selection), I have seen some publications where only "external" ...
0
votes
1answer
12 views

Can I use the Silhouette to measure quality of clusters in different dimensions?

Can I use the Silhouette to measure quality of clusters in different dimensions? For example, let's say we run kmeans for some $k$ using 6 features of the dataset. Mark the resulted silhouette as $...
0
votes
1answer
24 views

Interpretation of Eigenvalue vs. Singular Value plot

I'm doing some preliminary analysis on the feature matrix for a certain dataset (rows are observations, columns are feature dimensions). I have computed the SVD and PCA decompositions for this matrix ...
1
vote
1answer
21 views

Does it make sense to select SARIMAX parameters independently of covariates?

I have a vector of time series observations $y$ and a matrix of covariates $X$. I want to choose the best (in the sense of minimizing loss function, e.g. RMSE) $SARIMAX(p,d,q)×(P,D,Q)_{s}$ model ...
0
votes
1answer
11 views

Feature selection on full training set, does information leak if using Filter Based Feature Selection or Linear discriminate analysis?

In order to test a potential classification set, usually some data is kept as a holdout set, and not used for inner-cross-validation or model training. However, what happens if too many features ...
0
votes
0answers
19 views

Elastic net regression with uneven penalties for predictors

For a regression model where you are certain that y that depends on some predictors but are agnostic about whether some other predictors should enter, how should you incorporate this prior information?...
0
votes
0answers
12 views

Model Based Feature Selection vs Wrapped Method Feature Selection

I read about Wrapped Method Feature Selection, I get that it is to look at the features then test them against the predictive model that we need then find if it has an effect or not and then decide to ...
0
votes
1answer
14 views

Why are oracle inequalities called that way?

The oracle property is an asymptotic property of an estimator, and is about variable selection: An estimator $\hat \beta_n$ satisfies the oracle property if in the limit of $n\to \infty$, the ...
0
votes
1answer
40 views

Splitting large number of variables into three separate logistic models for variable selection

Is it appropriate to split up to 100+ variables into three groups then running each group into separate decision trees then run the new created features into their own separate logistic models to help ...
0
votes
0answers
13 views

The coefficients and p-values in the Firth logistic regression when the data set is imbalance

My question is that in the case where the contingency table has imbalance data in terms of binary response success and failure, can I confidently say that the two-level categorical predictor is ...
0
votes
0answers
17 views

Feature selection for a cartesian product of outputs

My understanding of feature selection is as follows: Pick an output. This is like the dependent variable of a functions. Lets call it $Y$. Provide all the columns that may or may not be related to ...
2
votes
0answers
21 views

Why does lasso return unstable features when using the same data?

I am using scikit-learn to shrink my data set having around 800 features. It is a very noisy data (market and economic data) To my best knowledge, lasso returns same features for the same data set. ...
1
vote
0answers
16 views

Caret rfe varImp: scaled variable imprtance for rfe results [closed]

I want to plot the scaled variable importance of a rfe object (recursive feature elimination). With the following code I compute the rfe model and the variable importance: ...
0
votes
0answers
5 views

Why 'sequentialfs' of MATLAB stops before the optimum feature subset is selected?

I am using "sequentialfs" of MATLAB to select features from 271 features for 871 subjects over 2 classes. I used the backward sequential function. I noticed the features selected after using the ...
0
votes
0answers
8 views

What is the methodology behind Filter Based Feature Selection (i.e. Pearson correlation, etc.) on Azure Machine Learning Studio?

Filter Based Feature Selection on Azure Machine Learning Studio supports feature selection and ranking through Pearson Correlation, Kendall Correlation, Spearman Correlation, Mutual Information, Chi ...
2
votes
2answers
68 views

What is the best way to get the most accurate results with this small dataset?

This is my first question here, I apologize if this is the wrong place or my formatting is not correct. My experience with machine learning and data science in general is a graduate level survey ...
0
votes
0answers
11 views

GLMM with Variable Selection and Non-Negativity Constraint

I am trying to run a fairly complex GLMM with random effects and smooths. There are about 10 of these independent variables. There is also another set of 1000 variables. From this set of 1000 ...
0
votes
0answers
12 views

Data Leak during data acquisition for credit scoring

I have a few questions about data leaks. Particularly, I'm interested in a credit scoring data can have leakages. I'm at the stage of data acquisition and I suppose I have target leak but not sure. ...
0
votes
0answers
22 views

How to choose drivers for forecasts based on vector autoregression

as mentioned in the title my question is how to choose from a large set of time series the best Driver for a forecast based on vector autoregression. I am sure that this question is very general. I ...
0
votes
0answers
9 views

Can annotation confidence affect MCAR missing feature values?

Concerning binary image classification, I've a dataset with over 4500 dimensional CNN and GIST features, though many of the feature values are missing NaN. This is ...
0
votes
0answers
13 views

Feature Selection Pipeline - Combining PCA and SelectKBest with Chi2 Test

This example applies a Feature Union using both PCA and SelectKBest: https://scikit-learn.org/stable/auto_examples/compose/plot_feature_union.html#sphx-glr-auto-examples-compose-plot-feature-union-py ...
0
votes
0answers
13 views

Manually creating target variable, getting f1 score 1

I am building a classifier for user engagement in my website. Basically, since there are no "proxy" for engagement, i.e. there is no pre-defined target variable, I came up with minimum thresholds ...
1
vote
0answers
62 views

XGBoost and AdaBoostClassifier feature importances

I try to compare XGBoost and AdaBoostClassifier (from sklearn.ensemble) feature importances charts. From this answer: https://stats.stackexchange.com/a/324418/239354 I get know that ...
0
votes
0answers
20 views

strong and independent extra explanatory variable doesn't improve linear regression

So I already have a linear regression on 3 predictors $Y = X_1 + X_2 + X_3$. Now I have an extra predictor $X_4$. Before I put in $X_4$, the original predictor using $\hat{Y} = X_1 + X_2 + X_3$ has ...
0
votes
0answers
6 views

Can normalized values and original values be combined in a feature vector classification?

I standardized feature vector before SVM classification in MATLAB. The feature vector consists of time domain signal features and its normalizations i.e., feature vector is a combination of actual ...
1
vote
1answer
35 views

Feature extraction vs Fine tuning with Restricted Boltmann Machines

I am reading a paper which uses a Restricted Boltzmann Machine to extract features from a dataset in an unsupervised way and then use those features to train a classifier (they use SVM but it could be ...
0
votes
0answers
19 views

How to do variable selection for Gradient boosting models like Xgboost and LightGBM

I am building a classification model with about ~110 variables and that gave me an AUC of about 71.96 on validation. I added about 10 more features and my AUC value decreased to 71.56 (which led to ...
0
votes
0answers
22 views

How to add new features to already trained model without training again on whole dataset?

Suppose, we have following features on which a classification model (Neural Network) is trained to predict whether a customer will buy Milk or not (0 :Will not buy, 1:Will buy) each week(n): ...
0
votes
0answers
50 views

Feature Selection For Random Forest

Random Forest aims to combine many decision trees to make good predictions for testing data in regression and classification. It is an ensemble learning method. I have a dataset with 100 samples, ...
0
votes
1answer
26 views

What is meant by Low-Order combination of features?

I came across a Machine Learning paper that talks about input with low-order combination of features. A statement says: The initial feature is used as the input of the model, and the non-linear ...
0
votes
0answers
25 views

Chi Square Test for Dimensionality Reduction

According to many resources, we should have categorical variable to be able to apply chi square test. ...
0
votes
1answer
38 views

Dimensionality Reduction - Feature Selection

For example, we have a dataset in which the samples contain 400 features. In this case, if we try to perform classification, we get very low accuracy because our learning model will become very ...
0
votes
0answers
15 views

Feature analyzing methods

I am a little confiused how to interpret following situation: I am trying to implement a image classification task using hog+SVM. For that i tried to analyze and understand the properties of the ...
0
votes
2answers
70 views

Choosing model for more predictors than observations

I'm working with a data consisting of 1000 observations of 2000 predictors and one variable we wish to predict. There are couple of problems I can't get around. I am aware that such setting has been ...
0
votes
0answers
8 views

Avoid learning certain known features / selecting alternative features in neural network training

In an application of neural network to a classification problem, often time one trains the network to pick out different features in the input data set (learnt by the hidden units) and classify the ...
0
votes
0answers
12 views

Partial Least squares regression - Variable Importance on Projection (VIP) method of selecting variables

I understand that partial least squares regression produces VIP scores for each predictor variable enabling variable selection (using a VIP threshold of >1). Does this method account for collinearity ...
1
vote
0answers
15 views

In Regression Analysis, should variable transformations occur before or after subset selection?

I'm looking at fitting a model that has many parameters. In order to simplify the model and prevent overfitting, I am planning to use the best subset selection for variable selection. My question is, ...
0
votes
0answers
15 views

Filter-based feature selection for binary classification with unbalanced classes

I have a data set with ~10k observations and ~50 features. Each observation is assigned one of two classes (labeled 0 and 1, say). Approximately 98% of the observations are class 0, and the remaining ...
0
votes
0answers
11 views

How can Item Response Theory be used to remove questions asked in a customer satisfaction survey?

I have results from a survey of around 30 Likert-style questions that are asked of customers on their opinion about company X. Each of the 30 questions belongs to a certain category. For example, ...
1
vote
0answers
8 views

factor of safety

I need to validate a specification on a part. The requirement is to withstand a minimum 1 kg of force (i.e. not fail at less than 1 kg). I was told that if I measure, on 6 specimens, a force of 2.5 ...
0
votes
0answers
39 views

Application of GAM on large dataset

I was suggested that my questions were too broad. As I commented below, I have nearly a million data points and perhaps a hundred variables. This may be a very basic modeling question: I am curious to ...