Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

1
vote
0answers
15 views

Strange encoding for categorical features

I am reading through https://arxiv.org/pdf/1609.06676.pdf which presents an extension of the isolation forest algorithm so that categorical features may be taken into account. On page 5, the authors ...
0
votes
0answers
15 views

Different set of predictors significant for different sample sizes - how to interpret results?

So I am trying a GARCH framework with external regressor(s) to predict returns. The external regressor, $y$, intuitively has useful lags that could predict the response. I'm slowly accumulating data ...
0
votes
1answer
10 views

How to deal with varying number of intervals and hence varying number of features dividing an audio signal while classifying these audio signals?

I've $2000$ audio signals, each divided into a number of time intervals/time frames of $50$ miliseconds (ms) and these signals have overlaps for $25$ ms. Now, the audio signals being of different time ...
7
votes
3answers
178 views

How to reduce predictors the right way for a logistic regression model

So I have been reading some books (or parts of them) on modeling (F. Harrell's "Regression Modeling Strategies" among others), since my current situation right now is that I need to do a logistic ...
1
vote
0answers
10 views

What is the connection of correlation between variables to decision tree feature importance

I am using decision tree regression to predict a variable. I realized that the highest correlating features to my target variable are not the ones I get returned with a high feature importance. ...
0
votes
1answer
10 views

Product Price Prediction - using online scrapped data [on hold]

AIM: To Predict Price of products based on data that I have taken from other online stores. e.g Predict price of Samsung Galaxy S10, data will be from multiple online stores. Problem: Which Machine ...
0
votes
0answers
19 views

Bootstrapping non-random samples

I perform some analysis (with the goal to select some relevant variables that are related between two experiments) on a group of samples (n = 158) but I would like to know how robust are, before going ...
2
votes
1answer
37 views

How to infer one-to-one/one-to-many relationship? [closed]

We have a file with IP addresses patterns as shown below: ...
1
vote
1answer
23 views

Which feature selection methods are suitable for regression problems?

I am using different feature selection methods for a regression problem in order to rank the features according to their importance. So far I have used scikit-learn ...
1
vote
1answer
27 views

How to choose a linear regression model when feature selection is used?

What is best way to choose a criterion for linear regression model performance? Should the best model refer to RMSE, adjusted R-squared or AIC values? What about when variable selection methods such ...
0
votes
0answers
12 views

How does multicollinearity affect the feature selection process?

I have a classification problem with a modest number of records (approx. 10,000) and dimensions (30 dimensions, 25 are categoric and 5 are numeric). The response variable has two classes (T/F). I'm ...
0
votes
0answers
14 views

A non random subset of a distribution is similar to the original distribution, why?

I have near 1 million correlations between two groups of variables. The distribution looks normal for the distributions of interest (centered around 0 and symmetric): Using some other methods (not ...
0
votes
0answers
22 views

Feature selection for mixed binary, continuous, and categorical data

I have a dataset including ~100 features to predict a response variable with 9 possible categories. The type of the features can be any of continuous, binary (0/1), or categorical with more than two ...
1
vote
1answer
23 views

Overrepresented Features in Clustering

So I was thinking, if I have a set of features (let's say $(X_1, X_2, X_3)$) that basically describe the same overarching feature $Y$, and can somehow be mapped $(X_1, X_2, X_3) \rightarrow Y$. In ...
0
votes
1answer
27 views

Should I use pretrained word2vec or train word2vec on my own dataset?

I am trying to perfrom fake news detection using machine learning naive bayes classifier. So far I have used BOW and TFIDF as my feature vectors. From research I have found that word embeddings plays ...
0
votes
1answer
19 views

(Feature Selection) Meaning of “importance type” in get_score() function of XGBoost

I'm trying to use a build in function in XGBoost to print the importance of features. My code is like ...
0
votes
0answers
7 views

Multiclass classification with a balanced dataset and one high-priority label

I have a balanced dataset for a multi-class classification problem with one high-priority label (this ought to be classified properly at all costs). How do I go about creating a workflow for this ...
1
vote
2answers
29 views

Score of importance from feature selection techniques

Can I get the score of importance for each feature in feature selection methos such as Chi2, Information Gain (IG), or Recursive Feature Elimination (RFE)? Or they just provide a list of important ...
0
votes
0answers
8 views

Timeseries space consumption optimization for machine learning model

I need to train machine learning model that can run on device with low memory. My features come from aggregations of time-series so I need to store all data in this time-series until it leaves the ...
0
votes
0answers
16 views

Number of lags in Ljung-Box test for feature extraction

I'd like to cluster time series based on static features, one of which is the Ljung-Box autocorrelation. After reading this question on "How many lags to use in the Ljung-Box test of a time series", I'...
0
votes
3answers
39 views

Lasso Regression as Variable Selection

Suppose we are initially given $p$ predictor variables. In lasso regression, we want to find estimates of the coefficients $\beta_1, \dots, \beta_p$ that minimize $\text{RSS}+ \lambda \sum_{j=1}^{p} |\...
2
votes
1answer
31 views

Variable Selection Methods

In many descriptions of variable selection, we are given the number of variables in advance and have to choose the variables according to some method. For example, in forward selection, we are given ...
0
votes
2answers
19 views

Test statistical significance with small number - Fisher's exact test

Before the part of the question related to the title, here is some background: I'm build a model that tries to predict a binary outcome ($Y$), given a set of features. Some of this features are ...
0
votes
0answers
20 views

R-squared from Backward elimination doesn't match that from linear model

I am trying to pick features using Backward Elimination on the Housing Prices dataset in Kaggle using the following function. ...
0
votes
0answers
32 views

Using temperature as feature in Neural Network

I'm currently putting together a Neural Network for doing Sales Forecasting for hundreds of products. The domain experts know that the sales spike when the temperature drops and so I started to use ...
0
votes
0answers
26 views

p value in backward elimination regression

I need some help with the backward elimination output from Minitab below. Can p values A, B, C, D be equal to 0.745? Or the p value should be smaller than 0.745?
1
vote
1answer
42 views

Select optimal points for Gaussian process with a well-known target function

I'm currently trying to select the optimal points for a Gaussian Process Regression, and the important thing is that i already know the whole target function. Therefore, it's not Online Learning ...
0
votes
0answers
16 views

Information Gain property

Studying about information gain I found in the web (from the presentation of a lecture) that $IG(C|X) = IG(X|C)$ is it true? How I prove it?
0
votes
1answer
22 views

Feature selection via conditional entropy

It looks like feature selection can be done with mutual information. Mutual information is related to conditional entropy by this equation: $I(X,Y) = H(X) - H(X|Y)$ Can we use conditional entropy ...
0
votes
1answer
25 views

How to give more importance to one variable in a logistic regresion model? [closed]

I'm adjusting a logistic regression model for prediction, but if the person interested says: All variables are important for me, but especially X2 is more important. How I give that variable more ...
0
votes
0answers
5 views

Variable selection in Bayesian spatio-temporal count regression

I am estimating a Bayesian spatio-temporal Poisson model. I have a relatively large set of explanatory variables (20ish) and each time I run the model it takes a few hours to complete. I have seen ...
0
votes
0answers
4 views

Iterative Orthogonal Feature Projection

I have read the paper Iterative Orthogonal Feature Projection for Diagnosing Bias in Black-Box Predictive Systems by Julius Adebayo and Lalana Kagal. Let's assume for example that we have a column ...
2
votes
0answers
46 views

Probability that feature selection in elastic net regularisation is meaningful - evaluating the statistical significance of chosen features

I have a research question - can I use baseline clinical features to predict my binary clinical outcome in individual patients? I am interested if the performance of my model is greater than chance. I ...
0
votes
0answers
22 views

Feature matrix as the kronecker product of two feature matrices. How to build an alternative?

I have two feature matrices $\textbf{X}$ and $\textbf{Y}$ which I encoded through one-hot encoding the rows of two feature matrices $\textbf{X'}$ and $\textbf{Y'}$. Thus, they are sparse with a few 1'...
0
votes
1answer
30 views

What are “good” features for feature selection?

I use sklearn's random forest for classification (two categories). The following code is used for "good" feature selection. ...
0
votes
0answers
12 views

Feature Selection with interactions in high dimensions

Is there any fast approach to find features considering interactions in many variables (~3000)? Many methods like RFE applying random forest would take very long. I tried MARS with degree=2 but it ...
8
votes
1answer
54 views

In multiple regression, why are interactions modelled as products, and not something else, of the predictors?

Consider multiple linear regression. This question might be deceptively simple, but I'm trying to intuitively understand why, say if I have predictors X1 and X2, then interactions between these ...
0
votes
0answers
48 views

Feature selection in xgboost vs GBM in H2O

I am working on a big data set( more than 100 variables) and 30 million observations. I tried to build 100 models with a grid search using both XGBoost and GBM in H2O (Sparkling Water). I realized ...
0
votes
1answer
61 views

Best Suitable feature selection method for ordinal logistic regression

I have 33 variables my dataset, I need to omit some less significant features then, which is the "best suitable feature selection method " for the Ordinal Logistic Regression?
0
votes
0answers
23 views

In supervised machine learning, why exactly shouldn't my input variables be correlated with my target variables?

I understand the reasons why my input variables should not be correlated to each other (multicolinearity). However, I also read that my input variables should not be correlated with the target ...
1
vote
0answers
21 views

How can i calculate Covariance between numeric attributes and Binary attributes?

I want to calculate covariance between binary attribute and numeric attribute For example: if x is numeric attribute, y is binary attribute. Then how can i calculate cov(x,y)?
1
vote
0answers
30 views

How to put list of unordered elements as features into a random forest model

Assume that I have 3000 unique rows of data to train with. For each data point, I have a special attribute called $SomeFeatureList. This is a list of independent features (categorical features) that ...
0
votes
1answer
31 views

Random Forest feature selection over-fitting doubt

My objective is to find genes that can be used as biomarkers with low error. I am using Random Forest (RF) using R package randomForest and following the steps in below link as it is has similar ...
0
votes
1answer
34 views

Is bagging involved in the split of node of a tree of a Random Forest?

I know that using Bagging method in a RF, implies that the subset we give to the root node of each tree, has randomly selected Features and Attributes. I also know that during the split of a node ...
0
votes
0answers
10 views

Caret: Feature selection with Chi2 / f_classif

I try to classify texts which I have converted to term-document matrices before. I would like to perform feature selection to reduce the number of predictors. In Python, you can do this by means of ...
1
vote
0answers
26 views

Using unnormalized log likelihood for model comparison with different features?

Is it possible to directly use the log likelihood from fitting a model, for the purpose of model comparison? For example, if I'm using a logistic regression model, and I want to see if adding ...
1
vote
2answers
38 views

Why is the Relief algorithm slow for large numbers of observations?

I am currently working on a multi-class classification problem and have a dataset of about 130 features and 120000 observations. Digging through literature I found the Relief feature selection method,...
0
votes
0answers
10 views

Predicting based on regressor measured over time

Suppose I want to predict whether a patient has post-operative complications. In addition to some 'usual' regressors, such as age and weights, I also have access to variables that are measured over ...
0
votes
0answers
28 views

Feature selection on training set without cross validation

I have a large dataset (1M+ samples) with 500 features. I need to create a predictive model that can be trained quickly. So, I want to perform an initial feature selection before building a classifier....
0
votes
0answers
10 views

correlation > 0.95 plus variance inflation factor for feature selection, dose it make sense?

I'm training a neural network for regression, the dimension of the neural network input is $140$. Based on domain knowledge, there are redundant predictors, so I want to select a set of uncorrelated ...