Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Methods and principles of selecting a subset of attributes for use in further modelling

0
votes
0answers
16 views

Modeling based on certain constraint

I am dealing with a dataset that contains exactly one dependent variable $y$ and 5 independent variables $x_i:x_1,x_2,x_3, x_4,x_5$. My goal is to find the best combinations of $x_2,x_3, x_4,x_5$ ...
0
votes
0answers
29 views

Variable Selection using automatic selection [on hold]

I have a data set with the following columns ...
0
votes
0answers
14 views

VAR model residual autocorrelation and variable selection

I have a question on VECM model. I have a set of variables I had planned to include in my VECM model where one particular variable may be trend stationary (@ 10% s.l. by ADF test) while the rest are ...
-1
votes
0answers
20 views

How to do data cleaning in dataset consisting more than 150 features?

I have one dataset with binary classification target variable. There are 185 variables in this dataset. Most of the data is missing and many features contains information like comments and ...
0
votes
0answers
8 views

Measure of goodness of 2D data points for classification

Is there a good measure of how good my dataset is for the task of classification. The ideal scenario for classification is that points for each class should be clustered closer and each cluster of ...
0
votes
0answers
5 views

A better way to tell whether a feature can separate classes better, in severely data imbalanced situation

I was working on feature generation. The goal is to find promising features that may separate the 2 classes better, BEFORE running the model. The data is severely imbalanced, positive:negative is ...
3
votes
1answer
30 views

remove features that has zero feature importance in random forest

We have 10 features that is pre-selected from domain knowledge. We ran random forest with those features. one of the feature has zero feature importance. My question is: For those features that has ...
0
votes
0answers
23 views

Determining which variables to use in regression model

So I'm trying to fit some binary outcome data to a logistic regression model. Besides the binary outcome I have several different metrics (numeric, integers, as well as factors) associated with each ...
0
votes
0answers
18 views

What methods can be used to select the lag variable for a multivariate time series?

I have a time series, it looks something like this (where X are the predictors and Y is the output variable): ...
0
votes
0answers
12 views

Is variable selection using cross validation prior to the selection of a class. method (eg. SVM, logistic regression, naive bayes, etc.) valid?

I plan on constructing a classifier using the following "algorithm": Use the caret package in R to select variables Train different classifiers using various methods on the variables found in (1) and ...
2
votes
0answers
12 views

Pairwise recommender system

How can I model the following situation? I am looking for high level recommendations. if say a man with certain attributes (income, age, etc ...) rates a car with a given attributes (color, ...
0
votes
1answer
26 views

RFE number of features with hyperparameter fine tuning within cros-validation

I would like to use cross-validation to select the number of optimal features to select (n_features_to_select) in the recursive feature elimination algorithm (RFE) and the optimal hyperparameter of an ...
1
vote
0answers
33 views

Reducing number of time series in VECM

I am exploring using VECM for several time series that are all I(1). However, I am hoping to avoid a model that is too large and was wondering if there is any way I can filter out several variables ...
1
vote
0answers
12 views

Problem to get z-score plot with boruta R [closed]

I am working with Boruta R package for selection variable. I am trying to replicate plots from Maya Gopal and Bhargavi: FEATURE SELECTION FOR YIELD PREDICTION USING BORUTA ALGORITHM (2018). https://...
0
votes
2answers
33 views

How do I select right features

I am working on Boston Dataset in which the aim is to predict the MEDV which is median value of owner-occupied homes in $1000s. ...
1
vote
1answer
30 views

How we can avoid making L2 regularization causing the model to learn a moderate weight for some non-informative features.?

Referencing to an example explained in free google machine learning course Imagine a linear model with 100 input features: 10 are highly informative. 90 are non-informative. Assume that all ...
1
vote
1answer
30 views

help with understanding strong relevant feature

According to Kohavi and John (page 5), in the XOR problem feature $X_1$ is strongly relevant, but I suspect this statement. The strong relevance definition implies $p(Y=y|X_i=x_i, S_i=s_i) \ne p(Y=y|...
1
vote
1answer
37 views

What happens if I train a model on a data set that includes a duplicated feature?

The Question Suppose I train a predictive model on a set of features $x_1, \dots, x_n$, but for some $i \neq j$ we have $x_i = x_j$ for every data point in the training set; i.e. one of these ...
1
vote
0answers
19 views

ANOVA F Score for feature selection

so I am trying to solve a binary classification problem but I have 104 features on hand. I am trying out a filter feature selection method implemented through SelectKBest - SKLearn in python. I've ...
1
vote
0answers
11 views

Feature ranking for ANN

I am doing a regression analysis on 1 hidden layer feed forward NN and have 18 input features with 1 output. I Intend to do some feature ranking so it can give me an idea of the most important to the ...
2
votes
0answers
32 views

Feature Selection in Linear Regression

I had to choose the best set of features from 200 of them. Currently the approach I am using is to: Loop through the features In each loop, add a feature, check the loss of the model, store this ...
2
votes
0answers
29 views

Feature selection Stability of Elastic net vs Lasso

I am new to regularized regression, and I was told that Elastic net overcomes many issues of the Lasso Regression. Especially, in the case of highly correlated predictors, Lasso variable selections ...
1
vote
0answers
10 views

Feature selection method based on target variable in R [closed]

I have dataset from 30 different features and a result variable. All the features have values like -1,0,1 and my result variable contains -1 and 1 , which 1 means record is healthy and -1 means ...
1
vote
0answers
29 views

can i use random forest for feature selection and then use poisson regression for model fitting? [duplicate]

Variables that are important in random forest don't necessarily have any sort of relationship with the outcome. So would it be wise to use random forest to gather the most important features and then ...
0
votes
0answers
50 views

How to interpret/choose alpha in ridge regression

I have questions on how to apply ridge regression on my data set, which has about 75 samples with 8 features (x's) and usually 3 targets (y's). I tried the following feature engineering methods. ...
2
votes
1answer
91 views

Modeling various skewed data

Below is a pair plot of the types of distributions (Time Series) I've been attempting to run models upon. Two of the features are strongly collinear (the distributions of last 2 on the diagonal of the ...
1
vote
1answer
39 views

Feature Selection Techniques [closed]

I am completely new to statistical modelling.I wanted to know what are the feature selection techniques. Say I have 10 variables but I need to what are actual important one's among them. I have read ...
0
votes
1answer
46 views

What to conclude for the data-set when the variance for principal components is too low or too high?

I am working on analysing and visualizing a dataset having 12 features and came across PCA. I reduced the dataset to 2 principal components which together explain a variance of 18%. I was able to plot ...
1
vote
2answers
48 views

What would it mean to select features in a “greedy” fashion?

I'm currently taking a machine learning course at university, and came across a concept that I'm having trouble wrapping my head around and would appreciate some help. We've recently been given an ...
0
votes
0answers
7 views

chaos complexity uncertainty - empirical measurement for numeric and rank-ordered data

I´m not sure if this is the right community but because my questions is explicitly targeted at the empirical operationalization I will post it here: At the moment I´m studying several subsamples (...
0
votes
1answer
26 views

unstable result using boruta for feature selection

The Boruta algorithm is a wrapper built around the random forest classification algorithm. It tries to capture all the important, interesting features in data with respect to an outcome variable. I'...
3
votes
3answers
53 views

For high dimensional data, does it make sense to do feature selection before running elastic net?

I have a dataset with $n = 800$ observations and $p = 2000$ features. I'm running elastic net for binary classification. My question is: Does it make sense to do some feature selection to reduce the ...
0
votes
0answers
11 views

One Hot Encoding: split feature into as many categories as possible, or lump data into smaller no of bins (including multiple split categories)?

I am working on a Machine Learning problem on a bike-share system database to predict the total number of bikes rented (per hour) based on other data. I used one-hot-encoding to split up the ...
0
votes
0answers
10 views

Can I use Gradient Boosting Classier to determine feature importance without worrying about precision, recall and accuracy?

My boss is interested in understanding how certain actions improve user retention WoW. I decided to build a GBDT model to assess those features. My question is: Does accuracy, precision or recall ...
1
vote
1answer
30 views

Using categorical feature as both a continuous feature, and also doing One hot encoding. Is this overkill?

I am working on a Machine Learning regression problem, with a data-set where I have data from a period of several years. From the "date" feature, I extracted the week number (0-53). Next I am doing 2 ...
1
vote
2answers
51 views

Dropped 2 Categories in Dummy Variables (Logistic Regression)

I understand that when modeling, dummy variables should be k-1 and the dropped category should be the baseline. However, I do not know how to interpret if after feature selection 2 more categories of ...
0
votes
1answer
56 views

t-test or paired t-test to detect drift for suicide prediction

Context and data I am studying suicides among the military. I created a table that aggregates certain metrics (number of holidays, number of hours worked, etc...) for each officer, for each month ...
1
vote
1answer
27 views

Is Elastic Net my best choice for finding sparse linear models in correlated features?

I have a linear regression problem, 1000 data points, but with 36 correlated features, those features are very highly correlated. And I know the ground truth must be linear. I know Lasso would give ...
1
vote
0answers
14 views

Use the features selected with RFE SVM linear for prediction of SVM rbf

I was wondering if the features selected with RFE with SVM linear kernel are still "good" features when we use a non linear model, like SVM rbf kernel. This comes in practice when you want to use SVM ...
0
votes
0answers
56 views

When can a feature independence assumption be reasonable and when not?

For example, Naïve Bayes assumes that the features are conditionally independent and they perform really well. Is there a time when assuming features are conditionally independent not so reasonable? ...
0
votes
0answers
17 views

Why statistical model significance test seldom seen in feature selection of Machine learning?

Background What I learned from the regression class in statistics told me that in order to remove a feature, or several features, we need to run a statistical significan test by checking the $F$-test ...
0
votes
1answer
45 views

Decision tree without the “tree”

I would like to construct something like a decision tree. However, instead of using "recursive partitioning" to build a tree, I would like to find an optimal set of "global" splits. For example, in a ...
2
votes
0answers
48 views

Feature selection using chi squared for continuous features

I'm looking at univariate feature selection. A method that is often described, is to look at the p-values for a $\chi^2$-test. However, I'm confused as to how this works for continuous variables. 1. ...
1
vote
0answers
29 views

Machine Learning - Feature Hashing Optimum Dimension

I am using the hashing trick for a machine learning problem: https://en.wikipedia.org/wiki/Feature_hashing#Feature_vectorization_using_the_hashing_trick In this particular problem there are a large ...
0
votes
0answers
26 views

Recursive feature elimination with cross validation and random forest classification: I cannot interpret the result

I ran an expirement with my dataset and using the recursive feature elimination with cross validation and random forest classification in a binary classification problem. I came up with this result: ...
0
votes
0answers
13 views

model calibration in complex data

I am working with a complex dataset (national inpatient sample) which has weight, cluster and stratum variables. My aim is to look for predictors of pediatric post-operative respiratory failure. I ...
0
votes
1answer
32 views

Can we improve a model by dropping unimportant features

I have a Random Forest Model which, after using StringIndexer and HotEncoder, has got around 1300 features. I calculated the importance of all these features and I found out that more than 500 ...
1
vote
1answer
15 views

Does it make any sense to apply a GETS modeling algorithm to a panel?

Let's assume to have a panel including observations for 88 individuals over 18 years (1584 observations). The panel is now populated with a broad set of 50 possible regressors that I would like to now ...
0
votes
0answers
9 views

Calculate gini importance or MDI on the OOB data?

"gini importance" or "mean decrease impurity" (MDI) is one of the methods of calculating feature importance in tree models. This downside of this method is that it bias towards variables with more ...
4
votes
1answer
46 views

how to deal with correlated/colinear features when using Permutation feature importance?

Permutation feature importance (PFI) is a nice way of getting feature importance in black-box models or models where it is difficult to characterise the relationship between the features and the ...