Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Methods and principles of selecting a subset of attributes for use in further modelling

0
votes
1answer
15 views

Anomaly detection and feature importance

A friend of mine gave me an idea. I tried to find anomalies with the help of DBScan and isolation forest and considered as anomalies the observation detected as anomalies by both algorithms. Now i ...
2
votes
0answers
18 views

Elastic Net and collinearity

I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical. I choose the alpha and lambda parameters through 10 fold cross ...
0
votes
0answers
7 views

Finding Correlations in Space

I have a 2D space of images. There are 40 images in this space. Each has an (x,y) co-ordinate. This 2D space was obtained through multi-dimension-scaling from a human experiment. I would like to ...
0
votes
0answers
10 views

Find the best “pair” of features to predict binary response with lda

Let's take the mtcars dataset: I want to find the 2 best features to predict vs 0 or ...
1
vote
0answers
14 views

How Gini importance works [duplicate]

I have searched for how feature importance in random forest works, but I could not understand. Can anyone explain how it works?. How random forest selects feature importance by Gini?
2
votes
1answer
25 views

Feature Selection for 50+ features

StackExchange newcomer here...I am looking for some advice on feature selection packages in R. Specifically, I am in search of functions that can identify the best features, out of 500+ features, ...
0
votes
0answers
10 views

How could a variable length binary string be encoded as an SVM feature?

I have data which is a binary string, e.g. 10001001 or 111100000001. The length can vary between 3 and 13 characters in length. It represents a pattern found in nature where the length is variable ...
0
votes
0answers
7 views

Suggestions on using model in production 1 test at a time

I have created an Artificial Neural Network with 4 categorical features and a binary outcome either 1 for suspicious or 0 for non-suspicious: ...
0
votes
0answers
22 views

Dimensionality reduction (feature selection) for multivariate time series data [on hold]

I have one dataset of multivariate time series data. Each sample has 60 variables and 500 time steps each. I don't want to use all 60 variables, but want to choose just few variables for my ...
1
vote
0answers
10 views

Input Variable Scaling Method Using Regression Coefficient

I am reading a paper with tite: Input Variable Scaling Method using Regression Coefficient. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=...
1
vote
0answers
29 views

Feature Selection and Neural Network [on hold]

Good Evening. I'm going to create a prediction model with response 0 1 with MLP and maybe Radial Basis Kernel NN. In order to predict 0 or 1 i will take something like 18 to 20 variable with 8000~9000 ...
1
vote
1answer
34 views

Leave insignificant features into logistic regression

I am making a model with logistic regression to predict binary sport events and have found some variables that may have an impact. Some features would however not contribute to the accuracy score of ...
0
votes
0answers
10 views

ANOVA, mutual information, Chi-squared feature selection are widely used and implemented in SKLearn - but where were they first published?

SKlearn is a widely-used, almost 'industry-standard' package for machine learning and implements a number of univariate feature selection methods (ANOVA, mutual information, Chi-squared) ... but the ...
0
votes
1answer
33 views

Random Forest Recursive Feature Elimination giving me different rankings

So I am trying to use RF recursive feature elimination to extract the most predictive features from my data-set. I've gotten the code to run fine and it gives me a nice table of rankings. However, ...
3
votes
1answer
39 views

Dimensionality reduction: Feature Selection or Feature Extraction?

Case: I have a dataset with a large number of features, which I want to reduce. Should I look for a method that identifies the most important ones and throw away the rest, or should I look for a ...
4
votes
1answer
56 views

Why lasso for feature selection?

Suppose I have a high-dimensional dataset and want to perform feature selection. One way is to train a model capable of identifying the most important features in this dataset and use this to throw ...
1
vote
0answers
19 views

Validating a formula on the relationships between the number of clusters and the maximum number of informative features

I invented a formula on the relationships between the number of clusters and the maximum number of informative features. I mean informative features that are necessary to achieve perfect degree of ...
0
votes
0answers
14 views

feature selection and classification - train and test on the sample?

I have a dataset of 93 records and 45 radiomics variables from various CT scans. I wanted to check if age and sex could be classified by the variables so I made a new variable with both sex and age. I ...
3
votes
1answer
39 views

Which model for feature importances?

When wanting to find which features are the most important in a dataset, most people use a linear model - in most cases an L1 regularized one (i.e. Lasso). However, tree based algorithms have their ...
1
vote
1answer
33 views

In machine learning how to find feature interdepencies?

Given a data set of N features, wherein some the features in this set were derived from other features from the same set, I am trying to discover inter dependencies between features (something like ...
0
votes
0answers
15 views

Which method should I use to extract feature/regress when I have multiple inputs and ones continuous, noisy output?

I come from a field working on physics signal processing and reconstruction. Our goal is to reconstruct a "peak" on a 1-D spectrum. We know that if the reconstruction is perfect the spectrum (i.e. ...
1
vote
2answers
31 views

Variables reduction required for Random Forest, Boosting, L1, L2 regularization

I have close to 10,000 variables. I know how random forest/XGB picks number of variables randomly for building the tree. Also regularization takes care of significance of variable by its coefficient. ...
0
votes
0answers
15 views

dummy variable in sklearn and feature importance

Imagine i got a mix of categorical and numerical features in a binary classification problem. I first transform my data to get dummy variable for the categorical data. I then use a random forest with ...
10
votes
4answers
384 views

What causes lasso to be unstable for feature selection?

In compressed sensing, there is a theorem guarantee that $$\text{argmin} \Vert c \Vert_1\\ \text{subject to } y = Xc $$ has a unique sparse solution $c$ (See appendix for more details). Is there a ...
0
votes
2answers
35 views

Methods for feature elimination of categorical variables?

I have a dataset with 56 columns, 4 numeric and the target variable which is also numeric. I am trying to eliminate some of the categorical variables from my model and wanted to get some understanding ...
0
votes
1answer
31 views

how to deal with categorical features (with distinct 10000+ values) other than conversion to one-hot encode and ordinal

Machine Learning Problem : I have a doubt in one of my feature which has an categorical value 1. One way of dealing with it would be like converting those values into numbers means in ordinal form. ...
0
votes
0answers
7 views

Best way to feature engineer dummy variables from other dataset on current dataset?

I have two datasets: Main dataset: consists of a lot of independent variables that are date variables and some categorical ones as well as a categorical dependent variable. Extra dataset: consists of ...
2
votes
0answers
37 views

Select 3 features from 4000 correlated features

I have been learning machine learning for only two months, so please pardon me and tell me how to imporve it if my question is phrased in a very naive way. The problem that I'm facing: There is a ...
0
votes
1answer
19 views

Should I remove co-varying factors before clustering?

I have a data set of around 850 factors representing 150 geographical areas. I am looking to cluster these geographical areas, and I am intending to use a K-means clustering algorithm to do this. My ...
-1
votes
0answers
28 views

Can I sample 500 out of 10,000 using bootstrap for variable selection?

I have a population dataset that has more than 10,000 subjects. When I used LASSO regression for variable/feature selection it did not work as almost everything was statistically significant because ...
3
votes
1answer
107 views

Feature Distribution in Cross-Validation

In the case of binary classification, stratified cross-validation only ensures that each fold contains roughly the same proportions of the two types of class labels. When does it make sense to also ...
2
votes
2answers
115 views

Feature selection using cross validation

I am dealing with a typical $p > n$ problem in the medical field. (typically $p \approx 3700$ and $n \approx 100$ ). The dependent variable is binary (healthy/sick) and features are continuous ...
2
votes
1answer
61 views

Feature filtering with LASSO and cross validation

In a linear regression problem, $y = (y_1, \cdots, y_{80})$ is the response, $X = (x_1, \cdots, x_{80})$ is a $4500 \times 80$ matrix of predictors. $k = (k_1, \cdots, k_{4500})$ is the vector of ...
1
vote
1answer
54 views

$\lambda \Vert k \Vert_0$ or $\Vert k \Vert_0 \leqslant n$

Say $Y \in \Bbb R^n$ is a response, $X = (x_1, x_2, \cdots, x_m)^T \in \Bbb R^{n \times m}$ are predictors. In a linear regression problem, we want to add an $l_0$ regularization for feature selection....
-1
votes
0answers
16 views

Influence of variable selection on standard errors

I know that variable selection reduces p-value and make them less correct (ie. increases type I error). I understand this happens because the selection step reduces the standard errors. Can you ...
2
votes
1answer
64 views

Tf-idf for text classification: On what should IDF be calculated?

The TF-IDF value of a word specifies how important a word for each document is. My setting is any text classification where one has multiple documents of with different classes: Let's take a lot of ...
0
votes
0answers
29 views

Feature Selection on Time Series of Panel Data

I am wondering if anyone can provide some clarity about feature selection with time series of panel data. Specifically I'm doing this with R. Lets say I have a time series of financial panel data ...
1
vote
0answers
11 views

Recognition accuracy decreases as the number of LDA features increase?

Helloy guys. I'm currently using KALDI building an HMM-GMM system and I added LDA transform to the original 12-dim features, wondering if I can get a better performance of the system. The input dim ...
1
vote
0answers
16 views

How to summarize score variable along time into a single variable?

I have a set of clients, and for each client I have a score (let's say between 1 and 6, where 1 is the best and 6 is the worst possible) given per month. For example: 01.2016: 2.25 02.2016: 1.88 ......
6
votes
0answers
112 views

Sparse linear regression 0-norm and 1-norm

We have a response $Y \in \Bbb R^n$ and predictors $X = (x_1, x_2, \cdots, x_m)^T \in \Bbb R^{n \times m}$ The problem we want to solve is $$\text{argmin}_{k \in \Bbb R^{m}} (\Vert Y - Xk \Vert_2^2 +...
1
vote
0answers
11 views

How to deal with random state affecting feature selection? (gradient-boosted trees)

I'm dealing with an imbalanced classification problem and I'd like to use feature importance from gradient boosting decision trees for recursive feature elimination in order to get rid of redundant ...
1
vote
2answers
33 views

Unsupervised learning with missing features

Assume I have a set of N vectors with M features each. If I want to create a latent space to project these vectors into, there are a variety of techniques available to me, such as Principle Component ...
5
votes
2answers
185 views

Regression as a way to determine variable importance

At my work, we employ a nearest neighbor algorithm to classify records. Part of this process, of course, includes determining which features to use as auxiliary information in the algorithm. Also, ...
0
votes
1answer
29 views

DNN methodology and feature concatenation

I'm using someone else's job and I have a question that I cannot solve. This work uses a DNN to match an electrical resistance to a bend angle. This is not very important, just for the context. So,...
0
votes
0answers
42 views

How to decide whether to use Ridge Regression/LASSO/Elastic Net or Random Forest for Feature Selection?

My understanding is rudimentary and high level but it seems like Ridge Regression/LASSO/Elastic Net would be better when the data is linear and Random Forest is better when the data is nonlinear? Also ...
1
vote
0answers
24 views

How to work with an unknown dependent variable? [closed]

I am working on a used cars price prediction project using the Craigslist data. I have the car price provided by owners/dealers but it doesn't mean they can sell the car at that price. Is there a good ...
0
votes
0answers
13 views

Assigning scalar values for PID for order in Neural Network

I have built a neural network using Windows Processes. I started off with only two features, the file path with parent process, and the file path with child process. I am slowly adding features for ...
0
votes
0answers
34 views

Variable selection using PCA in R [duplicate]

I need to preselect my environmental variables by a PCA before running RDA or CCA because the models are overparameterized. Please have a look at this publication if you ask whether or comment whether ...
-1
votes
1answer
16 views

How to weight features when doing text mining?

I have a case where I'm doing text mining over a list of product titles. In particular I want to run a clustering algorithm. But I also have some information about those products that I think can add ...
0
votes
0answers
22 views

Feature selection in the presence of extremely large feature set

Suppose you have a very large feature set (1000s to 1000,000 features) when building a machine learning model. How do you go about selecting the features? I know of the following methods for feature ...