Questions tagged [feature-engineering]
Feature engineering is the process of using domain knowledge of the data to create features for machine learning models. This tag is meant for both theoretical and practical questions regarding feature engineering, excluding questions asking for code, that would be off-topic on CrossValidated.
751
questions
0
votes
0
answers
18
views
Replacing 10k+ geo features with just their key (zipcode coordinates) in a GBDT model - a sound idea? [closed]
A "new broom" in the modeling department has swept clean the existing 5-figure number of dictionary geo features (kept in a key/value store), replacing them with just their key (more ...
- 163
0
votes
0
answers
11
views
Feature to detect a reverse timeseries
I have a multiclass timeseries classification problem with 11 classes. The class 0 is my negative class, and the classes 1~10 are the positives and they are generated (using some equations) based on ...
- 137
0
votes
0
answers
9
views
Do similar PCA feature importance in first few top PCs mean these variables are nearly same in the original space?
I am using PCA to do the data inspection. First 3 PCs explain nearly 82% of the total variance. Suppose the number of features is $n$. And I found 4 of the features have similar PCA feature ...
- 11
2
votes
1
answer
44
views
Can variables used for rule based labeling be treated as input features?
I am currently working on binary classification problem with imbalanced dataset (n=3419 and 69:31). However, based on the business expertise of the users, they have generated rule-based label based on ...
- 2,522
0
votes
0
answers
52
views
Feature contribution interview question I can't answer
A few weeks ago, I had an interview for a data science job. Of all the questions they asked me, I was unable to solve the following one. I couldn't even attempt it because I didn't know anything about ...
- 53
1
vote
1
answer
761
views
Can we do sine , cosine , tan and cot transformation in regression?
Can we transform the variables of a regression in MLR to sine , cosine, tan
Then how to interpret the results if I get a good $R^2$ and good adjusted $R^2$
- 59
0
votes
0
answers
10
views
What are the ways to use a LIST of features which are DYNAMIC (contents) in nature?
Any features which is represented as a list of 0 or more elements is what I call a Dynamic feature.
Let us suppose an example where there are 10 Million movies and ...
- 244
0
votes
0
answers
6
views
How to analyse temporal influences of Features
Let's say we have yearly end of year ratings (y) and their belonging features (X).
If you had to analyse temporal dependencies/influences of some of those features, how would you do that? What are ...
- 1
5
votes
2
answers
332
views
Options for 3D coordinate systems?
I'm trying to solve biochemistry problems (think protein folding) with DNNs. Are there 2D / 3D coordinate systems that are particularly well suited for deep neural networks (DNNs) to process?
For ...
- 251
2
votes
3
answers
205
views
What are the methods to increase the dimension of a feature space?
Is there a way to increase the number of dimensions through feature transformation in machine learning? If so what are the techniques involved?
0
votes
0
answers
27
views
When does target encoding lead to overfitting
Let us say, we are tasked with setting (average/list) prices that are likely to convert for heterogeneous products (e.g. used cars of all shape and sizes - made up example!). Let us also say that we ...
- 1,947
0
votes
0
answers
17
views
Why does making new features increase the validation accuracy but decrease the test accuracy?
I am working on a competition on kaggle. The competition is a classification problem. I tried to extract 2 new features(engineer features) from the data.
The accuracy on the validation data increased ...
- 1,350
1
vote
0
answers
19
views
Encoding ordinal categories as features
When we have categorical features in a regression (say a generalized linear model for now), it is typical to let one category be subsumed by the intercept and then code binary indicator variables for ...
- 51.2k
1
vote
0
answers
22
views
Should I destandardize the errors from training the neural network?
So, I am learning a bit about Neural Networks. I have built a code in PyTorch for regression, and I have standardized both the features and the target variables following this answer. My question is ...
- 111
0
votes
0
answers
6
views
How to calculate mouse movement features like speed and acceleration according to the description in paper?
I am trying to recreate the results of the paper Intrusion detection using mouse dynamics. The dataset looks like this:
I am trying to calculate the features according to how they mentioned it in the ...
- 193
0
votes
0
answers
11
views
What are usually the most strongly predictive engineered features used in sentiment analysis?
I'm curious which engineered features do data scientists generally employ in sentiment analysis? For example, I would think some of the most strongly predictive features would include:
Number of ...
- 641
0
votes
0
answers
16
views
FFT: peak spectral power vs peak frequency
I am having my head confused about this paper I am reading, where they used FFT to create frequency domain features.
Specifically, they calculated ...
- 121
0
votes
1
answer
61
views
How to drop one of any two highly correlated features having low correlation with target
I am working with the breast cancer dataset included in the scikit-learn's package, loaded like so:
...
- 121
1
vote
1
answer
108
views
Why data-scaling in range (0,1) is important? [duplicate]
Often a preprocessing technique to do is to normalize our data in a range (0,1) before we tow our model (example neural network) on them.
I understand why in a practical way (for example if we have ...
- 11
0
votes
0
answers
10
views
Recursive Feature Elimination selects different features when I reorder the columns
Why Recursive feature Elimination is selecting different features when I reorder/shuffle the columns? These selected features is always giving me different prediction results.
What could be the issue ...
- 1
2
votes
0
answers
34
views
Does it make sense to transform a feature containing hours (24h) into two features with xy-coordinates of each hour in the space? [duplicate]
I have a clustering problem that I might solve with an algorithm based on Euclidean distance (e.g. K-Means).
One potential feature is the "hour" at which each user began an interaction.
As ...
- 354
0
votes
0
answers
29
views
How to model categorical variables with word frequency vectors in a decision tree?
I have a dataset that describes car failures and the action made by the mechanic to fix them.
It is composed by 5 columns:
Fault Code, depending on car model and car year, categorical variable that ...
- 111
1
vote
0
answers
46
views
Why cannot I use silhouette score with ground truth labels?
I was looking into checking cluster positioning from a non-liner transformation. I do have the ground truth labels. Now, I want to use the transformed data points and see how good this transformation ...
- 111
0
votes
0
answers
22
views
Continuous and discrete input data types for neural network
I have a dataset where the input is comprised from both continuous and discrete variables:
...
0
votes
0
answers
68
views
Feature store for panel / time series data
This is more a technical question on design for a feature store, however I wanted to post this here rather than on a general tech forum because I'm looking for something more specific and convenient ...
- 1
1
vote
1
answer
43
views
Why is not a scalar feature enough to encode 3-component binary numbers in an autoencoder?
I am trying to build an intuition on what really a feature is. I created a toy example as following. In my mind a scalar feature should be enough to represent my data. Couldn't the model in this case ...
- 31
1
vote
1
answer
48
views
What is the best way of creating new features in a dataset?
I recently started working with sklearn, and found myself creating new features often (new features with K Bins, with various Encoders etc.).
What I noticed though, is that is very difficult to ...
- 11
2
votes
1
answer
23
views
Curve quantification
I have some longitudinal measurement data of 15,000. I smoothed that data with B-spline smoothing and got the following curve.
I then want to quantify this curve and extract features for clustering ...
- 21
1
vote
1
answer
83
views
How to encode categorical variable with multiple categories per datapoint?
Consider this question on a survey:
What desserts have you eaten?
Apple pie
Banana pudding
Coconut cake
Doughnut holes
The user can pick as many of the options as they like. How would one encode ...
- 137
1
vote
1
answer
25
views
Having to use features which have low correlation with the target
I'm applying LogisticRegression on breastcancer dataset.
Steps : -
1- A correlation matrix resulted in only four features having ...
- 13
4
votes
1
answer
963
views
Do we One Hot Encode (create Dummy Variables) before or after Train/Test Split?
I've seen quite a lot of conflicting views on if one-hot encoding (dummy variable creation) should be done before/after the training/test split.
Responses seem to state that one-hot encoding before ...
- 143
0
votes
1
answer
104
views
Feature engineering for anomaly detection when feature is directional
I want to train an anomaly detection model for intrusion and fraud detection. I have several features I know are correlated with sketchy behavior. However, those features are "directional" ...
- 1
0
votes
0
answers
39
views
Can we give higher priority certain features in machine learning models?
I am preparing a model for predicting the customer propensity to buy from our company. We are using the transaction data for modelling. On discussion with our sales team, we realized that some ...
- 213
0
votes
0
answers
13
views
Create synthetic variables where each cluster member has a specific minimum correlation to the synthetic variable
I'd like to reduce dataset by creating synthetic variables that correlate highly with existing variables. This is in the context of a metabolomics dataset where many of the variables are very similar ...
- 345
0
votes
0
answers
23
views
Adstock transform in non advertising varibles
In Marketing Media Mix models, adstock transformations are commonly seen in advertising variables. However, in other domains where the regression tool is intended to estimate the effect of variables ...
- 75
0
votes
1
answer
763
views
How to return feature_names_out with sklearn.preprocessing.FunctionTransformer? [closed]
My goal is to impute not with sklearn.impute.SimpleImputer. My goal is to impute with sklearn.preprocessing.FunctionTransformer. ...
0
votes
0
answers
18
views
Is there any approach to find out the association between the arithmetic-combination of variables to other single variable?
I have 30 set of data (each set has been recorded from one subject) that contain of a single gold-standard measurement value (recorded by standard medical device) and another 200 numerical values that ...
- 41
0
votes
0
answers
9
views
Using cumulated historic predictors in cross-validated models
Say we have a very simple dataset with columns y and time of occurence t. We want to predict ...
2
votes
1
answer
37
views
Using $predicted values of a randomForest object as predictor in training for another randomForest
I'm wondering if there is any problem in using the predicted values (based on OOB observations) of a randomForest object as a predictor in the prediction of another variable. Something like this using ...
0
votes
0
answers
38
views
chi square and Mutual information feature selection for numerical features
just to make my question clearer
suppose that I have patient dataset as follows the first column is the label which indicates if a patient is infected does have the disease or not , othe columns are ...
1
vote
1
answer
17
views
Training models with estimated features
A model (model X) is being trained with a set of features. One of the features (say, feature A) is not available on test data. But it can be accurately estimated using another model (model Y) using ...
- 316
1
vote
0
answers
64
views
Nominal and cyclical features encoding should be done before or after feature selection?
In a dataset I'm using to learn machine learning I have beside many others, one nominal and two cyclical features like bellow:
Location: "Orlando", "NewYork", "LosAngeles&...
1
vote
1
answer
260
views
Machine Learning for Time Series: Train and test set overlap due to lagged target as feature – problem of data leakage?
Situation:
My objective is to apply Machine Learning (for regression problems). Therefore, I have a panel dataset of time series with daily fund data from 2018-01-01...
- 11
1
vote
0
answers
33
views
How to handle features that rely on a category? where each category has a different set of features
Dataset description
My dataset features are:
some features not important for this question
Price (target)
Collection (categorical feature, there are 1.8k collections)
latest 10 prices (time-series, ...
- 11
0
votes
1
answer
114
views
Working with vector representation of individual feature
Generally, training data is of shape $\mathbb{R}^{n \times m}$ data where $n$ is the number of samples and $m$ is the number of features. The training data can be represented in a table such as
feat1
...
- 263
1
vote
0
answers
51
views
Distributions as Features in Machine Learning
The Problem
Let's assume I have a problem that seems perfect for supervised learning. However, some of the measurements I would like to use as features are not point estimates but are instead ...
- 153
0
votes
0
answers
8
views
Analyzing distribution of vector samples
I have a neural network that can be well trained with CNN features from AlexNet (a neural network that is pretrained for image recognition).
I have obtained these CNN features from previous studies ...
- 395
0
votes
0
answers
7
views
Different training curve for different feature types
I have a fMRI dataset in which some participants watch images and we have the corresponding brain activities.
If an image was "jungle", the corresponding label is either the word2vec ...
- 395
0
votes
0
answers
13
views
Time series feature enginnering, order of differencing, scaling and distribution transformation
Assume a number of operations should be performed on a time series before it's fed to a linear forecast model.
Differencing, for stationarity
PowerTransformation or logarithmic transformation for a ...
- 223
1
vote
0
answers
16
views
How to engineer and test seasonality features?
My task is multiclass classification of item to buy (next).
I have a purchase history dataset with a datetime feature.
From it I could engineer many new seasonality features:
Time of year (season, ...
- 11