Questions tagged [feature-engineering]

Feature engineering is the process of using domain knowledge of the data to create features for machine learning models. This tag is meant for both theoretical and practical questions regarding feature engineering, excluding questions asking for code, that would be off-topic on CrossValidated.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Is it possible to resize/add noise to a hidden layer during training?

I'm working on class for a project where my basic idea is to create a "drop-in" layer, so similar to drop out, I'm thinking of adding hidden units that are sampled from the same weight ...
0
votes
1answer
11 views

Avoiding Data Leakage from Bucketed Features During Cross-Validation

I am working on a classification problem and have engineered a few categorical features with high cardinality by dummying out the most frequently occuring values and then using the response variable ...
0
votes
0answers
26 views

What is the context between texture features and image anomaly detection?

I have been looking into deep image anomaly detection, more specifically the feature extraction component, for quite a while now. (My focus is on deep learning) I have encountered various papers about ...
2
votes
1answer
27 views

Overview Feature Extraction in images?

I have been searching for deep feature extraction approaches for a while now, but I did not find a single paper giving me a coarse overview on this matter. Apart from an overview, for example I would ...
0
votes
0answers
6 views

Feature engineering of closely related text

I am trying to do multi class classification of text. For many reasons I can't paste the data, atleast now in open. The problem is there is a text of closely related subjects like Anatomy and ...
0
votes
0answers
12 views

How to handle a date feature for demand forecasting

Imagine following use case. We want to predict the demand for food and drinks in a restaurant. After visualizing the target drinks in liter and ...
-1
votes
0answers
10 views

How to use PCA for feature extraction?

So what I have to do is given n images for training and m images for testing, I have to build a model for classification. To do that I need to extract features from the images. Now the person under ...
1
vote
1answer
23 views

Do typical NNs generate new features by applying some function to the input?

The toy network on playground.tensorflow.org has the option to generate new features by applying some function based on the input, e.g. with the inputs $X_1, X_2$ ...
0
votes
0answers
9 views

Quantify data leakage train test set for feature scaling

Topic: First appling feature scaling (e.g. standardisation) to a data set and then splitting the data set into train and test set, can introduce data leakage. Question: Although this is clear to me ...
0
votes
0answers
18 views

How should one transform a variable in which the further away from 0, the more significant it is?

Let's say you have a variable that ranges from -inf to +inf. The further you get away from 0, the more effect you think it has on the response. I am doing a logistic regression by the way. And ...
0
votes
0answers
8 views

Deep Learning feature extraction models

I want to build a deep learning model that extracts (texture) features out of images. In my research I found various kind of approaches, like using pretrained models or Variational Autoencoders for ...
0
votes
0answers
25 views

Feature importance in Random Forest and Decision Tree

First, thank you for time to read my question. I have a silly question that might need a little more insight into the algorithm behind the RF and DT. Why random forest and decision tree feature ...
1
vote
0answers
21 views

How to decide which features are important in this binary classification task?

Consider a binary classification problem, where the dataset is highly imbalanced, with only around 20% positive labels and 80% negative labels. Feature A has higher AuROC when considering all the data,...
0
votes
0answers
24 views

Deep Learning Approaches for Aggregating Data

I am looking for an automatic way of aggregating data to derive meaning features. For example, I have a table. It contains multiple customers. Each customer is associated with multiple loans. Each ...
0
votes
0answers
15 views

Feature engineering for spectral data

I am working on a model that takes 10 wavelengths (UV and NIR data) as input which measure the absorbances of a vegetable, to predict its nutritional density (protein, antioxidants, polyphenols) (...
0
votes
1answer
15 views

How can i cite Sequential Forward Feature Selection (SFFS)?

I've seen many papers/books about this technique but none cite its author. Is it ok to cite any machine learning theory book that explains it? Thanks.
3
votes
1answer
70 views

Up to what number of distinct values should I transform a categorical variable in a dummy variable?

When working with categorical variables, it's common to do some sort of transformation. Usually people apply a one-hot encoding. Putting it simply, we transform a categorical into a dummy variable. ...
0
votes
1answer
34 views

How to encode ordinal variables when null value is valid?

Tl;dr how can I encode a feature that has multiple distinct states each with different numbers of parameters. Am I going to have to break this into multiple models? I'm sorry, I'm sure this has been ...
0
votes
0answers
36 views

Can I change the rows / composition of a dataset without violating modeling assumptions?

I am building a few models to explain an outcome (let's say customer making a purchase during a visit to a site) from a dataset of ~200k rows with ~66 features (after dummying out categorical ...
1
vote
1answer
85 views

Is it possible to normalize features by one specific feature?

I have a dataset of genes with features that describe the genes at different scales (epigenetic, protein, cell, drug data etc. all numeric data). I use this dataset in supervised ML with a xgboost ...
2
votes
1answer
26 views

Ordinal Feature Encoding (Linear or Nonlinear?)

In most ordinal features, it seem that the scaling is linear. E.g. [1, 2, 3, 4] with higher score representing larger effect on the target variables But is it ...
0
votes
1answer
17 views

Word embeddings - Pre-trained tokenizers vs more involved methods

I'm drowning under all the various methods of converting my text corpora into embeddings. I'm currently using the HuggingFace Tokenizer (https://github.com/huggingface/tokenizers) to do this, using ...
1
vote
1answer
30 views

contaminate data with label then take it away

I have an idea for a training strategy (for an ML model), can you please tell me whether it has a name, and whether it makes sense. I need a model for binary classification with a massive class ...
0
votes
1answer
33 views

Improve XGBoost performance in a huge dataset with a lot of missing values

I have a dataset with around 250 features and 4 Millions samples and we obtained a model with Xgboost that has acceptable performance. The dataset has a high percentage of missing value, for an ...
2
votes
1answer
28 views

Is it possible to use a feature as a mediator of other features in machine learning?

I have a dataset of genes with features that describe the genes at different scales (epigenetic, protein, cell, drug data etc. all numeric data). I use this dataset in supervised ML with a xgboost ...
0
votes
0answers
17 views

RNN/LSTM Sequence Input Convert Into Single Value Input for Supervised Learning

Suppose Xij denotes feature j value for observation i. For examples, X11 = [3,2,1,2,2,5,10] is the sequence data for feature 1 and observation 1. The sequence data can be of numerical or encoded ...
0
votes
1answer
13 views

Which features based on orderbook information could be relevant for price prediction?

I have some orderbook data, including 5 ask prices, 5 bid prices, amount of asks and bids for every price, and midprice which is equal to (best bid + best ask)/ 2. I would like to predict absolute ...
2
votes
1answer
51 views

Feature engineering before or after scaling?

I am doing feature engineering on a set of features to reduce the size of the dataset. The features can have different scales. E.g, one feature has values that vary between 1000 and 1500, and the ...
1
vote
0answers
21 views

Modeling with Multiple Values per Variable, per Observation

I'm attempting to develop an autoencoder on top of medical claims data that have mutliple values per category because a claim often has multiple lines associated with it. For example, let's say (but ...
1
vote
0answers
21 views

Measuring magnitude of proportional difference between two items: cannibalisation and halo effects of products relative to promotions

I am trying to understand cannibalisation/halo effect between products when a promotion event occurs for one product and not another, this data is based in time, and the promotion status occurs at ...
6
votes
1answer
174 views

When should I do train test split?

I'm new to Machine Learning. I'm basically confused about when to perform train test split. Is the order given below correct? Split entire data into training and test set Extract Features from ...
4
votes
1answer
52 views

Should I add a new difference(z = x1-x2) feature into model?

Recently, I am thinking about this question:should I add new features based on raw features' differences? Setting Suppose I have 50k data and 20 features and it's a regression task. In data science ...
0
votes
1answer
22 views

Will Different Types of Labels Affect Feature Engineering Outcomes?

Here I would like to limit the question to 2 supervised learning tasks: classification and regression. My question is: for a given set of raw training features, will feature engineering be affected by ...
2
votes
1answer
30 views

Time series or features engineering?

I'm hesitating between these two techniques for business data (activity logs, purchases) for classification: I take all the data and consider it as a multidimensional time serie and use a deep ...
1
vote
1answer
96 views

Normalizing continuous features using sigmoid function

Can you use the sigmoid function to normalize continuous features that have no theoretical maximum value but tend to cluster around [-1, 1]? Although using the sigmoid function would be a non-linear ...
2
votes
2answers
42 views

Combining categorical and continuous features for neural networks

Is it OK to combine categorical and continuous features into the same vector for training deep neural networks? Say there is a categorical feature and continuous feature that I want to feed into a ...
1
vote
0answers
28 views

Similarity between two conditional discrete distribution

Having data of X:Y where X is a categorical feature and Y is multiclass label (i.e Sweet:Apple , Red: Apple, Sweet:Orange, Sugary: Banana ) I've created for each X the conditional frequency/ ...
0
votes
0answers
12 views

Combine 3 Features in Random Forest as 1 feature

I have a question about when it comes to Features in Random Forest. I will explain the example I am wondering about. I will put 3 sample below where the first column is the Hypothes and the 3 other ...
0
votes
0answers
13 views

Can a new independent variable be produced from the dependent variable in variable engineering?

I have a question. I have a data set, my goal here is to estimate player salaries. My variables are salary, number, year, error etc. I thought of creating a variable such as average salary by making ...
0
votes
0answers
14 views

Different features for different models

Is it considered 'fair' to use different input features when comparing forecasting accuracy between models? For example, adding feature x1, x2 and x3 into a random forest while adding x2, x3 and x4 ...
3
votes
1answer
79 views

Classification on Dataset with multiple rows per person

I have a data set where there are multiple rows per person. If person1 has 3 rows, out of 6 features, only couple of them change. The remaining features have the same values repeated. There are ...
1
vote
1answer
15 views

Engineering features that depend on more than one data point (classification with gradient boosting in particular)?

So I'm working on the Titanic data set (predicting survival of passengers), and would like to add a feature that indicates whether a given passenger's family survived or not (using the known training ...
1
vote
1answer
85 views

Can anybody understand this specification of a feature for a horses preference to a condition?

I have become interested statistical analysis of sports and came across a horse racing paper: "Computer Based Horse Race Handicapping and Wagering Systems: A Report" (found at: https://www....
0
votes
0answers
13 views

feature selection using RFECV vs trees?

the RFECV takes the estimator and finds which features are important according to that estimator. But, it gets slow when we are dealing with data with many features. But, if we use trees they give us ...
1
vote
1answer
27 views

feature hashing for high cardinality

when we are applying feature hashing in sklearn it asks us what should be the dimensions of feature required for us. If we decrease too much there will be more collisions which are not good. And we ...
1
vote
1answer
16 views

How to replace scalars with vectors in simple models, such as classification of sentences where 1-hot encoding is replaced with word vectors

I have a problem, which seems simple enough, but I don't know how it is solved in the industry. This has to do with the machinery of feeding data to a model, rather than trying to figure out the best ...
0
votes
0answers
5 views

Does changing reference variable of design matrix reduce multicollinearity?

It is stated here by Paul Allison that, if multicollinearity occurs within the design matrix of a categorical variable, multicollinearity can be reduced by setting the category with higher share of ...
0
votes
0answers
15 views

Month over month prediction for credit defaulters

I'm working on a month over month prediction using credit historical data. I've created more than 600 variables for this prediction including customer's delinquency in last 3/6/12/24 months etc. I've ...
1
vote
1answer
29 views

Scikit learn models gives weight to random variable? Should I remove features with less importance?

I do some feature selection by removing correlated variables and backwards elimination. However, after all that is done as a test I threw in a random variable, and then trained logistic regression, ...
1
vote
1answer
27 views

How to productionize a k-fold target-encoded feature?

I am attempting to build a model that has many predictors which are both categorical and large in cardinality. Target encoding looks to be a good solution for including these features, but I'm unsure ...

1
2 3 4 5
13