Questions tagged [feature-engineering]

Feature engineering is the process of using domain knowledge of the data to create features for machine learning models. This tag is meant for both theoretical and practical questions regarding feature engineering, excluding questions asking for code, that would be off-topic on CrossValidated.

Filter by
Sorted by
Tagged with
1
vote
1answer
230 views

Is feature crossing actually useful in deep learning, which uses activation functions?

So i know feature crossing is a way to transform the data such that it can be linearly separated, which makes it useful for things like classification. But in a DNN, activation functions replicate non-...
2
votes
2answers
45 views

Encoding ID variables for machine learning

Is it possible to encode ID variables for an ML model in a way that the model can work with IDs not seen before? For example, I want to predict the sentiment of a tweet, the account it may be ...
0
votes
0answers
11 views

What is the order when doing feature engineering? (imputation, encoding, etc.)

I am self learning machine learning right now, and I am confused with what should I do first. Should I impute the missing value before encoding the categorical variable? Also, I am learning from ...
1
vote
0answers
12 views

How to model a problem where the aim is to predict if atleast one of 3 inputs is “different” from the rest?

I have a tabular dataset with 10000 rows, each with a RowID and 30 numerical features. There are multiple rows with the same RowID. The aim is to come up with a model, which at test time would take 3 ...
2
votes
1answer
28 views

Dealing with text column of thousands different values

I have this dataset with some numerical and some text columns and want to create an ML forecasting model. The thing is that one column called 'diagnosis' is text (each entry is one sentence long) and ...
1
vote
0answers
13 views

How to calculate the coefficients on the original variables after engineering an interaction variable that is uncorrelated to the original variables?

$$x = (0,1,0,1)$$ $$y = (0,0,1,1)$$ $$z = x*y = (0,0,0,1)$$ $$r = \beta_0 + \beta_1x + \beta_2y + \beta_3z + \epsilon$$ Each $\epsilon_i$ is some error term with zero expected value. Alternatively, we ...
11
votes
2answers
642 views

URL Feature representations

I'm writing a bookmark classifier as a hobby/learning project. Currently I'm trying to decide on a feature representation. I have two pieces of information: The name (meta title attribute) The URL I'...
0
votes
1answer
26 views

Run-length encoding on the input sequence of a RNN

I have a dataset consisting of sequences of item embeddings a, b, c, ..., in which the length of consecutive runs of an item may be large (comparative to the ...
3
votes
1answer
35 views

When Should we apply the Aggregations(mean, sum , count, max, min) and how to deal with correlations features

I am beginners to machine learning , I worked on the some basics machine learning problems. I had a little bit confusion about feature Engineering. some people are using aggregations functions for ...
1
vote
0answers
34 views

What kind of machine learning models fit data of this kind?

I have been working with a manufacturing process. It would be very efficient to build a machine learning model for the kind of data that I have. So, my dataset has typically three inputs. VAl_1 VAl_2 ...
0
votes
0answers
20 views

Classification Problem using multiclass features input and ensemble methods

I am working on a classification problem. I am applying tree-ensemble methods (Histogram-Based Gradient Boosting and Random Forest) and evaluating premutation importance in order to understand ...
0
votes
1answer
45 views

Why won't my xgBoost regressor predict above a certain value, even though it sees higher in training?

I have created my first xgboost regressor. I input some self collected data, which I scale using sklearn's standardScaler. The model is trained on approximately 25,000 samples, and then tested on ...
0
votes
1answer
33 views

Can ML or DL model automatically pick up “difference” feature?

ML requires manual feature extraction whereas DL doesn't necessarily require feature engineering, since recent advanced models like transformers learn necessary features automatically during training -...
0
votes
0answers
14 views

General strategies to flag distribution shift in live text systems

Does someone know some basic strategies for flagging distributional shift in a live system, particularly for text systems? Many of the solutions online revolve around correcting for this; however, I ...
0
votes
0answers
14 views

How can I recognizing the reduced features after applying PCA?

I am working on the "global index of economic freedom" data which has 26 exploratory variables. I am trying to reduce the features using PCA and I am able to recognize that the first 5 PCs ...
1
vote
0answers
25 views

Is it reasonable to do Feature Engineering before Data Preprocessing? [closed]

I want to do data preprocessing with scikit learn and create a pipeline, among other things to avoid data leakage and streamline the process. The problem though is that after fitting the data to the ...
0
votes
0answers
28 views

Does this problem requires Supervised Learning or Unsupervised Learning

I have 50 Features in a Dataset to predict 1 Variable "Units Sold". I am currently using XGBoost model (Supervised Learning) to train all these 50 Features and the accuracy of the model on ...
0
votes
0answers
7 views

Using other predictions on same subject in data set as a feature to improve accuracy

I have a problem I'm working on which is similar in structure to this example. Say I'm predicting the score a student received on an exam based on a collection of their social media activity and the ...
0
votes
0answers
13 views

Decision tree as function

How should i formalize regression tree as $f: C \rightarrow R$ ? Where C is any feature space (not necessarily $\Re^p$ where $p$ is a dimension of an element in $C$)
0
votes
0answers
27 views

Multivariate Time Series PCA

I have many multivariate time series each with similar time length window and variables recorded. Example: one time series plot one time series Data: I am able to extract prin. components for ...
1
vote
1answer
46 views

Does it make sense to engineer new variables and keep the original variables?

While doing some feature engineering on a dataset, I recently thought: When I create new features, should I keep the original ones? Let me specify my questions a little bit more and give you an ...
0
votes
0answers
16 views

Is it a good practice to pad signal before feature extraction?

I have a question for you - is padding, before feature extraction with VGGish, a good practice? Our padding technique is to find the longest signal (which is loaded .wav signal) and then in every ...
2
votes
2answers
48 views

xgboost with Skewed data [closed]

I am attempting to run xgboost on response data ( Whether a product is sold or not).However, I have a feature 'number of employees' which is highly skewed. I'm thinking of convert this continuous ...
0
votes
0answers
18 views

What features can be extracted from a probability distribution and how does the features change from baseline to different category/cases?

I have been looking online regarding feature extraction and I am looking at extracting features from probability distribution in order to understand the characteristics of the distribution. I know the ...
0
votes
0answers
21 views

Why would random forest do better with one hot encoded categories?

I have some hand coded feature which is a category with values "High", "Low", and "Normal". I created this feature myself and my problem performance (classification) ...
0
votes
0answers
15 views

Feature Selection with Potentially “Useless” Features

Suppose in a regression setting, we have many features and potentially many of them have nothing to do with the target. We use Lasso and various regularization parameters and perform cross validation ...
0
votes
0answers
10 views

How can I improve accuracy in classifying 6 common interactions between two people in still images?

My project’s goal is to classify 6 common interactions between two people in images including Boxing, Facing, Handholding, Handshaking, Hugging, and Kissing. • Dataset (6 categories correspoding ...
0
votes
0answers
116 views

Feature engineering for fraud detection with Isolation Forest

I am researching and doing a project related to the detection of fraudulent transactions in the financial system. For this research we are working with unsupervised learning, more precisely we are ...
0
votes
0answers
12 views

Extract vector encoding from supervised LSTM

I want to use historical stock price data for some companies in the S&P 500 to predict future prices of the stocks and index as a whole. I'm taking the approach in this paper https://arxiv.org/pdf/...
3
votes
1answer
36 views

np.log() vs StandardScaler() in preprocessing of dataset variables

I'm anew to DS and now I'm passing this introductury Course on Kaggle. I'm trying to catch the logic behind this exercise, introduction. Particularly the part of data transformations is unclear: ...
0
votes
0answers
41 views

When to apply target encoding ( before or after target transform)?

I am trying to build a linear regression model. I have some high cardinal categorical features on which I want to apply target encoding. But my target (real-valued) variable distribution is highly ...
0
votes
0answers
27 views

Scaling method before PCA with non normally distributed data

I have a dataset with 314 examples and 63 features, all numerical. Of the 63 features, 23 have a large number of zeros (>=80%) and all the features are right-skewed. To try to solve the right-...
0
votes
0answers
6 views

Implementing nested features in unsupervised models

Our project has built an unsupervised model that uses data about a number of companies. Some of these companies are public and some are private. The ones that are public have much higher financial ...
0
votes
0answers
10 views

Can a dependent variable be a feature for another dependent variable?

I am working at feature selection for dependent variable of Market Capitalization which I narrowed down to 3 independent variables Dividend Yields, Price to Earnings and Price to Book ratios. However, ...
0
votes
0answers
7 views

Patch wise feature vector comparison

I have a image of size of 64*64. I am trying to compute HOG features for the image. I have skimage for my implementation, with the following parameters: ...
0
votes
0answers
24 views

Interpreting LASSO regression coefficients versus regression equivalent

I am using R to perform a Penalized Logistic LASSO regression to identify predictive features of an online survey (n=7000). After performing feature engineering on the survey data (~300 Boolean ...
1
vote
1answer
8 views

How to merge/encode a categorical feature's unique values in a regression problem

A feature contains more than 10 unique values in my case, and I want to merge them to improve my model speed. The problem is I don't know how to merge them in a scientific way. Now, my idea is to ...
2
votes
1answer
23 views

Dice Distance returning nan. Workaround?

Starting Point: I want to calculate the distances between nominal data. Furthermore, I want to see what difference it makes to only use important features (given some feature selection method). So I ...
1
vote
0answers
14 views

Resiudals of linear regression between two correlated explanatory variables as a new explenatory variable

I'm starting a new data science project with few explanatory variables, so I'm trying to keep as much variability as possible. Also I have a lot of response variables, so don't really now what I want ...
0
votes
0answers
15 views

Feature Importance for each instance (point) for an image classification system by unsing CNN

I have developed a simple binary classification system (true, false) by using convolutional neural networks in keras, My input image are color images with the shape of 100,100,3. I am just wondering ...
1
vote
0answers
16 views

Is it valid to use different scaling techniques for different features in a dataset?

I am currently working with a dataset that has a few different features. Ultimately, I would like to train a binary classification model on this dataset. I would like to scale my data, as I plan on ...
1
vote
0answers
31 views

Confusion on optimal lagged time and other associated parameters to use for attrition forecasting

I am working on employee attrition prediction with raw data that has more than 1 year's amounts of daily metrics, and have some confusion on the best data aggregation techniques/methods for a ...
0
votes
0answers
11 views

How to explain an important feature of Decision Tree? [duplicate]

I need to do storytelling about which factor drives an athlete to choose full Ironman race. Right now, let's assume Decision Tree performs well so my strategy is to consider the most important feature ...
0
votes
0answers
15 views

Handling time data

I'm trying to solve a classification problem that has records of customers of a company seeking some service. It has different attributes and one of the attributes is the time the customer has been ...
0
votes
0answers
3 views

Creating Interaction Features from Categorical and Continous Features

I've been looking but can't find anything that address creating interactlon features between categorical and continous features or between categorical features in machine learning models. Any guidance ...
0
votes
0answers
12 views

Latent space for cross domain features

I would like to find the shared latent space between two set of features. I have source and target domain features already extracted from images. I have 4 set of feature vectors for normal and ...
0
votes
0answers
3 views

adjusting house hold income using equivalence scales for regression

I have family/household data. My target is money_spend_on_food_in, which I would like to predict depending on certain properties of the household. One of these properties is number_of_inhabitants. I ...
2
votes
1answer
21 views

Need to do hyperparamter tuning for new features?

Suppose I have a set of features(say 100 features) and spent a lot of time doing hyperparameter tuning to get a good model. Now I have a few new features(say less than 5 new features) added into the ...
2
votes
1answer
56 views

Should we apply feature transformation for test data?

I am working on a regression problem. The data contains 13 features (after performing feature selection). to some of these features, I have applied log transformation and box-cox to fix the skewness. ...
0
votes
0answers
32 views

How do shared weight vectors work for CRF?

I am going through two materials regarding Conditional Random Fields. The first one is this (referred to as [1]) material by Charles Sutton and Andrew McCallum and the second one is this (referred to ...

1
2 3 4 5
13