Questions tagged [feature-engineering]

Feature engineering is the process of using domain knowledge of the data to create features for machine learning models. This tag is meant for both theoretical and practical questions regarding feature engineering, excluding questions asking for code, that would be off-topic on CrossValidated.

Filter by
Sorted by
Tagged with
1 vote
1 answer
19 views

How to forecast a Macro trend by multiple Index time-series Using LSTM Model?

I am new in machine learning and I found that lots of article only train the LSTM model by only one stock and do the forecast. ...
user avatar
  • 113
1 vote
0 answers
53 views

Without encoding, how can we solve high cardinality issue?

I already referred the posts here but this question is different. I don't wish to use categorical encoding. details given below I have a dataset of 3000 unique customers purchase data. The dataset ...
user avatar
  • 1,826
0 votes
0 answers
9 views

How to convert multiple rows into single rows in Python for prediction for next t days?

I have time-series data. I have taken the dataset from Kaggle [https://www.kaggle.com/code/kp4920/s-p-500-stock-data-time-series-analysis/comments]. So, how can I bring multiple rows into single rows ...
user avatar
0 votes
0 answers
13 views

PCA for Feature Engineering

PCA is finding features with high variance features and transforms (Gives more appropriate/variance captured features) For feature engineering, we tend to create features like rations and other ...
user avatar
0 votes
0 answers
25 views

Help with generalizing a formula for historical fencing matches factors

I am writing a small application to aggregate Historical Fencing matches results. This data is then used to calculate a couple of factors: Effectiveness: simply shows how many of your matches are won:...
user avatar
1 vote
0 answers
13 views

Encoding Geolocation data

I am working on routing bus from one stop to another, for which Geolocation data inform of latitude and longitude is required. In addition to xy coordinates, distance matrix of locations is also ...
user avatar
4 votes
1 answer
155 views

Can RandomForest multiplicatively combine features?

I have a relatively good understanding of how RandomForest mechanically works. However, here's what I want to understand: can RF model a multiplicative relationship? For example, if I have features A ...
user avatar
0 votes
0 answers
25 views

how to deal with features in pairwaise comparison models?

I am working on a dataset of ATP (Association of Tennis Professionals - men only) tennis games over several years. I want to predict the outcome of tennis so one way to do that is using a Bradley-...
user avatar
0 votes
0 answers
9 views

Searching for combination of feature-comparisons to optimize a metric on a subset of the data?

Suppose that I have a dataset with n features X1, ..., Xn, and label Y (consider it binary for now). Features can be constructed with "meta"-features by comparisons (>,<, >=, <=),...
user avatar
2 votes
1 answer
33 views

Creating a holdout set just for feature engineering?

I recently encountered a feature engineering technique that I haven't seen before: Create the usual training, validation, and test sets. Create another set by splitting the train set; call this the &...
user avatar
  • 11.5k
3 votes
1 answer
53 views

How to incorporate predictor variable without future information into a model

I will use an extremely simplified example to ilustrate the question, but I think the answer shsould hold for more generalised cases. Let's say I want to create a time series regression model (the ...
user avatar
  • 31
2 votes
1 answer
12 views

Extract valuable information from a list of dates for Machine Learning

I have to create a model to predict if a patient has a disease or not based on pharmaceutical prescription data. I created a neural network and a random forest using as features the number of ...
user avatar
1 vote
1 answer
25 views

Timeseries forecassting (Load forecasting) - Apparent shift in actual vs predicted values when applying regression model

Tools/languages/techniques I am using python scikit-learn different regression models (only linear regression is shown here for simplicity) I am working on a regression problem. The data I have is ...
user avatar
1 vote
0 answers
21 views

How to analyze ranked data with multiple variables

I have data points with 10 variable values each, each belonging to one of two groups (A and B) that look like this: Group A: datap1 = [0.2,0.4,0.7,...,1.4] datap2 = [0.1,0.5,0.9,...,1.9] datap3 = [1....
user avatar
  • 11
0 votes
0 answers
11 views

feature engineer 2 numeric features with odd values

I am not yet sure what type of regression model I will eventually use but I wonder what method(s) exist to leave 2 numeric features as numeric, when I know from some explorative data analysis that the ...
user avatar
  • 1,326
1 vote
0 answers
13 views

Empirical distribution for feature binning

In paper "A simple yet effective baseline for non-attributed graph classification" (https://arxiv.org/pdf/1811.03508.pdf) authors use empirical distribution for feature binning. Precisely, ...
user avatar
  • 179
0 votes
1 answer
33 views

Featuring Enginnering: High cardinality

I was reviewing a ML notebook when part of the EDA looks at the cardinality of categorical variables. As the notebook was prepared there was no strange result, but what if an attribute has a very high ...
user avatar
0 votes
0 answers
13 views

Are there any statistical principles that a neural network layer should obey?

In the neural networks (NNs), we have different layers. Probably the best resource on finding zoo of different layers is the Keras Layer API. Some of them used as only passing and reshaping. Most of ...
user avatar
  • 1,858
1 vote
1 answer
19 views

Feature with multiple attributes/properties

I want to build a machine learning model where each feature has further multiple attributes. Apologies for the lame example, but this will convey my doubt: Predict the animal on the basis of its ...
user avatar
  • 113
4 votes
2 answers
302 views

Target enconding in test data and target leakage

I understand target encoding, which is the average of the target value by category using out-of-fold mean within each fold. although you get slightly different means for the same value of a ...
user avatar
  • 243
1 vote
1 answer
26 views

When to Normalize Training Data

As I have seen so far, it seems like choices such as whether or not to normalize your training data are made based on the results you get after evaluating your model on the test data. Is there a more ...
user avatar
  • 111
0 votes
1 answer
14 views

How to find the expected mean of a very smalll subgroup, while still considering the larger group?

Context: For a Machine Learning challenge, I have a national exam dataset, containing over 3 million scores, from over 5k cities (unbalanced distribution). For example: ID City Other Score 01 NY ... ...
user avatar
1 vote
0 answers
49 views

Can I standardize data twice? Creating a composite covariate in regression

I am running a regression analyses with several covariates, and my goal is to examine the relative influence of each covariate on the dependent variable. Therefore, I have taken the approach of ...
user avatar
0 votes
0 answers
10 views

Feature Engineering for Meta-Learning?

I'm doing some stacked generalization/meta-learning. In blogs and posts, I have only seen people take the level 1 predictions and just directly use them as features for a level 2 model (no feature ...
user avatar
1 vote
0 answers
18 views

Best way to represent size classes features in neural network

I have features representing a some specific size category, where a data sample has assigned one out of 110 possible size classes. What it is important that these classes are sorted from smallest to ...
user avatar
0 votes
0 answers
17 views

Best statistical/ ML approach to combine numerically simulated outcome and unstructured data?

I have been assigned to a research project in collaboration with a manufacturing company who are looking to improve the accuracy of their model. Their research question is to improve their Finite ...
user avatar
0 votes
1 answer
20 views

No change in algorithm performance following removal of important variable

performed a classification task with XGBoost where I aim to predict cardiovascular disease (CVD) with a dataset of 12 vars and 70 0000 data points and got an f1 score of 0.73. After obtaining a ...
user avatar
  • 1
0 votes
0 answers
5 views

Handling extra channel axes

Most architectures handle inputs as (batch_size, channels, *spatial), where e.g. spatial = (H, W). However, what if we want an ...
user avatar
2 votes
2 answers
56 views

Generating interesting/useful features from existing useless features

I am new to ML and trying to work on a binary classification problem. I came to know that one of the main factors for success of ML is feature engineering. I am here to seek some inspiration/help from ...
user avatar
  • 1,826
0 votes
1 answer
34 views

Lag based numerical features or ID categorical variable?

I have to develop a Machine Learning regression model to predict customer’s delay in paying invoices. In addition to the invoice related variables, of course a very important variable is the customer. ...
user avatar
  • 85
0 votes
1 answer
67 views

normalizing and scaling are different?

This is the original data histogram, I have a data set and plot by DataFrame.hist(): After that I applied the zscore function to my data set and plot this histogram: After I have applied zscore, I ...
user avatar
1 vote
1 answer
45 views

Feature selection for uncorrelated dataset

I am working on a speech emotion recognition problem and my training dataset consists of about $4000$ points of $138$ features each. The highest (Pearson) correlation among the features is $0.3$ and ...
user avatar
  • 113
1 vote
0 answers
25 views

How to create/design a Hidden Markov Model?

I have a rough conceptual understanding of what Hidden Markov Models do. What I don't understand is how to really create/train one. Let me outline what I'm working on, and then I'll give more specific ...
user avatar
0 votes
0 answers
14 views

How to handle different distribution of feature column in below case?

I am working with a tabular dataset of around 20k entries for training(mixed with categorical and numeric features) and around 2k entries to predict. After cleaning, I wanted to keep just about 2% of ...
user avatar
1 vote
0 answers
39 views

Does the mentioned approach to train a 100 GB dataset right way to go about it?

I have a 100 GB dataset and I want to train an ML model on it. I have 5 versions of this dataset each containing a different set of engineered features. I want to find out which version of the dataset ...
user avatar
  • 21
1 vote
1 answer
28 views

Feature function for logistic regression

In his Natural Language Processing textbook, Eisenstein defines the logistic regression as follows: $$p(y|x,\theta) = \frac{\exp (\theta f(x,y))}{\sum_{y'} \exp (\theta f(x,y'))}$$ Where $\theta$ are ...
user avatar
  • 113
1 vote
1 answer
30 views

How does neural network learn feature functions?

I was wondering if there is some intuitive way of thinking about the fact that neural networks learn the feature function $\phi(x_i)$ along with the feature weights $\theta$. Because this is really ...
user avatar
  • 113
0 votes
0 answers
43 views

Why does this ensemble model score worse, when I add features via recipe function?

One of my costumers wants to get some feature engineering done in the near future. As I am using recursive ensembles from modeltime, I need to add some additional features via recipes, as ...
user avatar
0 votes
1 answer
31 views

How to set prior for covariate coefficients in Bayesian regression

I have a data set of around 1 million rows and around 30 possible features. My main objective is to build a classification model to predict probabilities for an output variable of interest. It is ...
user avatar
6 votes
0 answers
102 views

Why convert spectrogram to RGB for machine learning?

I've seen a few publications that feed an RGB image of a spectrogram to a neural net, and someone claiming a network does better with RGB than grayscale. A spectrogram is fundamentally a 2D ...
user avatar
1 vote
1 answer
40 views

Does feature selection always benefit the performance of a ML model?

In my ML pipeline, I normally perform feature selection, by performing a few of the tests mentioned below, the ones relevant to the model. I tend to drop features with negative outlier to the rest of ...
user avatar
0 votes
0 answers
37 views

Feature engineering zero-inflated distributions for Neural Networks

I am trying to fit a neural network on a set of various features. For most of the features I've followed some common rule of thumbs, so as to get nicely distributed features. Not so imbalanced binary ...
user avatar
  • 1,388
1 vote
2 answers
54 views

Feature selection- feature that most of its values are equal. How can I know if to drop it?

If I have a feature that most of its values are the same, how can I know if to drop it or to keep it? First of all, most of the model's features have low variance. Secondly, maybe the observations ...
user avatar
  • 67
2 votes
0 answers
36 views

How to deal with output transformation at inference/prediction time?

Suppose A machine learning model (e.g. RandomForest) which uses $x$ as input and produces $y$. Now as part of preprocessing and feature engineering, I applied some ...
user avatar
2 votes
1 answer
76 views

Common feature engineering techniques for time series data and streaming data

I'm trying to use ML algorithm to do classification on time series data and streaming data. Although I'm able to find certain ML algorithms applicable to such data, such as dynamic time warping, I ...
user avatar
  • 1,643
1 vote
0 answers
20 views

Feature engineering a license plate price data

Target = price. Suppose I have license plates in the form {letter}-{number}. For example, 'A-12345', or 'K-343' Letter can be any letter from A to Z, and numbers from 2-5 digits long. Here are some of ...
user avatar
0 votes
2 answers
39 views

Exploratory data analysis - Feature influence on outcome

Am new to data science. In my dataset, I have 100+ features in our dataset of 2000 rows. I guess using all this 100+ features will overfit. So, before I build ML model, I would like to only select ...
user avatar
  • 1,826
0 votes
1 answer
30 views

How to structure this multi-dimensional data for AR modelling?

I have a time-series dataset for each month for the past three years which represent quoted prices for the same product but with different delivery month. For example, Jul-19 is a dataset consisting ...
user avatar
  • 299
4 votes
2 answers
78 views

When interpreting machine learning models, should preprocessing steps be considered as part of "model"?

Suppose I have some inputs on which I first apply some feature engineering and then apply a machine learning algorithm such as random forest to make predictions. Now, if I want to interpret/explain ...
user avatar
0 votes
0 answers
17 views

Does adding calculation or transformation of variables(nonlinear interactions) matter in Gradient Boosting

In traditional linear model, adding variable interactions into the model before training is an important approach. I've heard that Gradient Boosting (Trees are deep enough) can learnt the interaction ...
user avatar
  • 101

1
2 3 4 5
15