# Questions tagged [data-imputation]

Refers to a general class of methods used to "fill in" missing data. Methods used for doing this typically are related to interpolation (http://en.wikipedia.org/wiki/Interpolation) and require assumptions about why the data is missing (e.g. "missing at random")

691 questions
Filter by
Sorted by
Tagged with
13 views

### multiple imputation with some missing baseline data and with some missing longitudinal data

how shall I impute the data in the following situation: I have some baseline covariates collected and longitudinal data. Both baseline covariates and longitudinal data have some missing data. Shall I ...
• 185
15 views

### Difference between copy increments and copy reference imputation

I'm reading up on reference-based imputation (Carpenter et al. 2013, but the explanation is a little bit too mathematical for me) and I'm not quite sure if I understand the difference between copy ...
24 views

### Multiple imputation mice pmm

I need to impute 20 yes/no variables in my dataset. In this dataset, 35% of the individuals have all 20 of these variables missing, while the remaining 65% have complete data for these variables. ...
• 11
20 views

### How does KNNImputer stores fitted values of the train set?

If someone here is familiar with the KNNImputer implementation of Scikit-learn, I would be eager to learn this from him. When you fit an Imputer transformer on your ...
• 43
13 views

### When imputing a numeric column, does it matter if the imputed value is impossible?

Suppose we have a table with flight data, and one of the numeric columns denotes airline ID. The same airline appears many times in the table, so this column can potentially have predictive value. ...
• 1,443
19 views

### Weighted KNN imputation

Consider the following piece of python code. ...
• 1,443
156 views

### Multiple Imputation for missing outcomes in Cox regression

Imagine an RCT with a time-to-event outcome which is analyzed using a Cox regression. There are four assessments (T1=before randomization, T2=3 weeks, T3=6 weeks, T4=12 weeks). Under the censoring at ...
• 149
8 views

### Handling valid missing data in prediction model features

I'm developing an elastic net model using caret, with k-nearest neighbours to handle missing data in the features. Some features are conditional on others such that a missing value is valid e.g., ...
1 vote
83 views

### What is the name/terminology for this application of OLS regression

I don't come from a statistics background and was instructed to follow these steps to fill in missing data. I'm wondering if there is a name for this specific method so that I can learn more of it and ...
53 views

### How to deal with missing values in a panel survey (for Propensity score matching analysis)

I would like to know what is the recommended method for data imputation for propensity score matching in panel survey data. This survey has 4 waves and I am examining the treatment effect between the ...
14 views

### Feature Selection Before or After kNN Imputation?

From my understanding, kNN imputation is dependent on the variables where any two cases do not have missing values. Thus, would it be ideal to do feature selection before or after imputation?
25 views

### Large Scale Missing Data & Imputation of Time Series Data in Neural Networks [duplicate]

I know there has already been a lot of discussion about this topic, but I have reasons to believe it still remains unanswered and lacks several justifications. Suppose we have an time series feature ...
• 101
1 vote
19 views

### Imputing a Continuous P-value Distribution from Discrete P-values [closed]

I'm exploring methods to create a continuous-like P-value distribution from discrete P-values. For example, consider the process where a balanced coin is tossed four times per experiment to note the ...
• 143
1 vote
40 views

### Question on Best Transformation (Negative, Zero, Positive Values) + Missing Data

I have a dataset with $5000$ observations, and 10 explanatory and 1 response variable (binary 0 or 1), and my task is to make a logistic regression model for prediction (but also needs to provide some ...
121 views

### Best way to impute missing values in a time series

I have a camera that detects every time it views a car. Each detection is recorded in a database. I then simulate this behavior as a time series by doing an each hour count of the records. The problem ...
• 141
22 views

### How can I combine fully measured with partially measured outcomes in an IPD (Network) Meta-Analysis?

I was asked to help in an Individual-Patient Data (IPD) Network Meta-Analysis (NMA). Outcomes are supposed to measure combined scores of condition 1 and condition 2 (thus: $Score=Score_1 + Score_2$), ...
• 1,252
1 vote
29 views

### After the mutiple imputation (MICE package in R), I still found that some variables are still with missing values. How to deal with it?

I have a relatively large data set with around 12000 samples with 550 variables. Originally, I have around 800 variables, I used a rule that if missing rate in each variable is larger than 80% I will ...
41 views

### Imputing missing observations of zip code level data

I am looking for a sufficient imputation method for missing observations in my zip code level data, using R. I have a random sample consisting of households which live in different zip codes within ...
30 views

### Vectorized Linear Regression with missing values

I have a few questions about handling missing data during matrix/array operations. Essentially, I'm doing a vectorized linear regression to perform a bunch of linear regressions in a few matrix/array ...
28 views

### Should EM algorithm's final imputed mean match the initial parameter?

I am running a manual EM (expectation-maximization) algorithm in r. My code is the following: ...
• 65
1 vote
42 views

### Manually program EM in r to updated multiple parameters and solve missing data [closed]

I am trying to use EM (Expectation-maximization) to fill in missing data in R, but am not sure how to model/code it for my specific case. I am generally trying to follow the example format used in ...
• 65
1 vote
29 views

### Impute missing value [closed]

In machine learning when I impute missing values which of the following I perform : 1-Impute data set and then split it? 2-Split dataset to Training and testing datasets and then Impute each datasets ...
• 335
1 vote
32 views

### median imputation or case deletion for small samples

I have a dilemma. I have a data set of different proteins from 4 groups of animals characterized by two factors: age (3 or 12 months) and presence of a certain gene (KO or WT). For each combination of ...
60 views

### Complete case analysis, multiple imputation, or instrumental variable analysis?

I ran a small study where all participants attended a psychological intervention, and completed measures at pre, post, and follow-up. Approx. 40 participants took part, of which approx. 5 did not ...
• 21
1 vote
20 views

### Interpret Mean vs. SMAPE results [duplicate]

I have following dataframe: ...
55 views

### Is it possible to replace missing data if 50% of total data are missing?

In my study, 40K completed household surveys. However, when we suggested visiting a nearby health center to measure their physical parameters (height, weight, blood pressure, and blood glucose), only ...
25 views

### What is the best method for imputing binary (or integral/count) data?

I have a longitudinal dataset, and I want to create a composite score by including five healthy lifestyles to measure the overall lifestyle over time (use as predictor). Each lifestyle is a binary ...
• 347
18 views

### References for effects of constant imputation

I have recently seen people simply adding a constant value in missing data before using regular statistical procedures, but those choices are not argued or do not have any resemblance to the rest of ...
14 views

### Alternative to hot deck imputation

I use hot deck imputation for a project that I work on. But I'm dissatisfied that it can't come up with new values. If my doner set had thousands of values in it, and they were nicely distributed, ...
97 views

### How to use priors to impute values at an individual level and replicate a distribution of the population?

I am trying to correct a variable from a survey that has measurement error. To do this, I have been taking this column as if it was missing and imputing new values based on the predictions of an ...
93 views

### Gaussian Processes regression for missing data?

Suppose I have data with three dimensions and the joint is $P(X,Y,Z)$. Were Z to be missing on occasion, Gaussian Process regression could infer distributions over these missing values. But suppose ...
• 3,382
58 views

### Would it be preferable to use statistical imputation instead of a subject matter expert's subjective estimate for missing data?

I'm working on a project where I need a variable for the total number of medications a patient is on. The PI is a clinician and I feel they would be able to use the resources at hand - case note and ...
• 741
48 views

### Missing data and missing not at random

My outcome is a child cognition scores (y). I will skip the exposure for now. My covariates are regular features like , ...
• 577
32 views

### How can I perform an analysis of the NHIS imputed income variables?

I have downloaded five family income variables from https://nhis.ipums.org/nhis-action/variables/group?id=economic_income (INCPPOINT1, INCPPOINT2, INCPPOINT3, INCPPOINT4, INCPPOINT5) for they years ...
104 views

### Does a relationship between missing values indicate MNAR?

I have a dataset where several features are missing values only if another feature is also missing a value. Does this indicate that the missing data is missing not at random (MNAR)? Additionally, how ...
• 55
14 views

### Choosing Between Training on Historical Data vs. Imputed Data for Time Series Nowcasting

I have several time series datasets which we can label as y, x1, x2, ..., ...
• 225
13 views

### Incremental Users On Platform Method

I have the following data (aggregated, user level data is not available). date | platform | total_watch_time platform = [mobile, mobile + desktop, mobile + desktop + tv] I want to determine the ...
27 views

### Bayesian imputation model for day of month when the year, month, and day of week are available

I'm considering including vehicle incident data paired to date, however the most extensive data for the region I am interested in does not have dates. I would like to include it in a multivariable ...
• 9,421
66 views

### Congeniality between imputation model and analysis model

I have a question about congeniality between imputation model and analysis model. Suppose: Research goal: To estimate the prevalence of disease A among a population as well as among different ...
50 views

### proof of missing at random

I have an longitudinal data with measurements of hand grip strength measured in kg by a hand-held dynamometer, using a maximum of three measurements taken with the strongest hand in an old population....
• 129
1 vote
78 views

### Multilevel multiple imputation in practice using R

I'm currently involved in a project where I want to address missing data using multiple imputation. I'm using healthcare data in a longitudinal setting with 16 time points, where observations are ...
20 views

### Need guidance in missing value imputation in count variables

I have a few count variables that are part of a larger dataset (the rest of the variables are either numerical or ordinal). Their distributions can be seen below. My goal is to impute the missing ...
18 views

### Include fully-observed variables as a response or a covariate in multivariate normal imputation?

I have a few questions regarding the specification of multivariate normal imputation model (MVNI): Suppose I have the following substantive analysis model, and there is missing values in X1. I want to ...
16 views

### Compare LASSO regressions as missing data increases

I'm comparing three LASSO-regression models for classifying two patient types. Each model has increasingly complex variables, which are less likely to be available. Consequently, the last model has ...
• 61
31 views

### How to impute the std deviation of a sample using the pooled mean and standard deviation of similar samples?

I want to perform a meta-regression based on the mean changes from baseline values.In one of the studies, the standard deviation has not been mentioned. How can I impute the value of the missed ...
84 views

### Creating deliberate missing data to impute a counter factual

This question is more out of curiosity than necessity. Any useful resources/literature will be massively appreciated. Recently started research in a new field that I’m unfamiliar with: economic policy ...
63 views

### How to perform inference on regression coefficients when some X values are interpolated?

This should be a simple question but I'm having a hard time finding an answer. Assume I have some data $y$ with covariates $X$. This set of data was obtained via a simple random sample. I want to ...
• 431
20 views

### Is it possible to define a specific criterion prior to imputing missing values? [closed]

it's hard to summarise the problem in a question so I'll explain it here. I have a variable called "Plant Species" that is missing a lot of values because of the sampling method used. For ...
• 21
1 vote
306 views

### How to calculate a linear combination of regression coefficients after multiple imputation?

A method to handle missing data is with multivariate imputation by chained equations. During this process, we create many datasets (as many as you specify) with imputed values for the missing data. ...
• 13
1 vote
I have a data set $\mathbf X$, with around 20 predictors, which is a matrix of parameters of a surrogate model. For each observation $\mathbf i$ of $\mathbf X$, the surrogate model was trained to ...