Skip to main content

Questions tagged [data-imputation]

Refers to a general class of methods used to "fill in" missing data. Methods used for doing this typically are related to interpolation (http://en.wikipedia.org/wiki/Interpolation) and require assumptions about why the data is missing (e.g. "missing at random")

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

multiple imputation with some missing baseline data and with some missing longitudinal data

how shall I impute the data in the following situation: I have some baseline covariates collected and longitudinal data. Both baseline covariates and longitudinal data have some missing data. Shall I ...
Kate's user avatar
  • 185
0 votes
0 answers
15 views

Difference between copy increments and copy reference imputation

I'm reading up on reference-based imputation (Carpenter et al. 2013, but the explanation is a little bit too mathematical for me) and I'm not quite sure if I understand the difference between copy ...
Anonyme Ironikerin's user avatar
0 votes
0 answers
24 views

Multiple imputation mice pmm

I need to impute 20 yes/no variables in my dataset. In this dataset, 35% of the individuals have all 20 of these variables missing, while the remaining 65% have complete data for these variables. ...
Jop's user avatar
  • 11
0 votes
1 answer
20 views

How does KNNImputer stores fitted values of the train set?

If someone here is familiar with the KNNImputer implementation of Scikit-learn, I would be eager to learn this from him. When you fit an Imputer transformer on your ...
Yann's user avatar
  • 43
0 votes
1 answer
13 views

When imputing a numeric column, does it matter if the imputed value is impossible?

Suppose we have a table with flight data, and one of the numeric columns denotes airline ID. The same airline appears many times in the table, so this column can potentially have predictive value. ...
Evan Aad's user avatar
  • 1,443
0 votes
0 answers
19 views

Weighted KNN imputation

Consider the following piece of python code. ...
Evan Aad's user avatar
  • 1,443
2 votes
2 answers
156 views

Multiple Imputation for missing outcomes in Cox regression

Imagine an RCT with a time-to-event outcome which is analyzed using a Cox regression. There are four assessments (T1=before randomization, T2=3 weeks, T3=6 weeks, T4=12 weeks). Under the censoring at ...
Survival's user avatar
  • 149
0 votes
0 answers
8 views

Handling valid missing data in prediction model features

I'm developing an elastic net model using caret, with k-nearest neighbours to handle missing data in the features. Some features are conditional on others such that a missing value is valid e.g., ...
megsk's user avatar
  • 3
1 vote
1 answer
83 views

What is the name/terminology for this application of OLS regression

I don't come from a statistics background and was instructed to follow these steps to fill in missing data. I'm wondering if there is a name for this specific method so that I can learn more of it and ...
code_monke's user avatar
0 votes
0 answers
53 views

How to deal with missing values in a panel survey (for Propensity score matching analysis)

I would like to know what is the recommended method for data imputation for propensity score matching in panel survey data. This survey has 4 waves and I am examining the treatment effect between the ...
user23960363's user avatar
0 votes
0 answers
14 views

Feature Selection Before or After kNN Imputation?

From my understanding, kNN imputation is dependent on the variables where any two cases do not have missing values. Thus, would it be ideal to do feature selection before or after imputation?
aj12302's user avatar
0 votes
0 answers
25 views

Large Scale Missing Data & Imputation of Time Series Data in Neural Networks [duplicate]

I know there has already been a lot of discussion about this topic, but I have reasons to believe it still remains unanswered and lacks several justifications. Suppose we have an time series feature ...
LazyAnalyst's user avatar
1 vote
0 answers
19 views

Imputing a Continuous P-value Distribution from Discrete P-values [closed]

I'm exploring methods to create a continuous-like P-value distribution from discrete P-values. For example, consider the process where a balanced coin is tossed four times per experiment to note the ...
irahorecka's user avatar
1 vote
1 answer
40 views

Question on Best Transformation (Negative, Zero, Positive Values) + Missing Data

I have a dataset with $5000$ observations, and 10 explanatory and 1 response variable (binary 0 or 1), and my task is to make a logistic regression model for prediction (but also needs to provide some ...
Alex Smirnov's user avatar
5 votes
1 answer
121 views

Best way to impute missing values in a time series

I have a camera that detects every time it views a car. Each detection is recorded in a database. I then simulate this behavior as a time series by doing an each hour count of the records. The problem ...
Rirro Romeu's user avatar
0 votes
0 answers
22 views

How can I combine fully measured with partially measured outcomes in an IPD (Network) Meta-Analysis?

I was asked to help in an Individual-Patient Data (IPD) Network Meta-Analysis (NMA). Outcomes are supposed to measure combined scores of condition 1 and condition 2 (thus: $Score=Score_1 + Score_2$), ...
Federico Tedeschi's user avatar
1 vote
0 answers
29 views

After the mutiple imputation (MICE package in R), I still found that some variables are still with missing values. How to deal with it?

I have a relatively large data set with around 12000 samples with 550 variables. Originally, I have around 800 variables, I used a rule that if missing rate in each variable is larger than 80% I will ...
Steven Xu's user avatar
0 votes
0 answers
41 views

Imputing missing observations of zip code level data

I am looking for a sufficient imputation method for missing observations in my zip code level data, using R. I have a random sample consisting of households which live in different zip codes within ...
Ottibanane123's user avatar
0 votes
1 answer
30 views

Vectorized Linear Regression with missing values

I have a few questions about handling missing data during matrix/array operations. Essentially, I'm doing a vectorized linear regression to perform a bunch of linear regressions in a few matrix/array ...
Victor Yerz's user avatar
0 votes
0 answers
28 views

Should EM algorithm's final imputed mean match the initial parameter?

I am running a manual EM (expectation-maximization) algorithm in r. My code is the following: ...
flâneur's user avatar
1 vote
0 answers
42 views

Manually program EM in r to updated multiple parameters and solve missing data [closed]

I am trying to use EM (Expectation-maximization) to fill in missing data in R, but am not sure how to model/code it for my specific case. I am generally trying to follow the example format used in ...
flâneur's user avatar
1 vote
1 answer
29 views

Impute missing value [closed]

In machine learning when I impute missing values which of the following I perform : 1-Impute data set and then split it? 2-Split dataset to Training and testing datasets and then Impute each datasets ...
zhyan's user avatar
  • 335
1 vote
0 answers
32 views

median imputation or case deletion for small samples

I have a dilemma. I have a data set of different proteins from 4 groups of animals characterized by two factors: age (3 or 12 months) and presence of a certain gene (KO or WT). For each combination of ...
Giacomo Diaz's user avatar
2 votes
1 answer
60 views

Complete case analysis, multiple imputation, or instrumental variable analysis?

I ran a small study where all participants attended a psychological intervention, and completed measures at pre, post, and follow-up. Approx. 40 participants took part, of which approx. 5 did not ...
catll0's user avatar
  • 21
1 vote
0 answers
20 views

Interpret Mean vs. SMAPE results [duplicate]

I have following dataframe: ...
Maxl Gemeinderat's user avatar
0 votes
1 answer
55 views

Is it possible to replace missing data if 50% of total data are missing?

In my study, 40K completed household surveys. However, when we suggested visiting a nearby health center to measure their physical parameters (height, weight, blood pressure, and blood glucose), only ...
Dr bappa's user avatar
0 votes
0 answers
25 views

What is the best method for imputing binary (or integral/count) data?

I have a longitudinal dataset, and I want to create a composite score by including five healthy lifestyles to measure the overall lifestyle over time (use as predictor). Each lifestyle is a binary ...
zjppdozen's user avatar
  • 347
0 votes
0 answers
18 views

References for effects of constant imputation

I have recently seen people simply adding a constant value in missing data before using regular statistical procedures, but those choices are not argued or do not have any resemblance to the rest of ...
o---o's user avatar
  • 1
0 votes
0 answers
14 views

Alternative to hot deck imputation

I use hot deck imputation for a project that I work on. But I'm dissatisfied that it can't come up with new values. If my doner set had thousands of values in it, and they were nicely distributed, ...
Leonhard Euler's user avatar
2 votes
1 answer
97 views

How to use priors to impute values at an individual level and replicate a distribution of the population?

I am trying to correct a variable from a survey that has measurement error. To do this, I have been taking this column as if it was missing and imputing new values based on the predictions of an ...
Santiago Valdivieso's user avatar
2 votes
0 answers
93 views

Gaussian Processes regression for missing data?

Suppose I have data with three dimensions and the joint is $P(X,Y,Z)$. Were Z to be missing on occasion, Gaussian Process regression could infer distributions over these missing values. But suppose ...
jbuddy_13's user avatar
  • 3,382
2 votes
3 answers
58 views

Would it be preferable to use statistical imputation instead of a subject matter expert's subjective estimate for missing data?

I'm working on a project where I need a variable for the total number of medications a patient is on. The PI is a clinician and I feel they would be able to use the resources at hand - case note and ...
Geoff's user avatar
  • 741
0 votes
0 answers
48 views

Missing data and missing not at random

My outcome is a child cognition scores (y). I will skip the exposure for now. My covariates are regular features like , ...
Science11's user avatar
  • 577
0 votes
1 answer
32 views

How can I perform an analysis of the NHIS imputed income variables?

I have downloaded five family income variables from https://nhis.ipums.org/nhis-action/variables/group?id=economic_income (INCPPOINT1, INCPPOINT2, INCPPOINT3, INCPPOINT4, INCPPOINT5) for they years ...
tryingtogetsmth's user avatar
2 votes
2 answers
104 views

Does a relationship between missing values indicate MNAR?

I have a dataset where several features are missing values only if another feature is also missing a value. Does this indicate that the missing data is missing not at random (MNAR)? Additionally, how ...
sla813's user avatar
  • 55
0 votes
0 answers
14 views

Choosing Between Training on Historical Data vs. Imputed Data for Time Series Nowcasting

I have several time series datasets which we can label as y, x1, x2, ..., ...
The One's user avatar
  • 225
0 votes
0 answers
13 views

Incremental Users On Platform Method

I have the following data (aggregated, user level data is not available). date | platform | total_watch_time platform = [mobile, mobile + desktop, mobile + desktop + tv] I want to determine the ...
lseactuary's user avatar
0 votes
0 answers
27 views

Bayesian imputation model for day of month when the year, month, and day of week are available

I'm considering including vehicle incident data paired to date, however the most extensive data for the region I am interested in does not have dates. I would like to include it in a multivariable ...
Galen's user avatar
  • 9,421
0 votes
0 answers
66 views

Congeniality between imputation model and analysis model

I have a question about congeniality between imputation model and analysis model. Suppose: Research goal: To estimate the prevalence of disease A among a population as well as among different ...
Guoqiang Zhang's user avatar
2 votes
1 answer
50 views

proof of missing at random

I have an longitudinal data with measurements of hand grip strength measured in kg by a hand-held dynamometer, using a maximum of three measurements taken with the strongest hand in an old population....
user358238's user avatar
1 vote
0 answers
78 views

Multilevel multiple imputation in practice using R

I'm currently involved in a project where I want to address missing data using multiple imputation. I'm using healthcare data in a longitudinal setting with 16 time points, where observations are ...
actual-garlic's user avatar
0 votes
0 answers
20 views

Need guidance in missing value imputation in count variables

I have a few count variables that are part of a larger dataset (the rest of the variables are either numerical or ordinal). Their distributions can be seen below. My goal is to impute the missing ...
pypau's user avatar
  • 3
0 votes
0 answers
18 views

Include fully-observed variables as a response or a covariate in multivariate normal imputation?

I have a few questions regarding the specification of multivariate normal imputation model (MVNI): Suppose I have the following substantive analysis model, and there is missing values in X1. I want to ...
Guoqiang Zhang's user avatar
0 votes
0 answers
16 views

Compare LASSO regressions as missing data increases

I'm comparing three LASSO-regression models for classifying two patient types. Each model has increasingly complex variables, which are less likely to be available. Consequently, the last model has ...
Fabian's user avatar
  • 61
0 votes
0 answers
31 views

How to impute the std deviation of a sample using the pooled mean and standard deviation of similar samples?

I want to perform a meta-regression based on the mean changes from baseline values.In one of the studies, the standard deviation has not been mentioned. How can I impute the value of the missed ...
victor james's user avatar
4 votes
1 answer
84 views

Creating deliberate missing data to impute a counter factual

This question is more out of curiosity than necessity. Any useful resources/literature will be massively appreciated. Recently started research in a new field that I’m unfamiliar with: economic policy ...
Rhys Maredudd Davies's user avatar
2 votes
0 answers
63 views

How to perform inference on regression coefficients when some X values are interpolated?

This should be a simple question but I'm having a hard time finding an answer. Assume I have some data $y$ with covariates $X$. This set of data was obtained via a simple random sample. I want to ...
ischmidt20's user avatar
2 votes
0 answers
20 views

Is it possible to define a specific criterion prior to imputing missing values? [closed]

it's hard to summarise the problem in a question so I'll explain it here. I have a variable called "Plant Species" that is missing a lot of values because of the sampling method used. For ...
DeeDee's user avatar
  • 21
1 vote
1 answer
306 views

How to calculate a linear combination of regression coefficients after multiple imputation?

A method to handle missing data is with multivariate imputation by chained equations. During this process, we create many datasets (as many as you specify) with imputed values for the missing data. ...
Reid's user avatar
  • 13
1 vote
0 answers
34 views

Imputation method for missing values that are irrelevant

I have a data set $\mathbf X$, with around 20 predictors, which is a matrix of parameters of a surrogate model. For each observation $\mathbf i$ of $\mathbf X$, the surrogate model was trained to ...
Florent H's user avatar
  • 153

1
2 3 4 5
14