# Questions tagged [missing-data]

When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing an analysis or test.

1,629 questions
Filter by
Sorted by
Tagged with
1 vote
15 views

### Statistical Non-Response and Drop Out

Hello Stats Community! This is a question I have. In statistical studies, it is possible that there might be biases: Someone groups of people are more likely to be represented compared to others ...
48 views

### Drop-outs in RCT

I have conducted a RCT with 34 participants, experimental group and a control group. We took measurement data before randomisation at baseline (T0), and then follow-up measurements every 4 months (so ...
• 21
9 views

### Multilevel/mixed model when group is missing for some of the data

A client has given me a dataset of corn samples gathered from different loads of corn that were delivered to a grain elevator. They are interested in whether the concentration of different fungal ...
• 295
131 views

### Multiple Imputation for missing outcomes in Cox regression

Imagine an RCT with a time-to-event outcome which is analyzed using a Cox regression. There are four assessments (T1=before randomization, T2=3 weeks, T3=6 weeks, T4=12 weeks). Under the censoring at ...
• 41
65 views

### Is it valid to use "withdrew from study" in a mixed model to help with dropout bias in longitudinal data?

Consider the hypothetical dataset where a number of study participants have no change over time in a particular outcome, but their probability to drop out of the study is related to the baseline value ...
• 225
1 vote
6 views

### Comparing assessment methods in longitudinal design

I am currently trying to figure out what to do but I can´t seem to find a proper solution. My problem is the following: I have a longitudinal design. For each day I have a variable stating whether an ...
• 11
26 views

### what are the consequences of re-encoding with 0 or -1?

I have a dataset of information about students and the last column is the target variable which is the final note. My goal is to make logistic regression and ordinal regression models to see whether ...
8 views

### Handling valid missing data in prediction model features

I'm developing an elastic net model using caret, with k-nearest neighbours to handle missing data in the features. Some features are conditional on others such that a missing value is valid e.g., ...
1 vote
32 views

### ML Modelling advice where a feature is partially missing but highly informative when present

I am building a model to predict a customer purchase event on a website. Specifically for those customers who, overnight when the model is run, have not yet purchased. Prediction is important, but ...
• 141
1 vote
14 views

### How to deal with MNAR data based on failures in a trial?

Background I have a dataset in which participants had to perform a certain task within a time frame. The outcome of this task is not binary, but there was a criterion that should be reached within the ...
• 115
39 views

### Missing Data in experiments, MIPO, MIPO|X vs MCAR, MAR, MNAR [closed]

Hello I was reading Field Experiments by Alan Gerber and Donald Green and was introduced to the idea of missingness independent of potential outcomes (MIPO). And MIPO|X which is missingness ...
• 518
16 views

### Reasons for failure of convergence in multiple imputation in wide format of (longitudinally) repeated Likert items

Our group is working on a dataset of approximately 1000 patients with 10 complete variables at the time of an acute disease, who subsequently completed a questionnaire of 5 Likert items (questions ...
• 305
1 vote
16 views

### mice: Default ridge parameter

Some methods in mice like pmm and norm apply a default ridge parameter of 1e-5 when the underlying data is nearly multi-...
• 31
1 vote
85 views

### Difference between truncated and unseen data

I have 2 related questions. Assume that we want to build a model to study of some random discrete variable $x$ that follows some known distribution with PMF $P(x)$, yet with unknown parameters that we ...
• 37
1 vote
105 views

### Combining regression models based on missing data patterns

I have a dataset that contains a few patterns of missingness. For this dataset, I have a training set that is complete and contains all input features. My test set has complete observations for the ...
43 views

### Cox Regression: handling immediate drop-outs

In an RCT with two groups, I‘m currently analyzing data using cox regression. While I‘m familiar with the concept of censoring, a rather substantial amount of participants (~32% and 35% in the groups) ...
• 41
123 views

### SEM with 50% missing data (due to distribution of items over various survey ballots/waves)

I want to test a multilevel mediation (particularly, test a number of hypotheses about possible mediators of the effect of social class on Right-Wing Authoritarianism) using largely categorical ...
• 51
16 views

### Data imputation for Laboratory Experiments

I have a timeseries dataset containing several experiments that were conducted in batches and concatenated (each batch is 48hours, with data recorded every 2 hours). However, there is a target ...
50 views

### How to deal with missing values in a panel survey (for Propensity score matching analysis)

I would like to know what is the recommended method for data imputation for propensity score matching in panel survey data. This survey has 4 waves and I am examining the treatment effect between the ...
6 views

### Comparing Two-Way Fixed Effects Model using Pooled OLS and Panel Data Fixed Effects Approach

I am working with an unbalanced panel dataset that has many missing values. I want to control for both firm-specific and time-specific effects in my analysis. I am considering two approaches: ...
• 83
16 views

### Missing features in decision tree based algorithms

I have a medium-sized dataset consisting of many features, some of which can contain missing values. I want to predict a variable using an algorithm that employs decision trees (specifically XGBoost, ...
• 75
23 views

### Large Scale Missing Data & Imputation of Time Series Data in Neural Networks [duplicate]

I know there has already been a lot of discussion about this topic, but I have reasons to believe it still remains unanswered and lacks several justifications. Suppose we have an time series feature ...
• 101
9 views

### Seeking Advice on Handling Systematic Missingness Due to Conditional Logic in Dataset for Binary Outcome Analysis

I am tackling a complex multivariate analysis project with a binary outcome variable. My dataset's unique challenge arises from structured missingness due to conditional logic responses, significantly ...
46 views

### How find the "closest" (in a sense of data generating process) time series?

Suppose we have overall $m$ time series, each with $n$ observations. We also have another time series with $n-k$ observations ($k>0$). Given the shortest series, I want to find from $m$ series ...
• 353
24 views

### Does survey R package allow me to do beta regression?

I have a complex survey dataset with a response (dependent variable) bounded between 0 and 1, where I have applied multiple imputation to the dataset to account for missing data. The response formally ...
• 1,446
8 views

### Weighing by missing observations

I have data where each row represents an observation and each column represents a variable. Variables are binary and has missing values. I want to calculate a sum of the presence of these variables. I ...
1 vote
38 views

### Question on Best Transformation (Negative, Zero, Positive Values) + Missing Data

I have a dataset with $5000$ observations, and 10 explanatory and 1 response variable (binary 0 or 1), and my task is to make a logistic regression model for prediction (but also needs to provide some ...
609 views

### How to analyze a dichotomous outcome with 50% missing data?

I am researching predictors of dropout from a training program. I want so to see if personality traits add incremental variance above well-established predictors like age, fitness, and education. So, ...
• 341
12 views

### Interpretation question about the article "Finite mixture modeling of censored data using the multivariate Student-t distribution"

I am reading this article and I am struggling to understand the following passage from page 6: \begin{align*} L_{i}(\boldsymbol{\theta}|\textbf{V}_{i},\textbf{C}_{i}) = f(\textbf{V}_{i}|\textbf{C}_{i},...
• 235
16 views

### Analyzing longitudinal data with deaths

I've been given some longitudinal data of animals given different glioblastoma treatments. In addition to conventional survival analysis, they want to look at animal weights and sizes of tumors along ...
• 1,059
1 vote
18 views

### How to report participant number with missing times?

I have used a linear mixed model for a set of data with children participating in experiments 1-4 times. The number of times they participated varies. Some kids skipped or participated in one or two ...
• 11
56 views

### Principal components analysis missing data SPSS

I have a large data set (27k) There are 63 variables of interest. I attempted PCA as a dimensionality reduction strategy. The items are scored on different metrics ranging from dichotomous variables ...
38 views

### Double selection lasso in and NA's handling

I work in a team where everyone uses Stata, and I work in R. I have created an efficient workflow that allows me to export the results quickly. The problem I ran into was when implementing the double ...
• 131
19 views

### Approach for multivariate outlier detection when treating missing values with FIML

I‘m calculating a simple regression with one predictor and one dependent variable. Missings treatment is done with full information maximum likelihood (FIML). Should I do outlier detection, i.e. ...
• 247
81 views

### Shap values associated to missing features

I have a model trained with Xgboost on some training data X_train, described by 10 features (x$_{1}$,...,x$_{10}$) and some of them might exhibit some missingness, i.e. some x$_{i}$ = NaN. This is ok ...
35 views

### imputation of NA on financial ratios

I'm a student doing a data science report on finance. I need to impute NA on some financial ratios. For example, ROAE. I have negative values and positive values. There are also NA, where they are ...
16 views

### Identifying the type of missing data and the post hoc test that can be carried out for Skillings Mack test

I have a non-normal paired sample dataset. Each row represents a dog that has been tested for an experiment. Each dog was provided with three cues (treatments): 5s cue (aka only face cue), vocalone (...
54 views

### I get very different and inconsistent model fit indices and test statistics every time I run the cfa.mi models with the same imputed categorical data

I am trying to test the measurement invariance in my one-latent-factor model. I have 24 questions coded "correct" and "incorrect" and 3 different groups in my data. The sample ...
41 views

### Multilevel model with random subset of conditions per participant

I have a repeated measures experiment with 5 factors each with 3 different levels. I'm trying to figure out a way to reduce the demand on participants by making them not have to sit through every ...
149 views

### Survival analysis with missing values for some variables

I'm performing a survival analysis. I did the univariant Cox regression model for all my variables (about 40). I even performed the univariant analysis in all those variables that had missing values ...
• 73
133 views

### how to deal with treatment dropouts in experimental designs

I conducted a between-subjects experiment with one 3-level factor (high group vs low group vs control group). Because of dropouts from the treatments, the final distribution is control group: 85 ...
• 41
59 views

### Using FIML (Full Information Maximum Likelihood) for simple correlation test on data with missings?

I have a dataset with missing values. Since my main hypothesis is just a simple regression, my adviser told me to use FIML regression. So I used lavaan package: ...
• 247
28 views

### What are the steps to simulate data showing bias when missing data are MAR

I have been trying to simulate data that shows bias in the estimated regression coefficients when there is data Missing At Random. In this case I am not interested in the estimated coefficients ...
• 155
16 views

### When using Weighted Estimating Equations (WEE) to estimate a linear regression model with missing data, what can do if missing probability is 1

When using Weighted Estimating Equations (WEE) to estimate a linear regression model with missing data. One way is to assume the missing at random and then compute the missing probability using some ...
4 views

### Estimating a time series' likelihood of a missing observation

I have this time series of seasonal monthly data, which sometimes has missing observations. The likelihood of an observation being missing is mostly dependent on the month, but also of the value had ...
573 views

### Missing data and maximum likelihood

I've heard it said that maximum likelihood estimation is an alternative to imputation methods for missing data. Does that mean any model fitted using maximum likelihood such as logistic regression, ...
• 687
204 views

### Why is multiple imputation not used more widely in Data Science? [closed]

I posted this question a few days ago on datascience.SE because I thought it was more relevant there: Why is multiple imputation not used more widely in Data Science? I have a background in ...
• 3,845
1 vote
39 views

### Multiple imputation: deleting cases before imputation

Note: The question has been edited to make it more focused, and the title has been changed to make it clearer. I have read questions/answers about how to select variables for imputation. This question ...
• 11
1 vote
52 views

### Getting individual level scores from factor analysis with lots of missing data

I have a setting where I'm doing factor analysis in a context where I have lots of rows and where ~90% of data is missing (it's a survey of a couple hundred thousand people, each person was asked a ...
• 1,491