3
$\begingroup$

I am imputing missing values in a longitudinal dataset using the Amelia package in R. Does it matter if I have the data in long format (with id, time, and value in each row) or in wide format (with id, valuet1,valuet2,valuet3 in each row) when I impute?

The two seem to produce different results. When I have the data in wide format, the imputed data is super consistent with the original data, but when it's in long format the imputed values don't fit nearly as well and are much more bunched around the mean.

$\endgroup$

1 Answer 1

4
$\begingroup$

By nature of pivoting data into long format, you necessarily have more rows of observations, which means you have more rows to impute. If you have the same missing value repeated multiple times from this transformation, naturally imputation must fill in more gaps. Additionally, if you are compressing more than one variable into one, you are also imputing data based off a single outcome versus multiple outcomes in wide format.

As an example with the air.quality data in R, I have compared a wide format imputation and long format imputation using mice. Before imputation, you may already notice that the wide format data has 153 rows and the long format data has 449 rows.

#### Load Libraries ####
library(mice)
library(tidyverse)
set.seed(123)

#### Wide Format Data ####
air.short <- airquality %>% 
  as_tibble()
air.short # 153 rows

#### Long Format Data ####
air.long <- airquality %>% 
  pivot_longer(cols = 1:2,
               values_to = "Value",
               names_to = "Measure")
air.long # 449 rows

#### Impute Datasets ####
imp.short <- mice(air.short)
imp.long <- mice(air.long)

#### Compare Imputations ####
densityplot(imp.short)
densityplot(imp.long)

Looking at the imp.short density, we get our two variables that have missing data:

enter image description here

We then compare to our single variable which is the pivoted version of both. You can already see that now we have a bimodal distribution based off one single outcome instead of two:

enter image description here

You'll notice if I don't add these two variables and instead pick two date variables to pivot, the imputation still changes based off the reasons I mentioned previously.

#### Second Pivot ####
air.long.2 <- airquality %>% 
  pivot_longer(cols = 5:6,
               values_to = "Date",
               names_to = "Type")
air.long.2 # 296 rows

#### Impute and Plot ####
imp.long.2 <- mice(air.long.2)
densityplot(imp.long.2)

Here we see that despite having our two outcomes unpivoted, the imputation still forces changes, especially in the taller peak for Solar.R.

enter image description here

$\endgroup$
4
  • $\begingroup$ Thanks so much for your response, Shawn. Am I right, based on your explanation, that if I have the same variable (loneliness) evaluated over multiple time points, it should be modeled as one distribution (rather than loneliness at each timepoints being its own variable and distribution) and therefore imputed in a long format rather than a wide one? $\endgroup$
    – Benji
    Jun 7, 2023 at 0:34
  • $\begingroup$ Stef van Buuren (the creator of this package) wrote a book on multiple imputation in R and discusses this specific topic in detail here: stefvanbuuren.name/fimd/sec-longandwide.html $\endgroup$ Jun 7, 2023 at 0:58
  • $\begingroup$ Thanks for sharing that. Based on that it sounds like it doesn't matter if I impute in wide or long format. Do you think I'm justified to impute in the wide format because it fits the original distribution better? $\endgroup$
    – Benji
    Jun 7, 2023 at 17:33
  • 1
    $\begingroup$ It depends on the structure of your data. So long as you don't have something like clustered data or clearly structured missingness, I think you could go ahead with wide in a way similar to what's shown here: stefvanbuuren.name/fimd/sec-fdd.html#sec:fdd $\endgroup$ Jun 7, 2023 at 23:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.