Questions tagged [missing-data]

When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing an analysis or test.

Filter by
Sorted by
Tagged with
0
votes
0answers
18 views

Justification for FIML under MAR

(To my knowledge...) Under MAR, the likelihood can factored as $L(\theta,\psi|X_{obs},R)=L(\theta|X_{obs})P(R|X_{obs};\psi)$, where $R$ are binary missingness indicators, $X_{obs}$ are the observed ...
0
votes
0answers
18 views

encoding categorical time series data with missing value

I am trying to pre-process (encode / normalize / standardize) my time series data where values are categorical and there exist, two types of nan values ...
2
votes
0answers
21 views

Variable is count total from all clinics within unit that reported cases, some didn't report. How do I deal with this type of missingness?

This is the first time I have posted (but I have read a fair few threads on this site before). This question is quite general in nature since I don't know where to start, or maybe I don't know the ...
1
vote
1answer
21 views

(How) do I need to account for censored data in LCMM?

I have a large dataset of within-individual repeated measures, where each participant has varying number of observations (multiple rows of data for each participant). Missing data is largeley non-...
0
votes
0answers
11 views

generalized estimating equations — some data measured only once

I have a longitudinal data set where about 200 clusters are measured repeatedly. I noticed, however, that about 15 of the clusters are only measured once. They are a part of a broader category, so I ...
2
votes
0answers
16 views

Impact of substituting default values for predictors in Regression Model

Suppose I have a regression model (it could be any model, but I am mainly interested in multiple linear regression) that is designed to predict Y as a function of a set of X predictors. In my field (...
1
vote
2answers
35 views

Comparing Multivariable-Regression Models Derived from Different Sample Sizes

I have a small dataset (n=39) with one dependent variable (y) and several independent variables (x1, x2, …, xn). For most of my independent variables I have 39 measurements. However, I am missing some ...
1
vote
1answer
39 views

Impact on exploratory factor analysis of limiting number of data imputations

I intend to use multiple imputation to deal with missing continuous data before conducting exploratory factor analysis (EFA) on survey data, and to obtain factor scores for each individual case. I ...
2
votes
1answer
26 views

finding patterns on missing data

Is there a way to analyse missing values of nominal data. I heard about multiple imputation, but for my understanding, this provides temporary values to the missing value. But all I want is to ...
0
votes
0answers
8 views

Which variables to include in multiple imputation?

I have collected data from 500 participants, each completing 10 questionnaires. There is a very small amount of missing data (2%, and only for a small number of the Likert items) which I'm excluding ...
0
votes
0answers
10 views

Imputing missing data with ineligible data

I'm running an analysis using medical data and need to perform an imputation on a few variables. However, due to the nature of the data I have a continuous variable which is only able to be present ...
2
votes
0answers
17 views

Pooled Panel Data Regression and Questions on “Missing Values”

Lets say i want to identify the effect of some categorical variables x1 x2 x3 on a numeric y. I got data from 3 waves on the same individuals, but there are missing values on some variables because of ...
0
votes
0answers
15 views

Do I impute missing values with the response?

I have a dataset with missing values in both predictors and the response. As far as I know, the data are missing not at random, so I cannot simply use listwise deletion. Instead, I employed the EM ...
0
votes
1answer
37 views

How can I properly analyze data with many missing values?

I have data 14581 obs. of 45 variables that contains missing values: ...
2
votes
1answer
39 views

Data analysis of unbalanced study - Interaction techniques in virtual reality

I'm planning a study on interaction techniques in virtual reality. That means I want to compare the performance of the participants on different interaction forms (e.g. selecting objects with a ray or ...
0
votes
1answer
33 views

What to assume if exact lower limit data values are not available

I have some data - about 100 values, maximum value 100. However, values below 10 are just written as <10, exact values for these are not available. How can I impute these data points. What value ...
1
vote
0answers
8 views

Consistency of EM for missing data in non-parametric setting

When we have missing data, a parametric model, and an expectation-maximization procedure, and we want to show that our procedure leads to consistent estimators, we can sometimes set up score functions ...
1
vote
1answer
40 views

Dataset with 10% missing without either using a multiple imputation technique or removing the samples. How would I address the missingness?

I have a dataset comprising of 10% missing fraction where missingness can be predicted from the data which I believe is Missing Not At Random. How would I address the missingness of a dataset without ...
0
votes
0answers
11 views

Interpret “if any of - values” when one of any is missing

I am making a research and I have the following dilemma: My material contains some specific allergic diseases, e.g. asthma, birch and egg. I have one variable that checks if there is any allergic ...
1
vote
0answers
18 views

Are my data a good candidate for EM imputation followed by exploratory factor analysis?

I am doing Exploratory Factor Analysis (EFA) in R, using principal axis factoring in the psych package. I have missing data that prevent me getting factor scores, so I am imputing data. I am using ...
0
votes
0answers
8 views

Does missing completely at random and imputing the mean together affect the bias or efficiency of the OLS estimator?

Suppose I am given a response variable $y$ and two predictors $A$, $B$. The correlations between these variables are given by $\rho_{AB} = 0.4395$, $\rho_{Ay} = 0.3141$, and $\rho_{By} = 0.9587$. ...
0
votes
0answers
27 views

Dealing with missing data for date variable where missingness is meaningful

I have the data where there are a lot of variables which contain dates when the event occured and NA when it did not have place. Now I wonder how to handle this issue properly. I'm working with this ...
0
votes
0answers
4 views

RBF-Kernel: Handling missing values

I want to compute the RBF-Kernel for a dataset which contains missing values: ...
3
votes
1answer
102 views

Cross Validation and Multiple Imputation for Missing Data

Using 10 fold CV for performance estimation of a logistic regression model, what is the appropriate way to incorporate multiple imputation for missingness across the predictors and outcome in which ...
1
vote
0answers
21 views

How to deal with a large proportion of missing values

I have to run a regression using a number of variables in a given dataset. However, more than 45% of the values are missing(NA). I am not even sure if they are missing at random, but I presume they ...
1
vote
1answer
22 views

Comparing two groups of measurements with missing values

There are two groups of normally distributed measurements, carried out under the same conditions: $x_i = \theta_{x,i} + \epsilon_x$, $y_i = \theta_{y,i} + \epsilon_y$. I would like to test the ...
2
votes
0answers
13 views

Comparing data when not all members of a population are measured at every time point?

In multiple cases, I have data where there are populations that don't have data points at for some measurements (due to subjects passing away or otherwise). If I wanted to, say, compute the mean of a ...
0
votes
1answer
32 views

Dataset that has approximately 1% missing values completely at random?

I have a dataset with approximately 1% of values missing completely at random. I have thought about using the Multiple Imputation technique but I am not sure if this would be the best solution. ...
5
votes
2answers
122 views

Can you delete the missing values before the analysis of a time series dataset?

The question is: Can I delete all the rows of a certain id because of its missing data, before the data analysis? To be more specific I will give the context and the specific problem: Context I have ...
1
vote
0answers
18 views

Shrinkage in a logistic regression model built from m datasets with imputed values

I am currently running a logistic regression model in R using a dataset where missing values were dealt with by using multiple imputation. I have some experience using the MICE package to deal with ...
2
votes
1answer
48 views

Why is a sample covariance not even semi-positive definite with missing data?

I am trying to estimate a sample covariance when I have less observations $n$ than variables $p$ ($n<p$). This will serve later on as basis for a shrinkage estimator. We know (see this post) that ...
1
vote
0answers
47 views

mice: glm.fit: algorithm did not converge

I have a dataset with about 12 categorical variables with levels ranging from 2 - 10, as well as other numerical variables. About 280 records. I'm using the mice ...
0
votes
0answers
14 views

How to calculate auto-regression with missing values for different surnames in one generation?

I do have a dataset consisting of surnames, years and values y. My aim is to analyze whether the value y is dependent on the corresponding value y of the previous generation. Unfortunately, I do not ...
0
votes
0answers
16 views

Missing data not at random imputation for a ranking

Let's say I am interested in the differences in intelligence of 200 individuals, based on a huge database which ranks almost the entire population. The database do not record intelligence scores, but ...
1
vote
1answer
28 views

Changepoint analysis with missing data

I’m searching for a changepoint algorithm to identify a single changepoint in normally distributed data with missing values and I have strong prior knowledge about where changepoint happens. I ...
0
votes
0answers
27 views

Gaps in time-series for LSTM classification model

I am using an Long short-term memory (LSTM) recurrent neural network model to perform classification of accelerometer sensor data. The experiments (for collecting the data) were run a few months apart ...
0
votes
0answers
24 views

An approach to missing values in time series: easy, universal (and wrong?)

Suppose I have time series data and am fitting some appropriate model to the dependent variable, with or without independent variables, but definitely with some temporal structure that I wish to ...
0
votes
1answer
26 views

Baseline carried forward missing data assumptions

Researchers assessed the effectiveness of a range of weight management programmes for weight loss. A randomised controlled trial study design, incorporating eight treatment arms, was used. Each ...
0
votes
0answers
15 views

How to implement single Imputation from conditional distribution?

In [*] page 264, a method of drawing a missing value from a conditional distribution P(X_mis|X_obs;Theta) which is defined as: I did not find any code ...
1
vote
0answers
17 views

Multiple imputation for count data

Date, Transect, Year, Species, Tr.Length, Month, numb.on.transect I am working with historical sea-survey data.... boat transects counting birds on water. only there are some transects ...
1
vote
0answers
30 views

How to Keep Missing Values in Ordinal Logistic Regression

I’m using mord package in python to do ordinal logit regression (predict response to movie rating 1-5 stars). One of my predictor variables is also ordinal but ...
5
votes
1answer
36 views

What proportion of missing data can be considered acceptable for inference with a mixed-effects model

I am wondering what proportion of missing data can be considered acceptable for use of a mixed effects model? I am analysing a clinical trial testing the efficacy of an agonist drug in reducing the ...
0
votes
0answers
18 views

Imputation in Time Series with Varying Start Dates

I have a set of time series variables, most of which start at the same time as my dependent variable, but some of which are missing at the beginning of the series (at start at varying dates). I want ...
1
vote
0answers
27 views

XGBoost model in Spark --> Missing value treatment [closed]

Unlike python, where missing value is handled internally by the XGBoost algorithm, While building XGBoost model in SPARK, the missing values are implicitly converted to 0.0(float?!). Is this okay ? ...
0
votes
1answer
26 views

Groups of correlated time series prediction

Consider two group of time series $X$ and $Y$. Both $X$ and $Y$ contain time series $\{x_0,..., x_N\}$ and $\{y_0,..., y_N\}$, respectively. Within each group, the time series are correlated and also ...
1
vote
0answers
15 views

Imputation that takes into account both relationships among variables and spatial adjacency?

I have a dataset with 13 variables and 50 observations representing the U.S. states. The variables represent the land use intensity of different agricultural industries in each state. Of those 650 ...
1
vote
1answer
36 views

Fit regression model to the data with 3 dependent variables

I have a dataset that contains different sediment fractions (particle sizes). The column 'Event' consists of unique sample sites. Each sample represents sedimentary material that is sorted by particle ...
2
votes
2answers
42 views

Is data that has been entered incorrectly treated the same as missing data?

I am doing an online study and have just started looking at the data. I noticed two of my participants have listed ages that they couldn't possibly be (e.g 450 and 220). I'm wondering what the ...
0
votes
1answer
29 views

Should reliability test (cronbach alpha) include missing values?

I am currently doing a reliability test for my questionnaires with 34 respondents. However, there are some missing values in the questionnaires which I'm not sure whether I need to include or exclude ...
1
vote
1answer
45 views

Handling missing data with weighting the non-missing data based on the variables that are complete in the dataset

I have a dataset of around 15000 subscribers to a Software, and I also have some meta data collected for each of these subscribers based on which I need to do some analysis. I need to decide whether ...