When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

0
votes
0answers
23 views

FIML using Mplus

I am trying to translate some logistic regressions from SAS to Mplus. Some are fairly straightforward, e.g., no random effects, and others are mixed models with random intercepts. The example here ...
0
votes
0answers
4 views

Can I analyze planned missing (MNAR) data in unbalanced design using SAS PROC MIXED (Generalized LM)?

In my experiment, there are 3 levels of Treatment (A (control), B, and C). All participants (N=109) underwent two of these treatments, the order of which was determined by random assignment to 1 of 4 "...
24
votes
4answers
4k views

Why do some people use -999 or -9999 to replace missing values?

I have a dataset. There are lots of missing values. For some columns, the missing value was replaced with -999, but other columns, the missing value was marked as 'NA'. Why would we use -999 to ...
0
votes
0answers
14 views

What is tolerance technique that use to deal missing data?

Current administrations of processing missing data can be approximately divided into three categories: tolerance, ignoring and imputation-based procedures. i- Missing data ignorance often refers to ...
2
votes
1answer
18 views

Are there any models that can handle out of sample features?

So I'm facing a regression problem where I have a categorical features (factor) where the levels are very commonly different between the training set and the test set. I have multiple measurements ...
0
votes
0answers
9 views

Unbalanced panel dataset estimation

I have a dataset with data concerning numerous variables relative to many countries over a 45 year period of time. The problem is that for many variables there are missing values for many years (worst,...
0
votes
0answers
10 views

How to simulate MNAR missing data in R? [duplicate]

For example, for the MAR case, I know that y <- rbinom(100, size=1, prob=0.1) r <- rbinom(100, size=1, prob=c(.4, .6)[y+1]) #depends on observed value I saw that there is another thread ...
0
votes
1answer
55 views

optimize a function in presence of NAs values in R

I would like to maximize the funcToOpt in the code. Description of the data: wb and X1 ...
0
votes
1answer
18 views

Multiple imputation with high missing rate in covariate

I know that this question has been asked quite a lot - but I did want to see what people's opinions currently were on how applicable Multiple Imputation (MI) is to perform on a dataset with a high ...
0
votes
0answers
9 views

R: how to use Fourier or MCMC (or other type) to reconstruct the data?

i have this time series, with many missing data. The distribution of the data is monthly, with strong seasonality. I would to reconstruct, if is possible, the missing data. In a previous work (other ...
0
votes
0answers
46 views

Missing data imputation in time series in R

I have got hourly temperature data from 2012 to 2016 as follows: ...
2
votes
0answers
20 views

Missing value imputation in huge dataset

I have a huge data (4M x 17) that has missing values. Two columns are categorical, rest all are numerical. Given the huge amount of data, running any imputation method runs forever. What should I do? ...
1
vote
0answers
17 views

How to rank with missing data?

I have 180 subjects that I'd like to rank based on their performance in a variety of categories. However, not every subject is present in each category. Only one of the categories has every subject ...
0
votes
1answer
10 views

Extended Binary Logistic regression - multinomial regression or something else?

I wonder whether you could help me decide which statistical test to use. Briefly, I am testing whether personality (Big Five) predicts problem solving, in N = 282 participants. For personality, the IV,...
0
votes
0answers
9 views

Regression with conditionally observed features

Suppose we are modeling some feature y on features X on our population, where the population is partitioned into classes. Some features are globally observed for everyone, some are class-specific. E....
0
votes
1answer
23 views

Goodness of fit: Generalized Linear Models with missing values in R

I am trying to compare two models and check which is the best fit of our data. The R script is below: ...
1
vote
0answers
7 views

Input missing values in categorical time series

I have a dataset similar to ...
1
vote
1answer
48 views

Shannon entropy and does missing data affect output

So I am doing analysis based on this paper: http://www.bioline.org.br/pdf?se10001 The author uses Shannon entropy in one part to calculate weights and he uses data which does not come with missing ...
1
vote
0answers
14 views

PCA & Cluster analysis for Typology with missing data - Choosing right approach

I am an ecology graduate with a decent practical familiarity with statistics in R, but limited experience of approaches such as PCA, and Cluster Analysis. I am currently faced with the challenge of ...
1
vote
1answer
22 views

Present data quantity

I have some data on widgets. Each widget has a universal identifier (ID), which I can use to compare data (from various sources) about the widget. I have data from some sources, and I wanted to see ...
2
votes
2answers
56 views

how to check missing data is missing at random or not?

I have a survey data, in which there are some missing data (not answered questions). I threw away those where the whole page(s) questions were missed, but there are still some with unanswered ...
0
votes
0answers
35 views

Demographics with missing data

I have been asked to provide the percentage of our clients who identify as Hispanic and the percentage who do not. The issue I am running into is that we do not have this data for 33% of our clients. ...
1
vote
0answers
17 views

Compute a Quadratic discriminant analysis (QDA) in R assuming not normal data and missing information

In this course, the professor is saying that we can compute a QDA with missing data points and non-normal data (even if this assumption can be violated). But the problem is that I don't know any ...
2
votes
1answer
27 views

estimating variance using only data at the tails without resorting to Gibbs sampling

Suppose we know that the population size is $n=1,000$ but for whatever reason, we only have the bottom $n_1=100$ observations and the top $n_2 = 200$ observations. Furthermore, suppose we know the ...
3
votes
1answer
37 views

Is it advisable to include variables that are not in the full model in the imputation model?

I have a dataset with several missing values. I know that the missing is MNAR. I'm trying to use MICE to impute the data; then apply a survival model on the imputed data. The MICE paper advises that ...
3
votes
1answer
57 views

Sensitivity Analysis for Missing Not at Random (MNAR) data

I currently have a dataset which contains variables with different degrees of missingingness. One of the key variables for my analysis has about 12% of the values Missing Not at Random (MNAR). From ...
1
vote
4answers
39 views

Good references on learning how to deal with missing data/imputation

Could you recommend up-to-date and well-supported references on the topic of data imputation?
0
votes
1answer
21 views

Testing for feature importance with missing values

I'm looking for an appropriate model to do the following analysis: I'd like to test which courses are the most important in determining if a student stays in or leaves a university program. Imagine I ...
1
vote
0answers
13 views

Factor analysis without listwise deletion

Can factor analysis be done in a manner which affords missingness on some items? With what methods can a factor analysis be performed in which subjects who are missing on one or two items are ...
2
votes
0answers
31 views

Missing data in path analysis using AMOS — data imputation, EM, multiple imputation, or ML?

This question reflects the dilemma I face in analyzing path models using AMOS with missing data. I have a dataset with n = 116, missing rate = 6% (Missing at Random) for a full path model with about ...
1
vote
0answers
30 views

How to handle missing values in a Random Forest model?

I have a data with a binary target variable and some predictors. I tried running a random forest model and failed. First of all, I found it hard to enter all predictors to the model. I did: ...
1
vote
1answer
33 views

Missing data and communicating bias

I'm a graduate student, so I'm looking for a little more expertise around including family income in a regression model because of a high percentage of missing data (50%). I'm hoping you all can ...
2
votes
1answer
73 views

GLM missing data

I've come across the problem of missing data when doing GLMs. I'm using GLMs to make predictions in R. My dependent variable is continuous and my independent variables are factors. The question arises,...
1
vote
0answers
16 views

Imputation introduces negative values when using imputePCA() from the missMDA package in R?

I am testing out various imputation methods on my data and would like to use imputePCA. It imputes the missing values with no error messages, but when I check the completeObs matrix some of the ...
1
vote
1answer
35 views

How Gower's dissimilarity handle missing values in numeric columns?

I would like to ask a question about Gower dissimilarity, I was wondering how Gower measure handle missing values in numeric columns, especially that Gower standardized each column based on the range ...
1
vote
1answer
20 views

Is there a way to set the desired range of an imputation algorithm?

Goal: I am interested to learn if there is a way to set the range of an imputation algorithm for Missing Not at Random (NMAR) data, such as Multiple Imputation or Maximum Likelihood Estimation. ...
1
vote
0answers
50 views

Multiple membership model random effects specification

We are looking at tournament performance of chess players over time and have a question about the random effects modeling for this. Specifically, every chess player belongs to at least one club, but ...
3
votes
0answers
16 views

How to analyse paired data lacking pairing information?

A colleague has a dataset from a before and after study looking at a continuous outcome that was measured in individuals before and after an intervention. An unknown but not insubstantial proportion ...
0
votes
1answer
42 views

Dealing with missing data in Repeated Measures ANOVA

Hi :) I have a data set of comprised of: different subjects, each tested within subject (three time points) on a continuous scale, on various (unrelated) dependant measures. Some of these subjects "...
2
votes
0answers
25 views

Estimate causal effect with lots of missing values

This is a generic question. Suppose you run an experiment with units randomized between a treatment and a control group. Imagine the response rate is very low in each group, say, below 10% and with a ...
0
votes
1answer
49 views

How to perform SVD to impute missing values, a concrete example

This might be a very stupid question, but I have read the great comments regarding how to deal with missing values before applying svd, but I would like to know how it is going to work, if I apply it ...
0
votes
1answer
13 views

Imputation for Time Series of Accumulated Value

I have a regular time series of accumulated values of a variable (usage) with some missing (sometimes consecutive) intervals. Is there an imputation method that methodologically considers this ...
1
vote
0answers
23 views

Why do I get an error when trying to impute missing data using PMM in MICE package in R?

I am trying to compare imputation methods for an 81 samples x 407 variables data set with ~17% missing values. Some of the variables will be correlated, some highly, that is the nature of the data. ...
0
votes
0answers
9 views

RBF SVM image classification with missing features

I have been working on a image classification problem (face recognition especifically) and my test set has some missing values: for some face images only the upper half is avaliable and for others the ...
0
votes
0answers
16 views

How can convergence (in distribution) be assessed in the context of multiple imputation by chained equations?

The MICE algorithm starts by randomly imputing the missing values in a dataset, and then proceeds to predict the missing values in each variable by modeling the relationship between the non-missing ...
3
votes
1answer
21 views

NA in glm model

I have the data here.But When I tried to build the logistic regression model using glm function its shows NA in TotalVisits. I have found similar question on stack overflow but that is answered for ...
1
vote
1answer
25 views

When is it ok to MI Data with MNAR predictor without further instructions

I have a data set with predictors that are mostly MAR(supposedly), however I do also have one that is likely to be MNAR in the sense that the missing of that predictor depends on an unobserved ...
0
votes
0answers
25 views

Why do I get chi values, when I've stated F test?

I am trying to do an lmer and find my MAM model, using the lme4 package, and have two problems/questions in that connection. Q1: My starting model is this: ...
0
votes
0answers
21 views

Preprocessing in R with Amelia

I'm new to analytics and model building. I had to preprocess a dataset as it had lot of missing values. I get 5 datasets of imputed values using amelia package. Here is where I'm stuck. I need some ...
1
vote
1answer
33 views

Opinion on when to impute data

I work with longitudinal data that tends to be “messy.” For example, we collect eye-tracking and physiological measures at multiple time points in young children. This causes data to be missing not ...