When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

1
vote
2answers
39 views

how to check missing data is missing at random or not?

I have a survey data, in which there are some missing data (not answered questions). I threw away those where the whole page(s) questions were missed, but there are still some with unanswered ...
0
votes
0answers
22 views

Demographics with missing data

I have been asked to provide the percentage of our clients who identify as Hispanic and the percentage who do not. The issue I am running into is that we do not have this data for 33% of our clients. ...
1
vote
0answers
16 views

Compute a Quadratic discriminant analysis (QDA) in R assuming not normal data and missing information

In this course, the professor is saying that we can compute a QDA with missing data points and non-normal data (even if this assumption can be violated). But the problem is that I don't know any ...
2
votes
1answer
25 views

estimating variance using only data at the tails without resorting to Gibbs sampling

Suppose we know that the population size is $n=1,000$ but for whatever reason, we only have the bottom $n_1=100$ observations and the top $n_2 = 200$ observations. Furthermore, suppose we know the ...
3
votes
1answer
32 views

Is it advisable to include variables that are not in the full model in the imputation model?

I have a dataset with several missing values. I know that the missing is MNAR. I'm trying to use MICE to impute the data; then apply a survival model on the imputed data. The MICE paper advises that ...
1
vote
0answers
17 views

Sensitivity Analysis for Missing Not at Random (MNAR) data

I currently have a dataset which contains variables with different degrees of missingingness. One of the key variables for my analysis has about 12% of the values Missing Not at Random (MNAR). From ...
1
vote
4answers
34 views

Good references on learning how to deal with missing data/imputation

Could you recommend up-to-date and well-supported references on the topic of data imputation?
0
votes
1answer
20 views

Testing for feature importance with missing values

I'm looking for an appropriate model to do the following analysis: I'd like to test which courses are the most important in determining if a student stays in or leaves a university program. Imagine I ...
1
vote
0answers
12 views

Factor analysis without listwise deletion

Can factor analysis be done in a manner which affords missingness on some items? With what methods can a factor analysis be performed in which subjects who are missing on one or two items are ...
2
votes
0answers
15 views

Missing data in path analysis using AMOS — data imputation, EM, multiple imputation, or ML?

This question reflects the dilemma I face in analyzing path models using AMOS with missing data. I have a dataset with n = 116, missing rate = 6% (Missing at Random) for a full path model with about ...
0
votes
0answers
25 views

How to handle missing values in a Random Forest model?

I have a data with a binary target variable and some predictors. I tried running a random forest model and failed. First of all, I found it hard to enter all predictors to the model. I did: ...
1
vote
1answer
32 views

Missing data and communicating bias

I'm a graduate student, so I'm looking for a little more expertise around including family income in a regression model because of a high percentage of missing data (50%). I'm hoping you all can ...
2
votes
1answer
65 views

GLM missing data

I've come across the problem of missing data when doing GLMs. I'm using GLMs to make predictions in R. My dependent variable is continuous and my independent variables are factors. The question arises,...
1
vote
0answers
9 views

Imputation introduces negative values when using imputePCA() from the missMDA package in R?

I am testing out various imputation methods on my data and would like to use imputePCA. It imputes the missing values with no error messages, but when I check the completeObs matrix some of the ...
1
vote
1answer
31 views

How Gower's dissimilarity handle missing values in numeric columns?

I would like to ask a question about Gower dissimilarity, I was wondering how Gower measure handle missing values in numeric columns, especially that Gower standardized each column based on the range ...
0
votes
1answer
10 views

Is there a way to set the desired range of an imputation algorithm?

Goal: I am interested to learn if there is a way to set the range of an imputation algorithm for Missing Not at Random (NMAR) data, such as Multiple Imputation or Maximum Likelihood Estimation. ...
1
vote
0answers
45 views

Multiple membership model random effects specification

We are looking at tournament performance of chess players over time and have a question about the random effects modeling for this. Specifically, every chess player belongs to at least one club, but ...
3
votes
0answers
15 views

How to analyse paired data lacking pairing information?

A colleague has a dataset from a before and after study looking at a continuous outcome that was measured in individuals before and after an intervention. An unknown but not insubstantial proportion ...
0
votes
1answer
27 views

Dealing with missing data in Repeated Measures ANOVA

Hi :) I have a data set of comprised of: different subjects, each tested within subject (three time points) on a continuous scale, on various (unrelated) dependant measures. Some of these subjects "...
2
votes
0answers
24 views

Estimate causal effect with lots of missing values

This is a generic question. Suppose you run an experiment with units randomized between a treatment and a control group. Imagine the response rate is very low in each group, say, below 10% and with a ...
0
votes
1answer
40 views

How to perform SVD to impute missing values, a concrete example

This might be a very stupid question, but I have read the great comments regarding how to deal with missing values before applying svd, but I would like to know how it is going to work, if I apply it ...
0
votes
1answer
10 views

Imputation for Time Series of Accumulated Value

I have a regular time series of accumulated values of a variable (usage) with some missing (sometimes consecutive) intervals. Is there an imputation method that methodologically considers this ...
1
vote
0answers
19 views

Why do I get an error when trying to impute missing data using PMM in MICE package in R?

I am trying to compare imputation methods for an 81 samples x 407 variables data set with ~17% missing values. Some of the variables will be correlated, some highly, that is the nature of the data. ...
0
votes
0answers
8 views

RBF SVM image classification with missing features

I have been working on a image classification problem (face recognition especifically) and my test set has some missing values: for some face images only the upper half is avaliable and for others the ...
0
votes
0answers
14 views

How can convergence (in distribution) be assessed in the context of multiple imputation by chained equations?

The MICE algorithm starts by randomly imputing the missing values in a dataset, and then proceeds to predict the missing values in each variable by modeling the relationship between the non-missing ...
3
votes
1answer
21 views

NA in glm model

I have the data here.But When I tried to build the logistic regression model using glm function its shows NA in TotalVisits. I have found similar question on stack overflow but that is answered for ...
1
vote
1answer
22 views

When is it ok to MI Data with MNAR predictor without further instructions

I have a data set with predictors that are mostly MAR(supposedly), however I do also have one that is likely to be MNAR in the sense that the missing of that predictor depends on an unobserved ...
0
votes
0answers
25 views

Why do I get chi values, when I've stated F test?

I am trying to do an lmer and find my MAM model, using the lme4 package, and have two problems/questions in that connection. Q1: My starting model is this: ...
0
votes
0answers
13 views

Preprocessing in R with Amelia

I'm new to analytics and model building. I had to preprocess a dataset as it had lot of missing values. I get 5 datasets of imputed values using amelia package. Here is where I'm stuck. I need some ...
1
vote
1answer
30 views

Opinion on when to impute data

I work with longitudinal data that tends to be “messy.” For example, we collect eye-tracking and physiological measures at multiple time points in young children. This causes data to be missing not ...
0
votes
0answers
27 views

Unequal timepoints longitudinal data with missing values

I have a longitudinal data with unequal time points with missing values. I am looking for methods to impute the missing data. I looked at R packages NORM and AMELIA II and SAS procedures PROC MI. All ...
0
votes
0answers
13 views

JAGS missing data problem

For missing data problem in RJAGS, in this site, there are a lot of posts talking about setting priors to generate data for the missing place. However, instead of filling up the places, I tend to ...
1
vote
0answers
11 views

Method for Estimating Autocorrelation in Severely Gappy Data

My question is somewhat long and boils down to "Does the following work?" I'm working on a project that involves timing analysis of astrophysical data sets that have large chunks of data missing due ...
1
vote
1answer
21 views

Is there a reasonable way to calculate when missing responses in a survey are too much?

I'm conducting analyzing a complex sample design survey of health institutions, which I've 70% of the overall planned sample. However, I've strata as high as 95% and as low as 47% response rate. To ...
3
votes
1answer
49 views

How to deal with dropouts from a waiting list control group?

Many treatment studies compare a treatment group with a waiting list control group, for example to adjust for spontaneous remissions. Unfortunately, many more participants drop out from the waiting ...
0
votes
0answers
19 views

Distinguishing between zeros and missing data

I have a panel data set with 12,000 observations of daily counts of visitors to a number of recreational sites. The data has been given to me with missing values recorded as zeroes. There are also "...
0
votes
1answer
52 views

Problems with Missing values

I have a data set for a predictive model(predicting survival rate with certain acute medical condition on some animals) with 25 predictors where around 30% of the predictors are complete, 3 predictors ...
0
votes
2answers
36 views

Should I convert raw data into growth rates if there are gaps in my data?

I have data from 214 countries that range from 1990 to 2014. My dependent variables (I'm doing more than one regression) are just primary/secondary net enrollment rates for both sexes/males/females, ...
0
votes
0answers
18 views

How to compare contributory values over time? (with missing values)

Context I am working on a project considering computers. I have a dataset containing around 50 product characteristics for each computer. Also, for each computer, I have data about the product price ...
1
vote
1answer
26 views

How to handle interruptions (0 values) over a certain period in time series analysis?

Is there a way that I can combine two models for time series? I am trying to predict the production of tomatoes per week per $m^2$ (black line), based on light (orange) and temperature (magenta). At ...
1
vote
0answers
37 views

Unbalanced two-factor repeated measures ANOVA with missing values

For my data set, I need to perform some sort of two factor repeated measures ANOVA. I have one between-subject factor called "Treatment" and one within-subject factor called "Frequency" with 8 levels. ...
0
votes
1answer
38 views

NA in ARIMA model, is it suggest that the model is over fit?

my professor says that if we see that there are NA values for the AR or MA terms(either the estimated values or the estimated se) in the R output for the arima models fitted using the arima() function,...
1
vote
1answer
25 views

How many multiple imputation datasets should we make?

Multiple imputation is based on making m different datasets and analyzing them each independently then aggregating over all their information as a whole. How many imputed datasets should we make?
0
votes
1answer
47 views

building up a predictive model with lots of features and missing data

I'm learning using R to build predictive models recently by myself and have many questions on how to attack a question. I'm given a data set of 8000 observations with 300 features. My goal is to build ...
3
votes
2answers
103 views

Meaning of “missing by design” in longitudinal studies

I'm French and I'm reading an English book. I don't understand the term "when missingness is by design" — what does "by design" mean?
3
votes
1answer
44 views

Missing value imputation with nearest neighbour

I'm using k-nearest neighbour imputation method for missing values. This method has two tuning parameters: k and the distance metric. I see two options for applying this imputation method: Inside ...
1
vote
1answer
31 views

Relationships of missing values for exploratory analysis

I have a survey with 30 questions on a seven item likert scale Not all of the questions were answered. I can use a heat map to visualize the missing answers but what i would like to know is if there ...
0
votes
0answers
49 views

Multiple imputation for predictive analysis using mice package in R

I am using the mice package to impute some missing values, and it works nicely. I am facing a tricky strategic question though. Basically, I am working on predictors of myocardial infarction (time 3)...
0
votes
1answer
79 views

K-nearest neighbour imputation of missing values

I have a dataset where the columns correspond to features and the rows correspond to data points. I have around 5'000 data points and 8 features. Now, I would like to impute the missing values with ...
0
votes
1answer
67 views

How to handle empty data in linear regression?

This question may seem like a duplicate but I haven't found quite another one that fits my case. So I am trying to run a linear regression on some variables including 1 dichotomous IV (Tool) that is ...