When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

0
votes
0answers
18 views

Unequal timepoints longitudinal data with missing values

I have a longitudinal data with unequal time points with missing values. I am looking for methods to impute the missing data. I looked at R packages NORM and AMELIA II and SAS procedures PROC MI. All ...
0
votes
0answers
11 views

How the expectation-maximization algorithm could be used for finding missing values in the data? [on hold]

How does the EM algorithm deal with missing data values? Is it the best method for this purpose?
0
votes
0answers
10 views

JAGS missing data problem

For missing data problem in RJAGS, in this site, there are a lot of posts talking about setting priors to generate data for the missing place. However, instead of filling up the places, I tend to ...
1
vote
0answers
6 views

Method for Estimating Autocorrelation in Severely Gappy Data

My question is somewhat long and boils down to "Does the following work?" I'm working on a project that involves timing analysis of astrophysical data sets that have large chunks of data missing due ...
1
vote
1answer
13 views

Is there a reasonable way to calculate when missing responses in a survey are too much?

I'm conducting analyzing a complex sample design survey of health institutions, which I've 70% of the overall planned sample. However, I've strata as high as 95% and as low as 47% response rate. To ...
0
votes
0answers
16 views

Listwise deletion then imputation?

I have a data set described in this post and as mentioned I have two predictors with 25% and 20% missingness that is partially due to the fact that they can only be measured when they are above ...
3
votes
1answer
47 views

How to deal with dropouts from a waiting list control group?

Many treatment studies compare a treatment group with a waiting list control group, for example to adjust for spontaneous remissions. Unfortunately, many more participants drop out from the waiting ...
0
votes
0answers
15 views

Distinguishing between zeros and missing data

I have a panel data set with 12,000 observations of daily counts of visitors to a number of recreational sites. The data has been given to me with missing values recorded as zeroes. There are also ...
0
votes
1answer
44 views

Problems with Missing values

I have a data set for a predictive model(predicting survival rate with certain acute medical condition on some animals) with 25 predictors where around 30% of the predictors are complete, 3 predictors ...
0
votes
2answers
33 views

Should I convert raw data into growth rates if there are gaps in my data?

I have data from 214 countries that range from 1990 to 2014. My dependent variables (I'm doing more than one regression) are just primary/secondary net enrollment rates for both sexes/males/females, ...
0
votes
0answers
17 views

How to compare contributory values over time? (with missing values)

Context I am working on a project considering computers. I have a dataset containing around 50 product characteristics for each computer. Also, for each computer, I have data about the product price ...
1
vote
1answer
21 views

How to handle interruptions (0 values) over a certain period in time series analysis?

Is there a way that I can combine two models for time series? I am trying to predict the production of tomatoes per week per $m^2$ (black line), based on light (orange) and temperature (magenta). At ...
0
votes
0answers
27 views

May I average over multiple observations within one bin of longitudinal data to alleviate the effect of unequal observations numbers?

I have a weekly assessment of a certain continuous variable (DV) (e.g. heart rate), but unfortunately a lot of missing data, because the data comes from a clinical population. After seeing, that ...
1
vote
0answers
19 views

Unbalanced two-factor repeated measures ANOVA with missing values

For my data set, I need to perform some sort of two factor repeated measures ANOVA. I have one between-subject factor called "Treatment" and one within-subject factor called "Frequency" with 8 levels. ...
0
votes
1answer
27 views

NA in ARIMA model, is it suggest that the model is over fit?

my professor says that if we see that there are NA values for the AR or MA terms(either the estimated values or the estimated se) in the R output for the arima models fitted using the arima() ...
1
vote
1answer
24 views

How many multiple imputation datasets should we make?

Multiple imputation is based on making m different datasets and analyzing them each independently then aggregating over all their information as a whole. How many imputed datasets should we make?
0
votes
1answer
34 views

building up a predictive model with lots of features and missing data

I'm learning using R to build predictive models recently by myself and have many questions on how to attack a question. I'm given a data set of 8000 observations with 300 features. My goal is to build ...
3
votes
2answers
98 views

Meaning of “missing by design” in longitudinal studies

I'm French and I'm reading an English book. I don't understand the term "when missingness is by design" — what does "by design" mean?
3
votes
0answers
31 views

Missing value imputation with nearest neighbour

I'm using k-nearest neighbour imputation method for missing values. This method has two tuning parameters: k and the distance metric. I see two options for applying this imputation method: Inside ...
1
vote
1answer
27 views

Relationships of missing values for exploratory analysis

I have a survey with 30 questions on a seven item likert scale Not all of the questions were answered. I can use a heat map to visualize the missing answers but what i would like to know is if there ...
0
votes
0answers
43 views

Multiple imputation for predictive analysis using mice package in R

I am using the mice package to impute some missing values, and it works nicely. I am facing a tricky strategic question though. Basically, I am working on predictors of myocardial infarction (time ...
0
votes
1answer
50 views

K-nearest neighbour imputation of missing values

I have a dataset where the columns correspond to features and the rows correspond to data points. I have around 5'000 data points and 8 features. Now, I would like to impute the missing values with ...
0
votes
1answer
49 views

How to handle empty data in linear regression?

This question may seem like a duplicate but I haven't found quite another one that fits my case. So I am trying to run a linear regression on some variables including 1 dichotomous IV (Tool) that is ...
0
votes
0answers
12 views

Comparing species diversity between sites when some sites have no species present

I've been given stream sampling data that consist of number of species and number of individual fish captured at sampling points above and below stream crossings. Our goal is to determine what factors ...
0
votes
1answer
21 views

Categorical Variable in Training Set does not capture all cases

I am training a predictive model on a training data set, which includes zipcode as one of the predictors. Since zipcode is nominal, I treat it as categorical variable and try to dummify it. The ...
1
vote
0answers
39 views

handling missing input variable for machine learning

In building feature-sets for a machine learning algorithm, I'm facing a situation where the input variable - which is a numeric variable, may or may not appear. What I mean is, the data set I'm ...
0
votes
0answers
17 views

How to handle unaswered items in a likert scale questionnaire?

I have a 20 items quesitionnaire, using a likert scale for anwers (strongly agree/strongly disagree). The questionnaire is ablout getting the opinion of participants about a topic. A few participants ...
1
vote
0answers
46 views

Checking for Missingness mechanism

This code is for simulating missing mechanism randomly that I extracted it from imputeR package : ...
3
votes
1answer
32 views

Nonparametric 2-way ANOVA for data with unequal n's due to possibly informative missingness

So my problem revolves around trying to find the right test for my data in R. I've been doing an experiment that is measuring sublethal effects in a toxic environment, and as a result, even though I ...
3
votes
1answer
51 views

When does the marginal MLE converge to the complete data MLE?

What I mean by the title is suppose we have a distribution $p(x,z\;|\;\theta)$, where the $x$ are observed and each $x_i$ depends on a hidden $z_i$. Then the marginal MLE is given by ...
0
votes
1answer
23 views

Number of missing data to be replaced by the average

One of the missing data processing methods is replacing them by the average. This method is applied watever is the numbre of missing data per subject? Or there is a number not to be ...
0
votes
2answers
62 views

Estimating failure-point when failed datapoints are unobservable

I have several datasets of two variables, y and time. Datapoints are objects subject to physical wear, and wear-rates are (roughly) constant over the life of the objects. y is defined as the ...
0
votes
0answers
19 views

Mixed effects model with missing count data

I have bird count data over a period of ten years from 15 different sites. For each site I have the month of the count, the year of the count, and the count number of birds. I need to analyse the data ...
0
votes
1answer
84 views

How to apply a model on dataset with missing data?

This question is similar to Missing input value during prediction of a generalized linear model. Consider the following scenario: I fitted a linear regression model on a training dataset with ...
0
votes
0answers
13 views

Pattern-mixture models

I am currently looking at pattern-mixture models but I don't see to understand them and I wonder if someone could help. I can see the model comes from the factorisation $ f(y,r;\phi, ...
0
votes
0answers
25 views

Filling incomplete data & forecasting in multivariate time series per kalman filter

I have a large set of timeseries, some with missing values. I want to fill in missing values. A given missing value in a column should be filled up conditional on the contemporaneous value of all ...
1
vote
2answers
34 views

handling many missing values within a regression

I am struggling with answering a question on how i should handle the vast number of NAs in my data. It is a behavioural study of the impact of traffic on certain mammals and i have approximately 500 ...
0
votes
1answer
69 views

Missing Data (mice) and Survey Package r

First, I am new to analyzing public opinion polls and the r package "Survey". I would like some advice. I am running a regression model with weights from a Pew survey, however, I noticed that a ...
0
votes
0answers
73 views

Difference between imputation and interpolation?

When dealing with data sets that have missing values, imputation replaces missing values with substituted values while interpolation replaces missing values with calculated values within some range. ...
2
votes
0answers
47 views

What statistical models / approaches can I use to estimate missing hourly values?

My dataset consists of hourly values by weekday across several sites, where the sites vary by spatial location and by other common characteristics, such as type, or 'cafe,' 'restaurant,' and 'bar.' ...
1
vote
0answers
39 views

Variable importance in regression with large number of missing values

I have a dataset with multiple (approximately 20) categorical and ordinal predictors and a numerical outcome and I am trying to understand which and how each of these predictors affect the outcome ...
0
votes
0answers
24 views

Calculating Objects Scores in CATPCA when you have missing values?

I am doing a survey for my workplace and I am analysing the Likert questions using a CATPCA. The problem I have is that the respondents were permitted to answer 'N/A' or 'I don't know' which have been ...
0
votes
0answers
27 views

CATPCA Missing values

I have posted a question on stackexchange and you answered me very well before. Thank you. I have another CATPCA question and I want to ask to find out how missing values are used if you select ...
0
votes
2answers
74 views

Framework of multiple imputation

I read this paper about ("Multiple Imputation For Missing Data: What Is It And How Can I Use It?")(http://www.csos.jhu.edu/contact/staff/jwayman_pub/wayman_multimp_aera2003.pdf) Does any one have ...
0
votes
2answers
69 views

Change mean imputation in MICE package

MICE Steps The chained equation process can be broken down into six general steps: Step 1: A simple imputation, such as imputing the mean, is performed for every missing value in the dataset. These ...
0
votes
1answer
37 views

How to handle missing data for prediction with small data set

I have to analyse the results of american football players. The goal is to predict the position group based on the results of about 20 exercises. Therefore I use SVM, Neural Network, Decision Tree, ...
0
votes
0answers
26 views

Minimize coefficient bias in regression with effects coded categorical variables where data is unbalanced and missing

I have a data set with two categorical variables that are effects coded. 6 out of 18 observations do not have records for the first categorical variable. 12 out of 18 observations do not have records ...
0
votes
0answers
82 views

Recommendation System in Python - weighted average with negative factors

I am building a recommendation system in python on the 100k movielens dataset. My code build a movie/user matrix where the (x,y) element is the rate that the user y gave to the movie x. Since i want ...
0
votes
0answers
31 views

How estimate Autoregressive model of order 2 on dataframe with missing values and multiple columns in R

I have a dataframe with 4000 companies, each company present as a column in the dataframe. The complexity of my dataframe is that companies belong to a stock exchange and all companies that had been ...
1
vote
0answers
28 views

Using MICE with Random forests taking into account clustering

I am using the mice package in R to create multivariate imputed datasets. For this, I am using mice(data, meth= "rf")function. I ...