When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

0
votes
0answers
6 views

Pattern-mixture models

I am currently looking at pattern-mixture models but I don't see to understand them and I wonder if someone could help. I can see the model comes from the factorisation $ f(y,r;\phi, ...
0
votes
0answers
8 views

Filling incomplete data & forecasting in multivariate time series per kalman filter

I have a large set of timeseries, some with missing values. I want to fill in missing values. A given missing value in a column should be filled up conditional on the contemporaneous value of all ...
0
votes
0answers
26 views

Missing Data (mice) and Survey Package r

First, I am new to analyzing public opinion polls and the r package "Survey". I would like some advice. I am running a regression model with weights from a Pew survey, however, I noticed that a ...
0
votes
0answers
21 views

Difference between imputation and interpolation?

When dealing with data sets that have missing values, imputation replaces missing values with substituted values while interpolation replaces missing values with calculated values within some range. ...
2
votes
0answers
45 views

What statistical models / approaches can I use to estimate missing hourly values?

My dataset consists of hourly values by weekday across several sites, where the sites vary by spatial location and by other common characteristics, such as type, or 'cafe,' 'restaurant,' and 'bar.' ...
1
vote
0answers
20 views

Variable importance in regression with large number of missing values

I have a dataset with multiple (approximately 20) categorical and ordinal predictors and a numerical outcome and I am trying to understand which and how each of these predictors affect the outcome ...
0
votes
0answers
22 views

Calculating Objects Scores in CATPCA when you have missing values?

I am doing a survey for my workplace and I am analysing the Likert questions using a CATPCA. The problem I have is that the respondents were permitted to answer 'N/A' or 'I don't know' which have been ...
0
votes
0answers
25 views

CATPCA Missing values

I have posted a question on stackexchange and you answered me very well before. Thank you. I have another CATPCA question and I want to ask to find out how missing values are used if you select ...
0
votes
2answers
55 views

Framework of multiple imputation

I read this paper about ("Multiple Imputation For Missing Data: What Is It And How Can I Use It?")(http://www.csos.jhu.edu/contact/staff/jwayman_pub/wayman_multimp_aera2003.pdf) Does any one have ...
0
votes
2answers
57 views

Change mean imputation in MICE package

MICE Steps The chained equation process can be broken down into six general steps: Step 1: A simple imputation, such as imputing the mean, is performed for every missing value in the dataset. These ...
0
votes
1answer
29 views

How to handle missing data for prediction with small data set

I have to analyse the results of american football players. The goal is to predict the position group based on the results of about 20 exercises. Therefore I use SVM, Neural Network, Decision Tree, ...
0
votes
0answers
15 views

Minimize coefficient bias in regression with effects coded categorical variables where data is unbalanced and missing

I have a data set with two categorical variables that are effects coded. 6 out of 18 observations do not have records for the first categorical variable. 12 out of 18 observations do not have records ...
0
votes
0answers
25 views

Recommendation System in Python - weighted average with negative factors

I am building a recommendation system in python on the 100k movielens dataset. My code build a movie/user matrix where the (x,y) element is the rate that the user y gave to the movie x. Since i want ...
0
votes
0answers
18 views

How to fit Autoregressive model with 2 lags on dataframe with missing values and multiple columns in R

I have a dataframe with 4000 companies, each company present as a column in the dataframe. The complexity of my dataframe is that companies belong to a stock exchange and all companies that had been ...
1
vote
0answers
17 views

Using MICE with Random forests taking into account clustering

I am using the mice package in R to create multivariate imputed datasets. For this, I am using mice(data, meth= "rf")function. I ...
0
votes
0answers
11 views

Using multiple imputation followed by repeated measures

I have missing data that I have done multiple imputation with. I want to then use the means or 'pooled' data from the five imputations to do a repeated measures ANOVA. It seems I can't do this in ...
1
vote
0answers
42 views

Using entropy to imputing missing value based on grey relational analysis and clustering

This algorithm contain three techniques : 1-fuzzy c-mean clustering 2-Grey relational theory 3-Entropy multiple imputation The frame work of this algorithm is as follows : My questions are ...
0
votes
0answers
14 views

Assess the variability of mis-classification errors over many imputed data set

I am conducting logistic regression analysis: The data includes 107 observations, dependent variable is a binary one, there is about 5 covariates which are both continuous, binary and ...
1
vote
0answers
35 views

Pairwise vs listwise deletion of missing data in regression?

I was wondering what would be the difference between using the pairwise versus the listwise option in a multiple regression? I have a dependent variable (reaction time) and several predictors ...
0
votes
2answers
31 views

Using missing data

I conducted a web-based survey. Started with 1000+ respondents and it gradually reduced to 400+. Some key questions were answered, for example key question re dependent variable is answered by N=750. ...
0
votes
0answers
28 views

what to do with missing value for ANOVA?

A researcher sent me a data set. The Y is intensity. X is A,B,C,D,E 5 treatments. A-D treatments all have 5 samples. However, E treatment only has 4 samples. When I ask the researcher about it, she ...
0
votes
1answer
97 views

How to know which imputation is best for impute my dataset from Multiple imputation by using mice

I used mice package to impute the missing value as follows: ...
3
votes
1answer
49 views

What is the motivation for the entropy term in the proof of EM algorithm?

Reading through the proof that EM algorithm monotonically increases the log-likelihood (until it converges), I noticed that the most important ingredient of the proof is the introduction of an entropy ...
1
vote
0answers
31 views

Calculating median for different subsets and sample sizes

I have a basic question about calculating median. The database I'm using contains registered records of criminals and the number of committed crimes per year each has committed in country X. Based on ...
0
votes
0answers
20 views

SAS HELP: Imputing both continuous and categorical variables in a single dataset

I am trying to impute data for missing observations within a longitudinal dataset following an arbitrary missing data pattern. Both continuous and categorical variables have missing data. I am ...
1
vote
1answer
52 views

Estimate missing data in time series

I am currently conducting a study regarding lifelong learning in EU member states. As part of this research, I analyze several indicators for the following time series: 2007, 2011 and 2013. I have ...
0
votes
0answers
13 views

Using known and complete data to predict data part of which is know and the rest is unknown

It is a general problem. I have a training set(size > 1000). Each data point has 1000 features. What I want to do is to use these training data to complete a data point which has 600 known features ...
0
votes
1answer
9 views

I would like to know whether expectation maximization is relevant to cost optimization imbalance

I have a cost matrix which has probability confusion matrix Here is the cost predict good-actually good: 0 predict good-actually bad: 3 consequence points (negative) predict bad-actually bad: 0 ...
0
votes
1answer
14 views

Missing at Random (MAR) and Ignorability

My professor told us in class that missing at random (MAR) data is generally ignorable. What is a case where missing data is one and not the other?
0
votes
0answers
36 views

Best ways to visualize multivariate data with missingness

I have a huge dataset that is multivariate, but not all the samples have data assigned in certain variables. To be more specific, I have data of microbial contamination (in logs) of milk (in two ...
1
vote
0answers
9 views

How much missing data is too much for FIML?

I am running a path model with 8 variables with missing data. I use full information maximum likelihood estimation. I have only a covariance coverage of 50-60% on some variables. I wondered if there ...
0
votes
0answers
19 views

Missing data associated with group

I am interested in estimating parameters and predicting y using a standard regression model: $y = \beta_0 + x_1\beta_1 + x_2\beta_2$ The data that I have consists of a population of two groups A and ...
1
vote
2answers
28 views

How to apply Wilcoxon test to small sample with NAs?

I have the following data which show some values before and after a drug. I want to test whether the drug increased the mean. I think I should use Wilcoxon test, but I don't know what to do with ...
0
votes
0answers
8 views

Can i check unit root of series having missing values with eviews or any other software?

I am trying to find the impact of selected expenditure on selected economic growth determinants for 6 developing countries from 1990 to 2013 . Some of the data in the series is missing. So, i want to ...
2
votes
1answer
149 views

How to simulate the different types of missing data

How do you create a missingness mechanism (MAR, MCAR, NMAR)? Can you generate it directly or do you do it by a model?
1
vote
1answer
74 views

Treating missing values in panel data set

You can often see in the empirical papers that do some kind of the regression analysis on economic data something like this "we drop the companies from our sample which do not have the observation in ...
0
votes
1answer
16 views

Handling large amount of missing values in categorical variables.

I have a binary Classification problem and I have a dataset with lot of categorical variables and many of them have missing information. I proceeded with dropping all the missing values from the ...
2
votes
1answer
63 views

Impute missing data before or after feature selection?

Will the results of the feature selection be biased if I perform the feature selection before imputing missing data? I have a large data set of 20000 samples and 130 variables. The data sets ...
0
votes
0answers
12 views

Excluding cases before or after calculating mean-centered variables?

I stumbled upon the following problem and was wondering how to address is adequately: I have a large dataset with around 60 groups, each of them with 4-6 people. For my study, each participant's ...
0
votes
0answers
17 views

How to estimate parameters of data sets that have different missing value type with 3PL model in R?

I had a 10 complete dichotomous (1-0) data responses which were responded by 100 person and I created with it three data sets that each of them has different type missing value (MCAR, MAR and MNAR). ...
0
votes
0answers
14 views

Computing design matrix from covariance matrix

Suppose I have the following regression model: $Y = b_1 \times T + b_2 \times Z + b_3 \times T\times Z + \epsilon$ where T is a randomly assigned treatment condition and Z is some covariate. I want ...
0
votes
0answers
32 views

Should 0.5 be used as a feature value in cases when it is not calculable (its range is in [0,1])?

I have a training dataset of positive and negative examples for a certain class. The problem is that, in some cases, the value of a particular feature is not calculable. So, since this feature ...
3
votes
1answer
158 views

ML covariance estimation from Expectation-Maximization with missing data

Assuming a multivariate normal distribution with missing data, is there a straightforward way to find the maximum likelihood estimate for covariance using an Expectation-Maximization algorithm? NOTE: ...
1
vote
0answers
53 views

Model multiple imputation with interaction terms

According to the documentation of the mice package, if we want to impute data when we're interested in interaction terms we need to use passive imputation. Which is ...
1
vote
0answers
21 views

Mis-aligned, censored, and bounded-error hierarchical annual time series

My job involves analyzing air pollutant emission estimates. I have a dataset of 5 timeseries, for which I would like to both backcast and forecast aggregate statistics (e.g. a range of expectation for ...
0
votes
0answers
6 views

non-administered as missing data?

I have data set which have student motivation, learning strategies, and the classroom climate. However, there are three different forms of student questionnaire. That means not every student answers ...
2
votes
0answers
25 views

Interpolating missing time-series data

I have time-series for creatinine levels in patients, which has missing samples, due to patients' irregular visits to doctors. The figure below represents the time-series for a patient. Task: I ...
2
votes
0answers
24 views

Can last observation carried forward (LOCF) be used in a meta analysis?

In an article I am reviewing, the authors used LOCF for missing data. In ordinary studies, I would frown on this and suggest multiple imputation. However, this study was a meta-analysis. Is LOCF an ...
0
votes
0answers
36 views

Non-negative forecasts with missing data and clustering

I have a data set of deposits and withdrawals from bank locations, so each record includes a bank identifier, date stamp, number of deposits, and number of withdrawals. I have included reproducible ...
2
votes
0answers
64 views

Does Little's MCAR test make sense?

Now I'm reading Applied Missing Data Analysis by Craig K. Enders ,and I try to understand why we do Little's MCAR test ? . On page 17 he wrote the following : In truth, testing whether an entire ...