When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

1
vote
1answer
31 views

How does one treat censored data in SAS?

I have some censored data and I'm not sure how to deal with it in my regression analysis. The study was not a time series and all examples I've seen in SAS have been in the context of survival ...
0
votes
0answers
13 views

How to fix dim(X) error in MICE (R)? [on hold]

I am trying to compute multiple imputation using the MICE package in R. I am working with a large scale study which includes 38 countries. For most countries I get reasonable results, but for some I ...
1
vote
0answers
11 views

Variance of sum of datasets with different sampling intervals

I have two randomly distributed datasets which are to be added together element-wise, and I am looking to calculate the uncertainty of the mean of the result. One dataset had a sampling rate of 1 ...
2
votes
1answer
26 views

Random Forest and missing values in numeric features

I'd like to use a random forest for predicting how long a person will stay a customer of our company. One feature I'd like to use is the average age of the customer's kids. The problem is some ...
0
votes
1answer
21 views

two-way anova when lacking one observation

I want to perform a two-way ANOVA in a data frame that lacks one observation. I have values for each (month,year) pair, and I am performing a two-way ANOVA to check if moving seasonality exists. ...
0
votes
0answers
17 views

How to handle data with 2 variables that have same missingness pattern?

I've not had much academic coursework on imputation, and I can't find anything online or in any texts regarding how one could handle missing data where there are two (or more, possibly?) variables ...
1
vote
1answer
43 views

How do I implement missing value patterns?

I have a training data set and I was able to find some interesting patterns in the missing values, and I used binary variables in order to represent the missingness. I am going to train a model, say a ...
1
vote
2answers
16 views

Hypothesis test for mutually exclusive samples from a single population?

Let's say I gave 100,000 people a survey on their favorite fruit, where they can bubble in "Apple" or "Pear" (but not both). They can also leave it blank. The results are 90,000 people left it blank, ...
0
votes
0answers
8 views

can I add a subgroup-characteristic to a model?

I'm doing a regression analysis in order to find out how two groups of people vary on an outcome variable. To do so I have number of independent variables and a dummy variable for the two groups. For ...
0
votes
0answers
12 views

how do I split my data for EM?

I'm going to use EM to replace my missing values, as my missing data is MCAR and less than 1% of the total. My variables are from several measures, taken at two time points, and from both a parent and ...
1
vote
0answers
11 views

Finding multicollinearity from data with missing values

I have a data frame with around ~30 columns and at least 20 features have missing data in between 60-70%. I am wondering if it is possible to calculate multicollinearity in this case. If yes, how? My ...
1
vote
0answers
20 views

Treatment of data that is contingent upon earlier item in multilevel modeling?

How should data be treated in multilevel modeling when they are not available because they are contingent upon how a previous question is answered? For example, if the participant answers "no" on ...
0
votes
0answers
21 views

How to deal with, and use, missing data (MNAR) in svm?

I am trying to predict future spending of customers based on past transactions. My data looks as follows: ...
1
vote
0answers
24 views

Is there a way to smooth cohort demography data in R? [closed]

I have a set of mortality data that I'd like to smooth so that I can run it through a Lee-Carter model for forecasting. The set of data focuses on the cohorts of aged 1-11 people for the years of ...
0
votes
0answers
9 views

Representative signal

I'm implementing machine learning with sensor data. I am having the problem that some sensors not always have good integrity, that is, not all data points arrive at destination because of ...
0
votes
0answers
50 views

Imputation in R: How to impute univariate data in R?

I am trying classification(2 classes) using Random Forest. Classes are - Red, Green. My dataset contains 1 numeric attributes(called X), and 51 binary attributes to classify a document into red and ...
0
votes
0answers
12 views

Imputation in R: How to impute univariate data in R? [duplicate]

I am trying to classify(2) using Random Forest. Dataset contains 1 numeric, and 51 binary attributes. Numeric data is essential(Blood Pressure) for the classification. However, 40% instances do not ...
1
vote
0answers
20 views

Orthogonal polynomials contrasts for regression (unbalanced)

We can obtain the sum of squares of a contrast for a regression of degree $j$ by: $$ SSR_j=\frac{\left(\displaystyle\sum_{i=1}^{I} C_{ji}T_i\right)^2}{rK_j}, $$ where $I$ is the number of levels of ...
0
votes
1answer
83 views

Applying Rubin's rule for combining multiply imputed datasets

I am hoping to pool the results of a pretty basic set of analysis performed on a multiply imputed data (e.g. multiple regression, ANOVA). Multiple imputation and the analyses have been completed in ...
1
vote
1answer
18 views

Univariate analyses with incomplete data set

I'm dealing with a fairly large ecological data set (field data, not collected by me) of approximately 60 attributes measured for about 400 individual trees. Very few trees (~20%) have complete data ...
0
votes
1answer
24 views

Regression imputation of missing data

Suppose a two-way experiment with interaction. Is it correct to estimate the missing values by OLS, input those values in the data (fill the blanks) and now perform a polynomial (or any kind of) ...
0
votes
1answer
44 views

Regression imputation of missing data based on OLS effects

Let's say we have a two-way with interaction experiment with missing data. Being the dataset: ...
1
vote
0answers
6 views

how can I do a 3-way repeated measures ANOVA on filtered data?

I have a sample of participants tested on a two-day memory task where they learn the task on day 1, then are tested on day 2. They are Parkinson's disease patients and are tested on or off their ...
1
vote
1answer
27 views

Index-variable as an independent variable

In my regression on gdp-growth, I also want to bring in something like a "freedom"-variable, to show how free a country is (press freedom, economic freedom). now there is no number for this, except ...
0
votes
0answers
10 views

Missing block (Multilevel Modelling, repeated measures)

Experiment: participants respond to a moving object. Explanatory variables: velocity of object (fast, slow), Type of object (A,B). There are four different types of blocks, each with 16 trials: ...
6
votes
1answer
275 views

Data visualization for missing data

I am a designer and am trying to plot a set of data over time. For example, Day1 Day2 Day3 Day4 Day5 10 53 21 67 38 ...
1
vote
0answers
28 views

How to deal with empty values in a cluster analysis

I'm currently working on my master's thesis. Part of the work is a customer segmentation by means of a cluster analysis. One variable for the cluster determination should be the chronological ...
1
vote
0answers
23 views

Fit multiple regression model with pairwise deletion (or on a correlation/covariance matrix) in R

I'm trying to fit a multiple regression model with pairwise deletion in the context of missing data. lm() uses listwise deletion, which I'd prefer not to use in my ...
0
votes
1answer
17 views

Interpolation model to estimate missing analytics

We have about 7 months of partially (30%) missing web analytics, that is apparently missing at random across all segmentations. We need to estimate the missing data to correctly compare current and ...
1
vote
2answers
39 views

EM algorithm to impute missing value for one variable

This is from Robert Hogg's Introduction to Mathematical Statistics 6th, exercise 6.6.5. p366, It says, Suppose $X_1$, $X_2$, $X_{n1}$, are a random sample from a $N(\theta,1)$ distribution. Suppose ...
0
votes
0answers
27 views

How is spline computed in simple time series in AMELIA

I am trying to use Amelia to perform missing data imputation in a simple time series where time is represented as minutes and each measurement is replicated 4 times. ...
0
votes
0answers
10 views

multiple imputation - likelihood base

in nonignorable mechanism, selection model or pattern-mixture model is a multiple imputation method or a likelihood-base method? I am confused, i know in MI, missing data were filled in and then We ...
1
vote
0answers
7 views

Affinity Propagation with missing data

I'm using Affinity Propagation to cluster some data, and I have to deal with missing values, so for points that I have the data, I can use it to change similarity between them, but for those that I ...
4
votes
1answer
41 views

Is precision-based (i.e. inverse-variance) weighting integral to meta-analysis?

Is precision-based weighting central to meta-analysis? Borenstein et al. (2009) write that for meta-analysis to be possible all that is necessary is that: Studies report a point estimate which can ...
3
votes
0answers
25 views

Planned missing data design not converging

I recently ran an experimental study with a planned missing design. Participants were randomly assigned to one of four groups that dictated which portion of a 26-item unidimensional measure they ...
0
votes
1answer
31 views

Comparing categorical outcome variables in repeated measures design

I am working on an observational prospective longitudinal study with a repeated measures design. The same categorical outcome is measured for five times, for all the participants, over a time period. ...
3
votes
0answers
45 views

Alternate weighting schemes for random effects meta-analysis: missing standard deviations

I am working on a random effects meta-analysis covering a number of studies which do not report standard deviations; all studies do report sample size. I do not believe it is possible to approximate ...
0
votes
0answers
14 views

What is the best practice to deal with NA values when calculating a dissimilarity matrix?

I need to calculate a matrix of distances between sites where different variables were measured. I will use it in a cluster analysis. The following is a sample of the matrix I am dealing with: ...
0
votes
0answers
29 views

Machine Learning when missing data is state-dependent (e.g. adaptive questionnaires)

I have the following problem: I am dealing with an adaptive questionnaire, meaning a questionnaire where there are questions that are only asked when a previous question had a specific answer. The ...
0
votes
0answers
23 views

Natural logarithm transfomation and zeroes [duplicate]

I am using Stata 13 to estimate a simple regression. Given a rather positive skew of a few of my covariates, I figured to ln-transform the variables. However, I have a substantial amount of zeroes in ...
1
vote
1answer
24 views

Multiple imputation in SAS for longitudinal data

I have a data set from a repeated measurement study comparing two groups with missing data due to lost-to-follow-up (~20%). I know how to apply multiple imputation method for cross-sectional data. ...
3
votes
1answer
47 views

Mixed-modeling when no observation contributes both X and Y

I'm working on a project investigating the relationship between (let's say) a face's perceived masculinity and its perceived competence. There was a large number of face stimuli (80). Two completely ...
0
votes
0answers
24 views

r package: CFA and missing data

I am running confirmatory factor analysis (CFA) with r. As I have several missing observations, I get two series of results, on "used" (152) and "total" ...
1
vote
0answers
37 views

How to treat missing values in a regression?

I have used the logarithmic form of wage as my independent variable in Stata. However, it contains missing values. Should I replace these with 0 or let them be?
2
votes
1answer
42 views

Is maximum likelihood a form of data substitution? Or not?

I’m using maximum likelihood with missing data. In this case of missing data, is maximum likelihood a form of data substitution? I’m significantly more familiar with multiple imputation which I ...
0
votes
2answers
85 views

How to handle NA values in shrinkage (Lasso) method using glmnet

I'm using "glmnet" for lasso regression in GWAS. Some variants and individuals have missing values and it seems that glmnet cannot handle missing values. Is there any solution for this? or is there ...
3
votes
1answer
51 views

Multiple Imputation methods

Suppose that a variable $Y_j$ has missing values. We can use regression to impute the data using the nonmissing observations: $$Y_j = \beta_0+\beta_{1}Y_{1}+\beta_{2}Y_{2} + \dots + \beta_{(j-1)} ...
0
votes
3answers
110 views

How to use Random Forest for categorical variables with missing value

I have a labelled dataset of 1M rows and 600 features. I am trying to build a supervised learning model for prediction. I am particularly working with Random forests in R.The data I have has following ...
2
votes
1answer
45 views

Prove the loglikelihood is strictly concave for ABO allele frequency blood type data

I am working through the problems in Kenn Lange's book Numerical Analysis for Statisticians. I am going to try and do all of the problems in the book, though none of them are specifically assigned for ...
-1
votes
1answer
61 views

rbinom() produces NA values. What's wrong?

I have to generate RV from a binomial distribution with R . I have a vector for $n$ and a vector for $p$ and for each component I have to generate a random variable. My idea was the following for ...