When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

0
votes
1answer
12 views

How SAS handles missing values while fitting linear regression model

While fitting a linear regression model in SAS, if the dataset has missing values (either missing y, or missing x or both), will SAS just ignore the records that have at least one missing value(y,x)? ...
1
vote
0answers
17 views

Classify streaming, partially complete data into groups defined by prior clustering

Suppose I have M observation vectors, offline, $y_t$, $ t =1 ... M$, and each observation is $n$ dimensional. I then cluster these observations into $k$ clusters. For computing the clustering ...
0
votes
1answer
13 views

Variance and autocorrelation with missing and/or unevenly spaced data in time series

This question concerns the general problem of working with data that might have missing and/or unevenly spaced values. Let’s call this real data. Specifically I am calculating rolling variance and ...
0
votes
0answers
17 views

Likelihood function for Missing data [on hold]

I am looking for information about likelihood function for missing data. I was wondering you could help me.
0
votes
0answers
14 views

sapply with function using multiple columns as input [migrated]

I have a data frame with let's say 2 columns and 4 rows (it's bigger... I am just making it simpler) like this: Value: 0.2, 0.3, 0.5, 0.8 R: 1, 0, 1, 0 I m trying to write a sapply line that given ...
1
vote
0answers
14 views

Cox PH model set up when some variables only measured occasionally

I am working on an analysis of observational data over 16 years. I'm attempting to conduct a Cox proportional hazards regression but am not sure my data set is in the correct format. I have about 10 ...
2
votes
1answer
33 views

Can we estimate a missing Y value from a three factor experiment?

In preparation for an examination I'm looking at some old questions, and I could use a little help with figuring out what to do here: In a three-factor experiment the researchers lost the results ...
1
vote
0answers
17 views

Miss Forest & Iterative PCA : How to handle very sparse matrix imputation?

I am currently benchmarking matrix completion methods (k-NN, RandomForest and iterative PCA) on multivariate normal data in which I introduce a certain proportion of NA (5 to 95%). My performance ...
0
votes
0answers
7 views

How to use Little's MCAR test with multiple scales?

I'm using SPSS and have a data set which includes lots of different scales and there is missing data across all of the scales. I want to know if the data is missing completely at random. I intend to ...
0
votes
0answers
6 views

How to fill in the missing labels which have more than 1500 levels

I have a question regarding "classification". I have a data set with the target label of more than 1500 nominal categories (there is no way to bucket them as they are just individual strings). The ...
0
votes
0answers
11 views

Should I perform Little's MCAR test and estimate missing values on each individual scale or on all of the data combined?

I have 6 different scales/variables (relationship quality, self-concept, physical self-concept, depressive symptoms, anxiety symptoms, illness acceptance), so I am unsure if I should calculate and ...
0
votes
0answers
15 views

Clustering pairwise comparisons between samples of sparse binary data

I have 100 samples, like so: pos 1 1: 3 0 2: 4 1 3: 5 1 4: 7 1 5: 8 0 --- 196: 489 1 197: 490 1 198: 492 1 199: 495 0 200: 498 0 ...
1
vote
1answer
36 views

Testing MAR assumption knowing the missing data

I am creating some artificial missing values in a dataset using the 2 well known mechanisms MAR and NMAR. I want to validate what I create, but I cant find any statistical test that given the ...
1
vote
0answers
14 views

Missing value in categorical variables and logistic regression

I have a data set on a disease and weight change (sample size=100). Both variables are categorical (disease=Yes/No, weight change= Yes/No). In weight change variable there is 40% missing observation. ...
2
votes
1answer
29 views

In what way could MNAR data affect the results of correlations?

I am studying the effects of exercise non-adherence in a pain sample (n=70). Pearson's correlations showed a moderate, negative, relationship between baseline pain and adherence behaviour at 3 ...
1
vote
0answers
11 views

Effective Linear Regression for datasets with missing values in explanatory (independent) variables

I have an econometric dataset of countries consisting of features such as GDP, GDP per capita, internet penetration rate, life expectancy, poverty etc. There are a ...
1
vote
1answer
36 views

How does one treat censored data in SAS?

I have some censored data and I'm not sure how to deal with it in my regression analysis. The study was not a time series and all examples I've seen in SAS have been in the context of survival ...
1
vote
0answers
14 views

Variance of sum of datasets with different sampling intervals

I have two randomly distributed datasets which are to be added together element-wise, and I am looking to calculate the uncertainty of the mean of the result. One dataset had a sampling rate of 1 ...
2
votes
1answer
27 views

Random Forest and missing values in numeric features

I'd like to use a random forest for predicting how long a person will stay a customer of our company. One feature I'd like to use is the average age of the customer's kids. The problem is some ...
0
votes
1answer
30 views

two-way anova when lacking one observation

I want to perform a two-way ANOVA in a data frame that lacks one observation. I have values for each (month,year) pair, and I am performing a two-way ANOVA to check if moving seasonality exists. ...
0
votes
1answer
22 views

How to handle data with 2 variables that have same missingness pattern?

I've not had much academic coursework on imputation, and I can't find anything online or in any texts regarding how one could handle missing data where there are two (or more, possibly?) variables ...
1
vote
1answer
44 views

How do I implement missing value patterns?

I have a training data set and I was able to find some interesting patterns in the missing values, and I used binary variables in order to represent the missingness. I am going to train a model, say a ...
1
vote
2answers
23 views

Hypothesis test for mutually exclusive samples from a single population?

Let's say I gave 100,000 people a survey on their favorite fruit, where they can bubble in "Apple" or "Pear" (but not both). They can also leave it blank. The results are 90,000 people left it blank, ...
0
votes
0answers
8 views

can I add a subgroup-characteristic to a model?

I'm doing a regression analysis in order to find out how two groups of people vary on an outcome variable. To do so I have number of independent variables and a dummy variable for the two groups. For ...
0
votes
0answers
12 views

how do I split my data for EM?

I'm going to use EM to replace my missing values, as my missing data is MCAR and less than 1% of the total. My variables are from several measures, taken at two time points, and from both a parent and ...
1
vote
0answers
11 views

Finding multicollinearity from data with missing values

I have a data frame with around ~30 columns and at least 20 features have missing data in between 60-70%. I am wondering if it is possible to calculate multicollinearity in this case. If yes, how? My ...
1
vote
0answers
20 views

Treatment of data that is contingent upon earlier item in multilevel modeling?

How should data be treated in multilevel modeling when they are not available because they are contingent upon how a previous question is answered? For example, if the participant answers "no" on ...
0
votes
0answers
45 views

How to deal with, and use, missing data (MNAR) in svm?

I am trying to predict future spending of customers based on past transactions. My data looks as follows: ...
1
vote
0answers
33 views

Is there a way to smooth cohort demography data in R? [closed]

I have a set of mortality data that I'd like to smooth so that I can run it through a Lee-Carter model for forecasting. The set of data focuses on the cohorts of aged 1-11 people for the years of ...
0
votes
0answers
11 views

Representative signal

I'm implementing machine learning with sensor data. I am having the problem that some sensors not always have good integrity, that is, not all data points arrive at destination because of ...
0
votes
0answers
53 views

Imputation in R: How to impute univariate data in R?

I am trying classification(2 classes) using Random Forest. Classes are - Red, Green. My dataset contains 1 numeric attributes(called X), and 51 binary attributes to classify a document into red and ...
0
votes
0answers
12 views

Imputation in R: How to impute univariate data in R? [duplicate]

I am trying to classify(2) using Random Forest. Dataset contains 1 numeric, and 51 binary attributes. Numeric data is essential(Blood Pressure) for the classification. However, 40% instances do not ...
1
vote
0answers
20 views

Orthogonal polynomials contrasts for regression (unbalanced)

We can obtain the sum of squares of a contrast for a regression of degree $j$ by: $$ SSR_j=\frac{\left(\displaystyle\sum_{i=1}^{I} C_{ji}T_i\right)^2}{rK_j}, $$ where $I$ is the number of levels of ...
0
votes
1answer
123 views

Applying Rubin's rule for combining multiply imputed datasets

I am hoping to pool the results of a pretty basic set of analysis performed on a multiply imputed data (e.g. multiple regression, ANOVA). Multiple imputation and the analyses have been completed in ...
1
vote
1answer
20 views

Univariate analyses with incomplete data set

I'm dealing with a fairly large ecological data set (field data, not collected by me) of approximately 60 attributes measured for about 400 individual trees. Very few trees (~20%) have complete data ...
0
votes
1answer
25 views

Regression imputation of missing data

Suppose a two-way experiment with interaction. Is it correct to estimate the missing values by OLS, input those values in the data (fill the blanks) and now perform a polynomial (or any kind of) ...
0
votes
1answer
47 views

Regression imputation of missing data based on OLS effects

Let's say we have a two-way with interaction experiment with missing data. Being the dataset: ...
1
vote
0answers
6 views

how can I do a 3-way repeated measures ANOVA on filtered data?

I have a sample of participants tested on a two-day memory task where they learn the task on day 1, then are tested on day 2. They are Parkinson's disease patients and are tested on or off their ...
1
vote
1answer
32 views

Index-variable as an independent variable

In my regression on gdp-growth, I also want to bring in something like a "freedom"-variable, to show how free a country is (press freedom, economic freedom). now there is no number for this, except ...
0
votes
0answers
12 views

Missing block (Multilevel Modelling, repeated measures)

Experiment: participants respond to a moving object. Explanatory variables: velocity of object (fast, slow), Type of object (A,B). There are four different types of blocks, each with 16 trials: ...
6
votes
1answer
286 views

Data visualization for missing data

I am a designer and am trying to plot a set of data over time. For example, Day1 Day2 Day3 Day4 Day5 10 53 21 67 38 ...
1
vote
0answers
31 views

How to deal with empty values in a cluster analysis

I'm currently working on my master's thesis. Part of the work is a customer segmentation by means of a cluster analysis. One variable for the cluster determination should be the chronological ...
1
vote
0answers
37 views

Fit multiple regression model with pairwise deletion (or on a correlation/covariance matrix) in R

I'm trying to fit a multiple regression model with pairwise deletion in the context of missing data. lm() uses listwise deletion, which I'd prefer not to use in my ...
0
votes
1answer
22 views

Interpolation model to estimate missing analytics

We have about 7 months of partially (30%) missing web analytics, that is apparently missing at random across all segmentations. We need to estimate the missing data to correctly compare current and ...
1
vote
2answers
48 views

EM algorithm to impute missing value for one variable

This is from Robert Hogg's Introduction to Mathematical Statistics 6th, exercise 6.6.5. p366, It says, Suppose $X_1$, $X_2$, $X_{n1}$, are a random sample from a $N(\theta,1)$ distribution. Suppose ...
0
votes
0answers
27 views

How is spline computed in simple time series in AMELIA

I am trying to use Amelia to perform missing data imputation in a simple time series where time is represented as minutes and each measurement is replicated 4 times. ...
0
votes
0answers
11 views

multiple imputation - likelihood base

in nonignorable mechanism, selection model or pattern-mixture model is a multiple imputation method or a likelihood-base method? I am confused, i know in MI, missing data were filled in and then We ...
1
vote
0answers
7 views

Affinity Propagation with missing data

I'm using Affinity Propagation to cluster some data, and I have to deal with missing values, so for points that I have the data, I can use it to change similarity between them, but for those that I ...
5
votes
1answer
48 views

Is precision-based (i.e. inverse-variance) weighting integral to meta-analysis?

Is precision-based weighting central to meta-analysis? Borenstein et al. (2009) write that for meta-analysis to be possible all that is necessary is that: Studies report a point estimate which can ...
3
votes
0answers
28 views

Planned missing data design not converging

I recently ran an experimental study with a planned missing design. Participants were randomly assigned to one of four groups that dictated which portion of a 26-item unidimensional measure they ...