When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

1
vote
0answers
8 views

J48 Handling Missing Value with Tree based Imputation

Aloha, currently i have some trouble and question zu implement some kind of special missing value handling in WEKA J48 algorithm using WEKA JAVA API. I want to test the performance of SHAPIRO ...
2
votes
1answer
14 views

conducting multi-level regression on ordinal DVs with imputed data in R

Do you know of an approach/package that facilitates mixed model regression of ordinal dependent variables on multiply imputed datasets in R? Ideally, the function takes: a list of multiply imputed ...
0
votes
0answers
16 views

R rpart classification tree error

I am trying to run a classification tree in R using the rpart package in R. I keep getting the following error: ...
1
vote
0answers
33 views

How to deal with “not applicable” values in categorical variables

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
1
vote
0answers
19 views

Predict/impute one cell of matrix using all other cells

The question: I want to predict/impute one missing cell of a matrix using the contents of all other cells. Anyone have ideas on how to do this? The context: The matrix is n people's responses to m ...
0
votes
1answer
11 views

missing value patterns

I am doing some data preparation with Python using Pandas and I am working with a dataset that has about 80 variables with missing values and I want to capture any patterns of missingness to cut down ...
1
vote
1answer
32 views

Cross-correlation in R – dealing with missing values

I have two time series. One is an environmental variable (n = 108) organized by year and month. The other is a biological variable, also organized by year and month, but I have no data for some months ...
1
vote
0answers
33 views

Can I do two factor repeated measures ANOVA with differing numbers of measures per group?

I want to compare reaction time in four groups but I have three measures (pretest,post test and delayed post test) only in one of the groups. it was impossible to give post test and delayed post test ...
0
votes
0answers
11 views

How to treat “Missing Value Analysis” test results (problems)

I have a problem with Missing Value Analysis. I am using SPSS version 20. I am trying to test whether missing values are at complete random. As I know in order to ensure missing values are completely ...
-1
votes
1answer
33 views

Imputation and Distributions

Suppose you impute a variable using a normal distribution with mean 10 and sd 5. Is it better to draw 1000 random samples from this normal distribution, take the average, and then use this to impute ...
3
votes
1answer
28 views

How to get only positive values when imputing data?

Suppose age is normally distributed with mean 20 and standard deviation 5. How do you ensure that you get only positive values when you sample age from this distribution? I am trying to impute ...
0
votes
0answers
13 views

Missing data and SVM

I know that in multiple imputation, one runs a regression for each imputed data set. The final regression coefficients are averages of the coefficients obtained from the imputed data sets. Can you do ...
0
votes
3answers
115 views

using random forest for missing data imputation in categorical variables ( in R)

I have following type of associated data. The following example step to generate associated variable. p number of variables and n is number of observations. ...
-2
votes
0answers
32 views

Imputing missing responses in test exam [closed]

I am working with a database that consist in the answers of 100 students to about 300 questions (1: right answer, 0: wrong ...
1
vote
1answer
15 views

Mean versus imputation for missing data in the case of an ordinal scale

Is mean or mode better for replacing missing data for an ordinal scale? I'm thinking mode is better because the respondent has to choose between integer values (1, 2 and so on) bu I am wondering is ...
0
votes
0answers
3 views

decimal_date() with missing data [migrated]

decimal_date(x) in package lubridate works OK if x is a valid date value but errors if x is NA. I am taking x from a data frame where some values are NA. Examples This works because there are no NAs ...
0
votes
1answer
19 views

Incomplete questionnaires

Is it recommented to delete the cases from a database if the responses are highly incomplete? Is there a percentage that can be taken into consideration? (for example, deleting a case if more than 50% ...
0
votes
1answer
24 views

interpretation of little's mcar test

My little's MCAR test revealed chi-square = 27.120, DF = 1974, and sig. = 1.000. 74 items and 151 cases. So, can I concluded that the data were missing completely at random since the p-value is not ...
1
vote
1answer
103 views

How to perform a bivariate regression using pairwise deletion of missing values in R?

Is there any way to perform bivariate regression using pairwise deletion of missing values in R? na.action options in lm() do not offer such a possibility – the default na.action is na.omit, which is ...
2
votes
1answer
25 views

Method to compare ratings from multiple different sources with missing data

I want a method to compare ratings from multiple sources and find a single measure that best reflects all the ratings. To give a specific example, let's call it "The fellowship review committee ...
4
votes
2answers
167 views

When is it a good idea to just use the average for imputation?

Suppose we have a data set test: 1 8 12 14 . . 19 The . denotes missing values. When would it be better to use the average ...
0
votes
0answers
37 views

temperature prediction algorithm

I found an interesting problem in a contest on temperature prediction: https://www.hackerrank.com/contests/expansion-challenge/challenges/temperature-predictions It is not about forecasting the ...
1
vote
0answers
7 views

Best Multiple Imputation Method for Multiple Surveys Mixed Together, Presented Randomly?

I am working with a dataset containing data from 15 different surveys. The surveys were presented all as one battery to participants, with questions from all surveys essentially placed into a pool and ...
1
vote
0answers
24 views

Repeated measures ANOVA with missing data

I have a DB with multiple missing values (scale, ordinal and nominal). I used multiple imputation to fill in these missing values. How can I perform a repeated measures ANOVA on these (5) datasets? ...
0
votes
0answers
31 views

Matrix completion approaches for healthcare big data

I am working on a prediction problem that leverage sparse clinical datasets. Missing data rate is in the range of 80%. 1- I am wondering if there is any example of application of matrix completion ...
1
vote
2answers
113 views

Machine learning algorithms to handle missing data

Am trying to develop a predictive model using high-dimensional clinical data including laboratory values. The data space is sparse with 5k samples and 200 variables. The idea is to rank the variables ...
0
votes
0answers
22 views

Spatial and temporal effects in water quality data

The data I have is 20 sampling points in a water distribution network that have been sampled weekly during 4 months for different parameters (chlorine, turbidity, disinfection by-products, ...). Some ...
4
votes
1answer
51 views

Missing at Random Data in GEE

For a continuous outcome being analyzed using GEE with a linear link, you have assurance that standard errors and point estimates are consistent with a first order trend regardless of distribution of ...
0
votes
0answers
17 views

Missing data in 3 dimensional Panel: Are comparisons across different samples valid?

I have a panel of 8 sectors across 24 OECD countries. Im estimating aggregate equations across all sectors, and 8 separate equations for each sector. Part of the analysis is to make comparisons across ...
3
votes
0answers
28 views

Is it helpful to have monotonic features when using a random forest for classification?

I am training a random forest for binary classification. Here is a plot of one of my features, which is an integer giving the number of months since an event. The y-axis gives the proportion of cases ...
4
votes
1answer
57 views

Handling missing data in a time series

Consider an epidemic curve like the one below, or any other count-based time series data: If, as it turns out from digging into the records, retrospective data collection, or just a series of case ...
0
votes
0answers
19 views

ANOVA, unbalanced designs, missing data, and multiple comparisons

I am having several problems with my dataset and how best to analyze it. I have measured a series of plant phenology characteristics (with a seperate model for each one - I do not want to combine them ...
7
votes
5answers
230 views

How to perform imputation of values in very large number of data points?

I have a very large dataset and about 5% random values are missing. These variables are correlated with each other. The following example R dataset is just a toy example with dummy correlated data. ...
0
votes
0answers
16 views

Poisson regression on partially missing covariates

I'm reading an article where the author fits a Poisson regression on some data. Some of the covariates (but not the outcome) are partially missing. The author writes that he: conducted the ...
0
votes
2answers
61 views

Simultaneous imputation of multiple binary variables in R

I have a dataset with multiple correlated binary variables (0/1). Can anyone point me toward a solution for imputing completely random missing values based on information in the other variables? ...
3
votes
2answers
101 views

How to approach missing data…ignore or not to ignore?

What would be the best option when you come across missing data. Do you exclude that person completely from the data analysis or not. If not then how do you go about it? i have data on two groups at ...
4
votes
3answers
254 views

Why doesn't Random Forest handle missing values in predictors?

What are theoretical reasons to not handle missing values? Gradient boosting machines, regression trees handle missing values. Why doesn't Random Forest do that?
1
vote
1answer
36 views

can matrix completion work in the presence of many missing values?

I have a matrix with about 550k elements (2500 x 220) with 100k values known and the rest are unknown. Would it make sense to use matrix completion in this case, or are there too many values which ...
1
vote
1answer
51 views

Individual level prediction of a person’s probability of voting without their vote history

Is it possible to create individual level predictions of a voter's probability of voting when you do not know their vote history? In the data set provided in my homework assignment, I am given data on ...
1
vote
0answers
12 views

What to do when complete cases vs. imputed cases and train vs. test disagree?

In a fairly complex survival analysis case with considerable missing data, I split the data set into training and testing and ran models for both complete cases and imputed cases (I used multiple ...
2
votes
1answer
70 views

Dealing with missing data - glmer in lme4 package

Dear StackExchange community, I have an unbalanced data set / data set with missing values, consisting of 20 submersible acoustic receivers that have been range tested on 8 days (Both receiver ID ...
0
votes
0answers
44 views

Thinking about Neural Networks

I am confused about the output I am getting for a neural network analysis. What I am trying to do is take in the features from features.csv and use them to create a neural network to predict the ...
0
votes
0answers
28 views

Difference of unequal sample size

I am a newbie in statistics. I have two columns of numerical data of unequal length, motion data. I need difference of the two columns, by shortening the longer column. ...
0
votes
1answer
45 views

Interpreting linear regression with NA for estimates

I have 3 variables. Income is the dependent variable(continuous), sex(binomial), workhrs(categorical) i fitted a linear model and obtained the result: My question is: Is this model usable? The NA ...
1
vote
0answers
17 views

Should I use missing data imputation with a model that already allows incomplete data?

I'm just starting to learn about missing data imputation methods, and I'm confused. In every introduction I've read, the author starts by describing listwise deletion and says that it's a bad idea ...
1
vote
1answer
49 views

Data Fusion/Statistical Matching: Match Keys & Algorithms?

I need to fuse multiple surveys using statistical matching, and would like to comply with any best practices. I have studied multiple documents but -- despite the plethora of information -- it's ...
1
vote
2answers
58 views

Missing data due to absent parent

I am using the following regression: $$\text{Test score} = \beta_0+\beta_1\text{Mother's employment}+\beta_2\text{Mother's education}$$ where "Mother's employment" is a set of dummy variables ...
0
votes
0answers
18 views

FIML assumptions and violations

From what I know about full information maximum likelihood (FIML), the assumptions are missing at random (MAR), which can't be tested, and multivariate normality. The advantages of FIML are ...
1
vote
1answer
53 views

How do decision tree learning algorithms deal with missing values (under the hood)

What are the methods that decision tree learning algorithms use to deal with missing values. Do they simply full the slot in using a value called missing? Thanks.
1
vote
1answer
111 views

Missing Values NAs in the Test Data When using predict.lm in R

I have two data sets Train data Test data (with no dependent variable values but I have data on independent variable or you can say I need to forecast). Using the training data (which has some ...