When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

0
votes
0answers
19 views

how to handle the imbalanced data in regression analysis

The problem here is very similar to the problem asked by someben in 2012 (link:Sampling for Imbalanced Data in Regression). It involves the linear regression analysis using an unbalanced dataset. Say, ...
1
vote
0answers
34 views

Mean imputation/estimation of missing data

Could someone please refer me to papers that have imputed the mean to missing values of a continuous variable? (i.e. papers that have used mean imputation) I have imputed my missing IMD values using ...
-4
votes
0answers
90 views

Exploratory regression analysis for data with missing values

Recently I have performed an exploratory regression analysis, using lavaan R package and observed the following output with some warning messages in it. I have the ...
-1
votes
0answers
27 views

A data set with many variables having NA Values , Some NA Values is to be removed and in some variables mode value is to be imposed HOW [closed]

I have a dataset with 25 variables. To see the dataset click on "Dataset snapshot1" link below. There is a variable named "Metro" and "PC_Count_Approx". Both variable having missing Values. Metro ...
1
vote
0answers
20 views

How to handle uninterested missing (unobserved) data?

Suppose I have $N$ items $\{x_n\}_{n=1}^N$ (also denote the feature vector as $x_n$) and I have observed $M$ pairwise preferences $\{x_{i_m} \succ x_{j_m}\}_{m=1}^M$ from a user. Now I want to predict ...
0
votes
0answers
10 views

rebuild model based on what columns a record has

I've built a model in R using glm, and in the new dataset that I need to predict, there exist some new levels for the columns that are non-numeric. I know there are so many approaches to deal with ...
3
votes
1answer
162 views

Determining probability distribution for datasets with missing values

As a part of my exploratory data analysis (EDA) prior to further analysis, I'm trying to determine a probability distribution of my pilot dataset's variables. A particular feature of this dataset is a ...
1
vote
2answers
20 views

Missing values in SPSS for Chi-Square Tests

I'm a postgraduate student, doing a accent perception study, this is my first time using SPSS! I'm conducting the above tests on my data, where my participants used Likert scales to analyse the ...
0
votes
0answers
6 views

Missing values on clinical scoring system in SPSS

I have a SPSS database with 747 patients with sickness symptoms and lab results as variables. With those symptoms and results I have generate two scoring systems. I sum the scores of all the variables ...
0
votes
1answer
23 views

Measures of goodness-of-fit using multiply imputed data in Zelig

I am running a logistic regression model in R using multiply imputed data created using Amelia II, which I am then analyzing using Zelig. I would like to be able to report some measures of ...
0
votes
0answers
13 views

Categorical Missing Value Replacement

I want to ask how I could replace categorical missing values with reference please?
2
votes
0answers
14 views

Multiple Imputation Variance

I have been reading up on multiple imputation, and I am interested in the between-imputation variance. However, not in the estimation of the parameters, but in the imputed values themselves. (1) ...
1
vote
2answers
37 views

How to replace only SOME missing data

I have two types of missing data (non-responses and "Don't know or does not apply") and I want to replace only the non-responses (with mean or by using EM or by using Multiple Imputation) and to see ...
1
vote
0answers
37 views

Multiple imputation constraining for not applicable values

I want to do a multiple imputation of missing data using SPSS. I have a nominal (categorical) variable X with missing values codified as '9' and not applicable values codified as '8'. How can I ...
1
vote
1answer
22 views

Covariance matrix estimation in presence of missing data

I want to estimate a covariance matrix from data with some missing values. Ideally I'd like an R package but python could be ok. R has some built in ways of doing this. You can use ...
0
votes
1answer
12 views

Data Analysis of No-Opinion Options

In my questionnaire, I had a "no-opinion" option for several items on a 5-point-likert scale. I know, there is a lot of discussion about sense and non sense of those, but the fact is, i've got a data ...
0
votes
0answers
26 views

Estimate linear regression using items randomly selected from an item pool

I am asking this question against the background of a linear regression with single predicted variable $Y$ and multiple predictors $X$. $X$ comes from a survey using an "item pool" which suggests that ...
-1
votes
1answer
31 views

Marginal probability density of x (obs) obtained by integrating x(missing)

I am reading a text book on missing data, and a sentence below is slightly challenging for me to understand. The marginal probability density of $ \left ( x_{obs}\ \right)$ is obtained by ...
1
vote
1answer
35 views

LASSO or other regularized regression with censored (missing) data

Here is my problem. I am looking at various time series curves. Let's call them total spend aggregated over all customers on various products versus time. At any given time, I want to predict the ...
1
vote
2answers
68 views

How can I partition a distribution into two sub-populations with fixed bias? (simulation)

I am trying to simulate a selection model for a variable $Y$ dependent on covariate vector $X$, so that two groups/sub-sets $S=(0,1)$ of observations on $Y$ are created, which have a fixed difference ...
0
votes
0answers
9 views

How to replace missing values with unsupervised random forest?

from http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#unsup : Formulating it as a two class problem has a number of payoffs. Missing values can be replaced effectively. Outliers ...
1
vote
0answers
24 views

Modeling Counts With Small Observations

I am new to Cross Validated SE so I am going to try and formulate my question to the best of my ability. I have a large data set that contains $5$ different fields. The fields are ...
1
vote
1answer
31 views

Missing values - for cases of “I do not know” or “Does not apply”

If I have a questionnaire with items on a Likert scale (1 to 5) and a sixth option "I do not know/Does not apply", is it okay to consider a "I do not know/Does not apply" response as a missing value ...
3
votes
1answer
43 views

How to generating MAR data with fixed proportion of missing?

To generate data with missing at random (MAR) mechanism, usually we can first generate a complete data set then model the missing probability for variable Y, i.e. $Pr(Y=missing|{\bf X})$, using ...
0
votes
0answers
16 views

Missing Value in Commodities Prices

I am trying to model the prices of four energetic commodities with ARIMA models in R. Unfortunately the price series is not regular, as for some days, like Christmas, no price is given. My series is ...
0
votes
0answers
8 views

Missing value replacement when 3 out of four scale items are missing

I have a subscale with four items and 316 participants. Five participants missed all four items, two participants missed three of the four items and five participants missed only one item (three ...
0
votes
1answer
23 views

Overfitting on Missing Value Imputations

When performing Missing Value imputations, should we be concerned about overfitting the data? Why or why not? For example: If I impute a variables missing value using a CART regression tree, should ...
1
vote
0answers
17 views

J48 Handling Missing Value with Tree based Imputation

Aloha, currently i have some trouble and question zu implement some kind of special missing value handling in WEKA J48 algorithm using WEKA JAVA API. I want to test the performance of SHAPIRO ...
2
votes
1answer
43 views

conducting multi-level regression on ordinal DVs with imputed data in R

Do you know of an approach/package that facilitates mixed model regression of ordinal dependent variables on multiply imputed datasets in R? Ideally, the function takes: a list of multiply imputed ...
1
vote
0answers
65 views

How to deal with “not applicable” values in categorical variables

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
1
vote
0answers
21 views

Predict/impute one cell of matrix using all other cells

The question: I want to predict/impute one missing cell of a matrix using the contents of all other cells. Anyone have ideas on how to do this? The context: The matrix is n people's responses to m ...
0
votes
1answer
16 views

missing value patterns

I am doing some data preparation with Python using Pandas and I am working with a dataset that has about 80 variables with missing values and I want to capture any patterns of missingness to cut down ...
1
vote
1answer
45 views

Cross-correlation in R – dealing with missing values

I have two time series. One is an environmental variable (n = 108) organized by year and month. The other is a biological variable, also organized by year and month, but I have no data for some months ...
1
vote
0answers
37 views

Can I do two factor repeated measures ANOVA with differing numbers of measures per group?

I want to compare reaction time in four groups but I have three measures (pretest,post test and delayed post test) only in one of the groups. it was impossible to give post test and delayed post test ...
0
votes
0answers
18 views

How to treat “Missing Value Analysis” test results (problems)

I have a problem with Missing Value Analysis. I am using SPSS version 20. I am trying to test whether missing values are at complete random. As I know in order to ensure missing values are completely ...
-1
votes
1answer
36 views

Imputation and Distributions

Suppose you impute a variable using a normal distribution with mean 10 and sd 5. Is it better to draw 1000 random samples from this normal distribution, take the average, and then use this to impute ...
3
votes
1answer
34 views

How to get only positive values when imputing data?

Suppose age is normally distributed with mean 20 and standard deviation 5. How do you ensure that you get only positive values when you sample age from this distribution? I am trying to impute ...
0
votes
0answers
14 views

Missing data and SVM

I know that in multiple imputation, one runs a regression for each imputed data set. The final regression coefficients are averages of the coefficients obtained from the imputed data sets. Can you do ...
0
votes
3answers
161 views

using random forest for missing data imputation in categorical variables ( in R)

I have following type of associated data. The following example step to generate associated variable. p number of variables and n is number of observations. ...
1
vote
1answer
18 views

Mean versus imputation for missing data in the case of an ordinal scale

Is mean or mode better for replacing missing data for an ordinal scale? I'm thinking mode is better because the respondent has to choose between integer values (1, 2 and so on) bu I am wondering is ...
0
votes
1answer
23 views

Incomplete questionnaires

Is it recommented to delete the cases from a database if the responses are highly incomplete? Is there a percentage that can be taken into consideration? (for example, deleting a case if more than 50% ...
0
votes
1answer
70 views

interpretation of little's mcar test

My little's MCAR test revealed chi-square = 27.120, DF = 1974, and sig. = 1.000. 74 items and 151 cases. So, can I concluded that the data were missing completely at random since the p-value is not ...
1
vote
1answer
149 views

How to perform a bivariate regression using pairwise deletion of missing values in R?

Is there any way to perform bivariate regression using pairwise deletion of missing values in R? na.action options in lm() do not offer such a possibility – the default na.action is na.omit, which is ...
2
votes
1answer
28 views

Method to compare ratings from multiple different sources with missing data

I want a method to compare ratings from multiple sources and find a single measure that best reflects all the ratings. To give a specific example, let's call it "The fellowship review committee ...
4
votes
2answers
171 views

When is it a good idea to just use the average for imputation?

Suppose we have a data set test: 1 8 12 14 . . 19 The . denotes missing values. When would it be better to use the average ...
0
votes
0answers
44 views

temperature prediction algorithm

I found an interesting problem in a contest on temperature prediction: https://www.hackerrank.com/contests/expansion-challenge/challenges/temperature-predictions It is not about forecasting the ...
1
vote
0answers
10 views

Best Multiple Imputation Method for Multiple Surveys Mixed Together, Presented Randomly?

I am working with a dataset containing data from 15 different surveys. The surveys were presented all as one battery to participants, with questions from all surveys essentially placed into a pool and ...
1
vote
0answers
30 views

Repeated measures ANOVA with missing data

I have a DB with multiple missing values (scale, ordinal and nominal). I used multiple imputation to fill in these missing values. How can I perform a repeated measures ANOVA on these (5) datasets? ...
0
votes
0answers
38 views

Matrix completion approaches for healthcare big data

I am working on a prediction problem that leverage sparse clinical datasets. Missing data rate is in the range of 80%. 1- I am wondering if there is any example of application of matrix completion ...
2
votes
2answers
192 views

Machine learning algorithms to handle missing data

Am trying to develop a predictive model using high-dimensional clinical data including laboratory values. The data space is sparse with 5k samples and 200 variables. The idea is to rank the variables ...