When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

0
votes
0answers
20 views

Machine Learning when missing data is state-dependent (e.g. adaptive questionnaires)

I have the following problem: I am dealing with an adaptive questionnaire, meaning a questionnaire where there are questions that are only asked when a previous question had a specific answer. The ...
0
votes
0answers
20 views

Natural logarithm transfomation and zeroes [duplicate]

I am using Stata 13 to estimate a simple regression. Given a rather positive skew of a few of my covariates, I figured to ln-transform the variables. However, I have a substantial amount of zeroes in ...
0
votes
0answers
5 views

Multiple imputation in SAS for longitudinal data

I have a data set from a repeated measurement study comparing two groups with missing data due to lost-to-follow-up (~20%). I know how to apply multiple imputation method for cross-sectional data. ...
3
votes
1answer
42 views

Mixed-modeling when no observation contributes both X and Y

I'm working on a project investigating the relationship between (let's say) a face's perceived masculinity and its perceived competence. There was a large number of face stimuli (80). Two completely ...
1
vote
0answers
17 views

r package: CFA and missing data

I am running confirmatory factor analysis (CFA) with r. As I have several missing observations, I get two series of results, on "used" (152) and "total" ...
1
vote
0answers
19 views

How to treat missing values in a regression?

I have used the logarithmic form of wage as my independent variable in Stata. However, it contains missing values. Should I replace these with 0 or let them be?
0
votes
0answers
13 views

create_WX() always creates NA [migrated]

Using spdep for R (a collection of tools for spatial econometrics), I wanted to build a regression matrix myself. The matrix is the combination of two matrices, one ...
1
vote
0answers
21 views

Is maximum likelihood a form of data substitution? Or not?

I’m using maximum likelihood with missing data. In this case of missing data, is maximum likelihood a form of data substitution? I’m significantly more familiar with multiple imputation which I ...
0
votes
2answers
38 views

How to handle NA values in shrinkage (Lasso) method using glmnet

I'm using "glmnet" for lasso regression in GWAS. Some variants and individuals have missing values and it seems that glmnet cannot handle missing values. Is there any solution for this? or is there ...
3
votes
1answer
33 views

Multiple Imputation methods

Suppose that a variable $Y_j$ has missing values. We can use regression to impute the data using the nonmissing observations: $$Y_j = \beta_0+\beta_{1}Y_{1}+\beta_{2}Y_{2} + \dots + \beta_{(j-1)} ...
0
votes
3answers
41 views

How to use Random Forest for categorical variables with missing value

I have a labelled dataset of 1M rows and 600 features. I am trying to build a supervised learning model for prediction. I am particularly working with Random forests in R.The data I have has following ...
2
votes
1answer
36 views

Prove the loglikelihood is strictly concave for ABO allele frequency blood type data

I am working through the problems in Kenn Lange's book Numerical Analysis for Statisticians. I am going to try and do all of the problems in the book, though none of them are specifically assigned for ...
-1
votes
1answer
43 views

rbinom() produces NA values. What's wrong?

I have to generate RV from a binomial distribution with R . I have a vector for $n$ and a vector for $p$ and for each component I have to generate a random variable. My idea was the following for ...
1
vote
2answers
104 views

Treating missing data in voting pattern analysis

I'm trying to analyze voting patterns of Ukraine's parliament deputies. I scraped all the data on their voting during last session. Each data entry has following information: Deputy name, date, bill ...
2
votes
2answers
58 views

How much missing data is too much? Multiple Imputation (MICE) & R

Currently I am working on a large data set with well over 200 variables (238 to be exact) and 290 observations for each variable (in theory). This data set is missing quite a lot of values, with ...
0
votes
2answers
13 views

pre-post analysis in two groups with missing post in control group

A bit background of the study: Case group: tumor patients underwent a treatment Control group: a healthy subjects with matched gender, age and some other variables Target parameter: a continuous ...
0
votes
0answers
24 views

Reference request: EM algorithm and hidden Markov model books with solutions

I am studying missing data problems and the applications of the EM algorithm to missing data problems, like mixture models and hidden Markov models. We have been using Schafer's book Analysis of ...
2
votes
1answer
24 views

Missing data paired samples t-test

I have a list of insurance policies that filed claims in pre/post periods. I am trying to test whether the average cost of insurance claims per policy varied. Some of my policies don't have any claims ...
1
vote
0answers
35 views

Forecasting one dataset using data and correlation from another using R: commercial centers entrances and restaurant sales figures

Please, be kind, as I'm totally noob in stats and R... I'm the owner of a small restaurant in a commercial center, and I managedd to collect two main dataset, commercial-center (cc) and restaurant ...
0
votes
0answers
34 views

one way anova in R

I have only just started using R in my first year at university and I am completely stuck. I have done an anova test as the data is normally distributed and parametric. However, this is my result. I'm ...
0
votes
0answers
21 views

Full Information Maximum Likelihood, Imputation and Classification

I need to do a classification of a dataset, I have some missing data and I would like to try some "missing data techniques" to achieve the best accuracy. I already tried multiple imputation and ...
1
vote
2answers
60 views

Which interpolation technique should I use?

I have an annual data set, but I have a few missing values in the series. I do not know which interpolation technique should I use to fill the missing values. ...
0
votes
1answer
29 views

R Linear model step NA values

My aim is to carry out a generalized linear model (glm) with 1 response variable and 13 explanatory variables.Unfortunately 3 out of the 10 explanatory variables contain NA values (2/3 of data set of ...
2
votes
0answers
60 views

What methods can I use to aid in modeling a smaller data set when I have a significantly larger data set with fewer variables?

I currently have a data set with about 4,000 rows. The current model I have established for it is not very good, and I am going to receive more data for about 150 of these points, and I'm hoping that ...
1
vote
0answers
35 views

R Linear model step NA values

My aim is to carry out a generalized linear model (glm) with 1 response variable and 13 explanatory variables.Unfortunately 3 out of the 10 explanatory variables contain NA values (2/3 of data set of ...
1
vote
0answers
15 views

Strange likelihood construction for information borrowing

I recently encountered a strange model while consulting in industry. The goal was to collect information about the parameter of event $A$, say $\theta_A$, but we only observed events $B$ and $C$, ...
0
votes
0answers
8 views

Running a Two-Way Repeated Measures ANOVA with missing data

I'm trying to run a Two-Way rep measures Anova on a data set with two independent variables, and three time factors for each variable. However due to the severity of the experiment, not all ...
6
votes
2answers
35 views

Extrapolation of 2d movement

I have a problem with missing data in my dataset. My dataset is timeseries which contains x,y coordinates. I'd like to extrapolate missing values and use the assumption that I know speed and direction ...
0
votes
3answers
67 views

Missing data and imputation in general

Handling missing data is a bit confusing for me. My questions are: Is it better to calculate imputations than simply leave out NAs and leave it to the (appropriate) model to handle it? Is there a ...
0
votes
0answers
27 views

R caret and NAs

I very much prefer caret for its parameter tuning ability and uniform interface, but I have observed that it always requires complete datasets (i. e. without NAs) even if the applied "naked" model ...
0
votes
0answers
10 views

Longitudinal design with missing values at first time point

I carried out an intervention with 3 time points (2 pre-intervention, 1 post-intervention) with 2 groups ( n =18 intervention, n = 22 control). Basically I am missing some observations from some of ...
2
votes
0answers
29 views

Impute missing data for mixed effects models?

Although I will not provide a reference, because I cannot recall where I did read it, I have several times read or heard that missing data is accommodated automatically in mixed models. Can anyone ...
2
votes
1answer
47 views

factor analysis with missing values

I have data on about 25 subjects and 30 variables with about 20 missing values. The data is missing at random. What will be the best approach to perform factor analysis. How is factor analysis versus ...
0
votes
0answers
11 views

Significance of m parameter in m-estimate

To assign a probability to events that have not occurred yet(for a fixed set of events), one of the simplest methods is to use the m-estimator, which is defined as the following: $$Pr(A) = ...
0
votes
0answers
26 views

Learning from streams with concept drift in real world - how to handle missing class problem?

In currently delves into learning from streams with concept drift. As more I learn I think about how I can use learning algorithms on real data. Most of drift detection algoritms to evaluate is ...
1
vote
0answers
29 views

Apply trained MDS model to new data

I have both a distance matrix and the original vectors, and am using MDS (Multidimensional Scaling) with R to generate vectors in more dimensions for the data. With dimensionality reduction (for ...
1
vote
0answers
33 views

Missing values in Time Series

Missing values are very common in large time series data. How should the missing values of a time series be estimated? Is interpolation useful or I need to forecast them from the past values?
3
votes
1answer
57 views

Multiple Imputation Using Amelia [duplicate]

I am using Amelia for multiple imputation, and I am satisfied with the imputed results. But I want to restrict the imputed variable to positive values. Is there a way that Amelia can handle it or ...
1
vote
0answers
32 views

Best approach to predict significant factors without any complete cases

I have a dataset that contains records of donors with various biographical info (city, state, zip, number of children) and the total amount they donated over 10 years. Some never donated and thus the ...
0
votes
0answers
48 views

Handling missing data in Gaussian Process Regression

I am trying to handle missing data in a model using Gaussian Processes. I have two spatial dimensions sharing the same length hyper-parameter, one dimension for time. Additionally I have split one ...
1
vote
0answers
51 views

Regularized regression with missing data?

Are you aware of any regularized regression methods (i.e. Lasso, elastic net) which allows for using cases with incomplete (missing) data (e.g. using EM estimation)? And if yes, is the method ...
1
vote
1answer
41 views

filling out NA values using clustering analysis

I have a data frame with a large number of NA values. I do not wish to leave out all these rows as that would reduce the size of my training set drastically. I filled out these missing values in a ...
0
votes
0answers
19 views

How does missing data (not at random) affect Bayesian models?

When I was a student learning about Bayesian models, we were taught that missing data was not a problem because they would be imputed. However I am wondering about how missing not at random (MNAR) ...
0
votes
0answers
15 views

Significance of a correlation considering missing data

So I'm dealing with two timeseries, of which one contains quite some missing data (between 10-60%). I'm excluding some datasets which contain exceptional much missing data, but still I would like to ...
3
votes
0answers
64 views

Missing values with Community structure in networks?

Is there a way to predict Missing values with Community structure in networks? I have a data set with a couple dozen variables, such as age, level of education, self-assessed (via a Likert scale) ...
0
votes
1answer
44 views

NA values in linear model in r

I have the following dataset: desingmatrix <-read.csv("path of csv with data", sep=";", dec=".") View(desingmatrix)# vision de los datos Then I try to set ...
1
vote
0answers
19 views

Repeated Measures Analysis for a Single Treatment with Missing Data

I have some data here that I am at a loss on how to analyze. The following data set represents a subset of some data I have where a number of subjects (4 here, about fifty in the actual set) ...
0
votes
0answers
40 views

Using gbm in R with non-random missing data

The way gbm handles missing variables in R is by using surrogate splits. Is this appropriate to use when the data is not missing at random?
0
votes
1answer
80 views

Multiple Imputation and Regression Model Diagnostics

When I run regression analysis I find it important to run some model diagnostics, such as detection of outliers, influential observations, multi-collinearity (much like these examples ...
1
vote
1answer
50 views

How to create scaled scores using Likert Data with N/A options?

I have 7 survey questions that I asked students about regarding their online learning experience. Each question is on a 5 point Likert scale including -1 for not applicable. Now that I have the ...