When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

1
vote
0answers
23 views

How to deal with empty values in a cluster analysis

I'm currently working on my master's thesis. Part of the work is a customer segmentation by means of a cluster analysis. One variable for the cluster determination should be the chronological ...
0
votes
0answers
9 views

R script - dataframes filling missing values [on hold]

I am trying to fill na values in a dataframe with 0. I tried this: unique_contacts <- apply(unique_contacts,2,function(col) col[is.na(col)] = 0) The ...
0
votes
0answers
7 views

combining 2 dataframes, replacing values of one frame with other R [migrated]

Maybe this is easy, but I can't figure out a way to do this. My real issue is with a much larger data set, so I'm looking for something more elegant than what I'm currently doing. I have 2 data ...
1
vote
0answers
7 views

Fit multiple regression model with pairwise deletion (or on a correlation/covariance matrix) in R

I'm trying to fit a multiple regression model with pairwise deletion in the context of missing data. lm() uses listwise deletion, which I'd prefer not to use in my ...
0
votes
1answer
15 views

Interpolation model to estimate missing analytics

We have about 7 months of partially (30%) missing web analytics, that is apparently missing at random across all segmentations. We need to estimate the missing data to correctly compare current and ...
0
votes
0answers
20 views

EM algorithm to impute missing value for one variable

This is from Robert Hogg's Introduction to Mathematical Statistics 6th, exercise 6.6.5. p366, It says, Suppose $X_1$, $X_2$, $X_{n1}$, are a random sample from a $N(\theta,1)$ distribution. Suppose ...
0
votes
0answers
23 views

How is spline computed in simple time series in AMELIA

I am trying to use Amelia to perform missing data imputation in a simple time series where time is represented as minutes and each measurement is replicated 4 times. ...
0
votes
0answers
9 views

multiple imputation - likelihood base

in nonignorable mechanism, selection model or pattern-mixture model is a multiple imputation method or a likelihood-base method? I am confused, i know in MI, missing data were filled in and then We ...
1
vote
0answers
7 views

Affinity Propagation with missing data

I'm using Affinity Propagation to cluster some data, and I have to deal with missing values, so for points that I have the data, I can use it to change similarity between them, but for those that I ...
4
votes
1answer
36 views

Is precision-based (i.e. inverse-variance) weighting integral to meta-analysis?

Is precision-based weighting central to meta-analysis? Borenstein et al. (2009) write that for meta-analysis to be possible all that is necessary is that: Studies report a point estimate which can ...
3
votes
0answers
19 views

Planned missing data design not converging

I recently ran an experimental study with a planned missing design. Participants were randomly assigned to one of four groups that dictated which portion of a 26-item unidimensional measure they ...
0
votes
1answer
24 views

Comparing categorical outcome variables in repeated measures design

I am working on an observational prospective longitudinal study with a repeated measures design. The same categorical outcome is measured for five times, for all the participants, over a time period. ...
3
votes
0answers
38 views

Alternate weighting schemes for random effects meta-analysis: missing standard deviations

I am working on a random effects meta-analysis covering a number of studies which do not report standard deviations; all studies do report sample size. I do not believe it is possible to approximate ...
0
votes
0answers
8 views

What is the best practice to deal with NA values when calculating a dissimilarity matrix?

I need to calculate a matrix of distances between sites where different variables were measured. I will use it in a cluster analysis. The following is a sample of the matrix I am dealing with: ...
0
votes
0answers
26 views

Machine Learning when missing data is state-dependent (e.g. adaptive questionnaires)

I have the following problem: I am dealing with an adaptive questionnaire, meaning a questionnaire where there are questions that are only asked when a previous question had a specific answer. The ...
0
votes
0answers
22 views

Natural logarithm transfomation and zeroes [duplicate]

I am using Stata 13 to estimate a simple regression. Given a rather positive skew of a few of my covariates, I figured to ln-transform the variables. However, I have a substantial amount of zeroes in ...
1
vote
1answer
11 views

Multiple imputation in SAS for longitudinal data

I have a data set from a repeated measurement study comparing two groups with missing data due to lost-to-follow-up (~20%). I know how to apply multiple imputation method for cross-sectional data. ...
3
votes
1answer
47 views

Mixed-modeling when no observation contributes both X and Y

I'm working on a project investigating the relationship between (let's say) a face's perceived masculinity and its perceived competence. There was a large number of face stimuli (80). Two completely ...
0
votes
0answers
21 views

r package: CFA and missing data

I am running confirmatory factor analysis (CFA) with r. As I have several missing observations, I get two series of results, on "used" (152) and "total" ...
1
vote
0answers
28 views

How to treat missing values in a regression?

I have used the logarithmic form of wage as my independent variable in Stata. However, it contains missing values. Should I replace these with 0 or let them be?
2
votes
1answer
39 views

Is maximum likelihood a form of data substitution? Or not?

I’m using maximum likelihood with missing data. In this case of missing data, is maximum likelihood a form of data substitution? I’m significantly more familiar with multiple imputation which I ...
0
votes
2answers
55 views

How to handle NA values in shrinkage (Lasso) method using glmnet

I'm using "glmnet" for lasso regression in GWAS. Some variants and individuals have missing values and it seems that glmnet cannot handle missing values. Is there any solution for this? or is there ...
3
votes
1answer
39 views

Multiple Imputation methods

Suppose that a variable $Y_j$ has missing values. We can use regression to impute the data using the nonmissing observations: $$Y_j = \beta_0+\beta_{1}Y_{1}+\beta_{2}Y_{2} + \dots + \beta_{(j-1)} ...
0
votes
3answers
62 views

How to use Random Forest for categorical variables with missing value

I have a labelled dataset of 1M rows and 600 features. I am trying to build a supervised learning model for prediction. I am particularly working with Random forests in R.The data I have has following ...
2
votes
1answer
41 views

Prove the loglikelihood is strictly concave for ABO allele frequency blood type data

I am working through the problems in Kenn Lange's book Numerical Analysis for Statisticians. I am going to try and do all of the problems in the book, though none of them are specifically assigned for ...
-1
votes
1answer
54 views

rbinom() produces NA values. What's wrong?

I have to generate RV from a binomial distribution with R . I have a vector for $n$ and a vector for $p$ and for each component I have to generate a random variable. My idea was the following for ...
1
vote
2answers
116 views

Treating missing data in voting pattern analysis

I'm trying to analyze voting patterns of Ukraine's parliament deputies. I scraped all the data on their voting during last session. Each data entry has following information: Deputy name, date, bill ...
2
votes
2answers
116 views

How much missing data is too much? Multiple Imputation (MICE) & R

Currently I am working on a large data set with well over 200 variables (238 to be exact) and 290 observations for each variable (in theory). This data set is missing quite a lot of values, with ...
0
votes
2answers
15 views

pre-post analysis in two groups with missing post in control group

A bit background of the study: Case group: tumor patients underwent a treatment Control group: a healthy subjects with matched gender, age and some other variables Target parameter: a continuous ...
0
votes
0answers
24 views

Reference request: EM algorithm and hidden Markov model books with solutions

I am studying missing data problems and the applications of the EM algorithm to missing data problems, like mixture models and hidden Markov models. We have been using Schafer's book Analysis of ...
3
votes
1answer
26 views

Missing data paired samples t-test

I have a list of insurance policies that filed claims in pre/post periods. I am trying to test whether the average cost of insurance claims per policy varied. Some of my policies don't have any claims ...
1
vote
0answers
45 views

Forecasting one dataset using data and correlation from another using R: commercial centers entrances and restaurant sales figures

Please, be kind, as I'm totally noob in stats and R... I'm the owner of a small restaurant in a commercial center, and I managedd to collect two main dataset, commercial-center (cc) and restaurant ...
0
votes
0answers
36 views

one way anova in R

I have only just started using R in my first year at university and I am completely stuck. I have done an anova test as the data is normally distributed and parametric. However, this is my result. I'm ...
0
votes
0answers
25 views

Full Information Maximum Likelihood, Imputation and Classification

I need to do a classification of a dataset, I have some missing data and I would like to try some "missing data techniques" to achieve the best accuracy. I already tried multiple imputation and ...
1
vote
2answers
63 views

Which interpolation technique should I use?

I have an annual data set, but I have a few missing values in the series. I do not know which interpolation technique should I use to fill the missing values. ...
0
votes
1answer
30 views

R Linear model step NA values

My aim is to carry out a generalized linear model (glm) with 1 response variable and 13 explanatory variables.Unfortunately 3 out of the 10 explanatory variables contain NA values (2/3 of data set of ...
2
votes
0answers
64 views

What methods can I use to aid in modeling a smaller data set when I have a significantly larger data set with fewer variables?

I currently have a data set with about 4,000 rows. The current model I have established for it is not very good, and I am going to receive more data for about 150 of these points, and I'm hoping that ...
1
vote
0answers
38 views

R Linear model step NA values

My aim is to carry out a generalized linear model (glm) with 1 response variable and 13 explanatory variables.Unfortunately 3 out of the 10 explanatory variables contain NA values (2/3 of data set of ...
1
vote
0answers
15 views

Strange likelihood construction for information borrowing

I recently encountered a strange model while consulting in industry. The goal was to collect information about the parameter of event $A$, say $\theta_A$, but we only observed events $B$ and $C$, ...
0
votes
0answers
8 views

Running a Two-Way Repeated Measures ANOVA with missing data

I'm trying to run a Two-Way rep measures Anova on a data set with two independent variables, and three time factors for each variable. However due to the severity of the experiment, not all ...
6
votes
2answers
41 views

Extrapolation of 2d movement

I have a problem with missing data in my dataset. My dataset is timeseries which contains x,y coordinates. I'd like to extrapolate missing values and use the assumption that I know speed and direction ...
0
votes
3answers
73 views

Missing data and imputation in general

Handling missing data is a bit confusing for me. My questions are: Is it better to calculate imputations than simply leave out NAs and leave it to the (appropriate) model to handle it? Is there a ...
0
votes
0answers
40 views

R caret and NAs

I very much prefer caret for its parameter tuning ability and uniform interface, but I have observed that it always requires complete datasets (i. e. without NAs) even if the applied "naked" model ...
0
votes
0answers
11 views

Longitudinal design with missing values at first time point

I carried out an intervention with 3 time points (2 pre-intervention, 1 post-intervention) with 2 groups ( n =18 intervention, n = 22 control). Basically I am missing some observations from some of ...
2
votes
0answers
44 views

Impute missing data for mixed effects models?

Although I will not provide a reference, because I cannot recall where I did read it, I have several times read or heard that missing data is accommodated automatically in mixed models. Can anyone ...
2
votes
1answer
59 views

factor analysis with missing values

I have data on about 25 subjects and 30 variables with about 20 missing values. The data is missing at random. What will be the best approach to perform factor analysis. How is factor analysis versus ...
0
votes
0answers
12 views

Significance of m parameter in m-estimate

To assign a probability to events that have not occurred yet(for a fixed set of events), one of the simplest methods is to use the m-estimator, which is defined as the following: $$Pr(A) = ...
0
votes
0answers
29 views

Learning from streams with concept drift in real world - how to handle missing class problem?

In currently delves into learning from streams with concept drift. As more I learn I think about how I can use learning algorithms on real data. Most of drift detection algoritms to evaluate is ...
1
vote
0answers
36 views

Apply trained MDS model to new data

I have both a distance matrix and the original vectors, and am using MDS (Multidimensional Scaling) with R to generate vectors in more dimensions for the data. With dimensionality reduction (for ...
1
vote
0answers
45 views

Missing values in Time Series

Missing values are very common in large time series data. How should the missing values of a time series be estimated? Is interpolation useful or I need to forecast them from the past values?