When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

2
votes
1answer
21 views

Missing data paired samples t-test

I have a list of insurance policies that filed claims in pre/post periods. I am trying to test whether the average cost of insurance claims per policy varied. Some of my policies don't have any claims ...
1
vote
0answers
25 views

Forecasting one dataset using data and correlation from another using R: commercial centers entrances and restaurant sales figures

Please, be kind, as I'm totally noob in stats and R... I'm the owner of a small restaurant in a commercial center, and I managedd to collect two main dataset, commercial-center (cc) and restaurant ...
0
votes
0answers
33 views

one way anova in R

I have only just started using R in my first year at university and I am completely stuck. I have done an anova test as the data is normally distributed and parametric. However, this is my result. I'm ...
0
votes
0answers
18 views

Full Information Maximum Likelihood, Imputation and Classification

I need to do a classification of a dataset, I have some missing data and I would like to try some "missing data techniques" to achieve the best accuracy. I already tried multiple imputation and ...
1
vote
2answers
51 views

Which interpolation technique should I use?

I have an annual data set, but I have a few missing values in the series. I do not know which interpolation technique should I use to fill the missing values. ...
0
votes
1answer
24 views

R Linear model step NA values

My aim is to carry out a generalized linear model (glm) with 1 response variable and 13 explanatory variables.Unfortunately 3 out of the 10 explanatory variables contain NA values (2/3 of data set of ...
2
votes
0answers
59 views

What methods can I use to aid in modeling a smaller data set when I have a significantly larger data set with fewer variables?

I currently have a data set with about 4,000 rows. The current model I have established for it is not very good, and I am going to receive more data for about 150 of these points, and I'm hoping that ...
1
vote
0answers
29 views

R Linear model step NA values

My aim is to carry out a generalized linear model (glm) with 1 response variable and 13 explanatory variables.Unfortunately 3 out of the 10 explanatory variables contain NA values (2/3 of data set of ...
1
vote
0answers
13 views

Strange likelihood construction for information borrowing

I recently encountered a strange model while consulting in industry. The goal was to collect information about the parameter of event $A$, say $\theta_A$, but we only observed events $B$ and $C$, ...
0
votes
0answers
4 views

Running a Two-Way Repeated Measures ANOVA with missing data

I'm trying to run a Two-Way rep measures Anova on a data set with two independent variables, and three time factors for each variable. However due to the severity of the experiment, not all ...
6
votes
2answers
32 views

Extrapolation of 2d movement

I have a problem with missing data in my dataset. My dataset is timeseries which contains x,y coordinates. I'd like to extrapolate missing values and use the assumption that I know speed and direction ...
0
votes
3answers
59 views

Missing data and imputation in general

Handling missing data is a bit confusing for me. My questions are: Is it better to calculate imputations than simply leave out NAs and leave it to the (appropriate) model to handle it? Is there a ...
0
votes
0answers
17 views

R caret and NAs

I very much prefer caret for its parameter tuning ability and uniform interface, but I have observed that it always requires complete datasets (i. e. without NAs) even if the applied "naked" model ...
0
votes
0answers
10 views

Longitudinal design with missing values at first time point

I carried out an intervention with 3 time points (2 pre-intervention, 1 post-intervention) with 2 groups ( n =18 intervention, n = 22 control). Basically I am missing some observations from some of ...
1
vote
0answers
21 views

Impute missing data for mixed effects models?

Although I will not provide a reference, because I cannot recall where I did read it, I have several times read or heard that missing data is accommodated automatically in mixed models. Can anyone ...
2
votes
1answer
37 views

factor analysis with missing values

I have data on about 25 subjects and 30 variables with about 20 missing values. The data is missing at random. What will be the best approach to perform factor analysis. How is factor analysis versus ...
0
votes
0answers
9 views

Significance of m parameter in m-estimate

To assign a probability to events that have not occurred yet(for a fixed set of events), one of the simplest methods is to use the m-estimator, which is defined as the following: $$Pr(A) = ...
0
votes
0answers
24 views

Learning from streams with concept drift in real world - how to handle missing class problem?

In currently delves into learning from streams with concept drift. As more I learn I think about how I can use learning algorithms on real data. Most of drift detection algoritms to evaluate is ...
1
vote
0answers
23 views

Apply trained MDS model to new data

I have both a distance matrix and the original vectors, and am using MDS (Multidimensional Scaling) with R to generate vectors in more dimensions for the data. With dimensionality reduction (for ...
1
vote
0answers
27 views

Missing values in Time Series

Missing values are very common in large time series data. How should the missing values of a time series be estimated? Is interpolation useful or I need to forecast them from the past values?
3
votes
1answer
48 views

Multiple Imputation Using Amelia [duplicate]

I am using Amelia for multiple imputation, and I am satisfied with the imputed results. But I want to restrict the imputed variable to positive values. Is there a way that Amelia can handle it or ...
1
vote
0answers
31 views

Best approach to predict significant factors without any complete cases

I have a dataset that contains records of donors with various biographical info (city, state, zip, number of children) and the total amount they donated over 10 years. Some never donated and thus the ...
0
votes
0answers
32 views

Handling missing data in Gaussian Process Regression

I am trying to handle missing data in a model using Gaussian Processes. I have two spatial dimensions sharing the same length hyper-parameter, one dimension for time. Additionally I have split one ...
1
vote
0answers
45 views

Regularized regression with missing data?

Are you aware of any regularized regression methods (i.e. Lasso, elastic net) which allows for using cases with incomplete (missing) data (e.g. using EM estimation)? And if yes, is the method ...
1
vote
1answer
34 views

filling out NA values using clustering analysis

I have a data frame with a large number of NA values. I do not wish to leave out all these rows as that would reduce the size of my training set drastically. I filled out these missing values in a ...
0
votes
0answers
17 views

How does missing data (not at random) affect Bayesian models?

When I was a student learning about Bayesian models, we were taught that missing data was not a problem because they would be imputed. However I am wondering about how missing not at random (MNAR) ...
0
votes
0answers
13 views

Significance of a correlation considering missing data

So I'm dealing with two timeseries, of which one contains quite some missing data (between 10-60%). I'm excluding some datasets which contain exceptional much missing data, but still I would like to ...
3
votes
0answers
62 views

Missing values with Community structure in networks?

Is there a way to predict Missing values with Community structure in networks? I have a data set with a couple dozen variables, such as age, level of education, self-assessed (via a Likert scale) ...
0
votes
1answer
38 views

NA values in linear model in r

I have the following dataset: desingmatrix <-read.csv("path of csv with data", sep=";", dec=".") View(desingmatrix)# vision de los datos Then I try to set ...
1
vote
0answers
19 views

Repeated Measures Analysis for a Single Treatment with Missing Data

I have some data here that I am at a loss on how to analyze. The following data set represents a subset of some data I have where a number of subjects (4 here, about fifty in the actual set) ...
0
votes
0answers
28 views

Using gbm in R with non-random missing data

The way gbm handles missing variables in R is by using surrogate splits. Is this appropriate to use when the data is not missing at random?
0
votes
1answer
57 views

Multiple Imputation and Regression Model Diagnostics

When I run regression analysis I find it important to run some model diagnostics, such as detection of outliers, influential observations, multi-collinearity (much like these examples ...
1
vote
1answer
42 views

How to create scaled scores using Likert Data with N/A options?

I have 7 survey questions that I asked students about regarding their online learning experience. Each question is on a 5 point Likert scale including -1 for not applicable. Now that I have the ...
0
votes
1answer
46 views

Self-Organising Maps and missing data (NAs) in R

The SOM algorithm should be able to deal with some datapoints containing NAs: to find their Best Matching Units, it would be possible to compute Euclidean distances with the neurons ignoring the ...
2
votes
2answers
179 views

Machine learning feature encoding

I'm new to Machine Learning. I've just finished the Coursera course. :) And for my first practical attempt I wanted to "analyse" a local used cars selling website in order to compose a modal that ...
3
votes
2answers
80 views

How to deal with values that don't exist, as opposed to are missing?

I am working with a dataset where the dependent variable is $y$ (level of use of a line of credit) and the key independent variables are $x_1$ and $x_2$ (two different types of interest rates). Some ...
2
votes
1answer
13 views

Using variables that are only available for part of the data-set in a classification model

I have Data X1, X2, and y. X1 has the same variables as X2, + some extra variables that X2 does not have. I want to use the data X2 to predict binary variable y. I suspect the extra variables In ...
0
votes
0answers
39 views

Missing data in Multilevel Longitudinal Model with Stata

I normally use xtmixed in Stata to test hierarchical linear models (e.g. performance of students nested in schools). Now it's the first time I need to test a ...
0
votes
0answers
12 views

An observation too short / missing data in panel

I have a panel data set with 7 lines or concepts from 1948 to 2013. However there is an 8th concept that I need that is only from 1993-2010. Is there a way in which I could estimate this variable's ...
1
vote
0answers
11 views

Strength of Little's MCAR

My Little MCAR is significant, but i use a dataset of N=10.000, so almost everything is significant. Is it possible to see if the Missings are really not random, or that its just becouse of the huge ...
0
votes
0answers
41 views

Itemfit to IRT model with missing data

I have matrix of dichotomus correct/false answers with many random missing data. (The data comes from an ability test where questions were randomly drawn from an item bank.) I am trying to find out ...
1
vote
3answers
42 views

How to compare 2 predictive models where one uses predictor with missing values

I am developing a model to predict y from a dataset (N=20,000) that contains x1, x2. Say I ...
2
votes
1answer
31 views

Hidden Markov Model to fill missing elements in a sequence

In my project I have a set of sequences (elements are letters from English alphabet) and some of the sequences have missing elements. I need to fill them with the most probable elements. I've been ...
0
votes
1answer
145 views

“The EM algorithm failed to converge in 25 iterations”

When I Replace Missing Values - Expectation-Maximization in SPSS, I receive the following message: The EM algorithm failed to converge in 25 iterations. Should the algorithm be able to converge? Can ...
3
votes
2answers
54 views

Survey analysis with missing data by design

I have a survey with 400 responses looking at the satisfaction of customers with a company's service overall, as well as on various specific aspects (website, account manager, invoicing, etc.). ...
0
votes
0answers
26 views

Imputing missing values of predictor for use in Regression Models

I have a panel data set that extends from January 2013 to July 2014. The response variable is complete for the entire period, however all of the predictor variables have values only up to June 2014. ...
0
votes
0answers
60 views

R MICE imputation failing

I am really baffled about why my imputation is failing in R's Mice 2.22 package. I am attempting a very simple operation with the following data frame: ...
0
votes
1answer
28 views

Principled way of combining time series with different spans and granularity into an econometric model

I want to forecast the price of something given various time series as inputs. The problem is that they are of different frequency (annual, quarterly, monthly, daily) and time periods (the more ...
1
vote
0answers
35 views

Find Weibull parameters from incomplete sample

I have samples with known number of missing elements (expressed as NaN's) that range from 10%-90% missing. The same function, Weibull, works for all cases. My question is then, how can I ...
1
vote
1answer
45 views

Dealing with missing values where the question was not asked

I have a question about missing values. We used 3 versions of a questionnaire were the possible answers were numeric (0-10) with 0 = no pain and 10 = worst possible pain. We had 4 pain questions ...