When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing a analysis or test.

learn more… | top users | synonyms (2)

3
votes
0answers
41 views
+50

Missing values with Community structure in networks?

Is there a way to predict Missing values with Community structure in networks? I have a data set with a couple dozen variables, such as age, level of education, self-assessed (via a Likert scale) ...
0
votes
1answer
30 views

NA values in linear model in r

I have the following dataset: desingmatrix <-read.csv("path of csv with data", sep=";", dec=".") View(desingmatrix)# vision de los datos Then I try to set ...
1
vote
0answers
16 views

Repeated Measures Analysis for a Single Treatment with Missing Data

I have some data here that I am at a loss on how to analyze. The following data set represents a subset of some data I have where a number of subjects (4 here, about fifty in the actual set) ...
0
votes
0answers
12 views

Using gbm in R with non-random missing data

The way gbm handles missing variables in R is by using surrogate splits. Is this appropriate to use when the data is not missing at random?
0
votes
1answer
29 views

Multiple Imputation and Regression Model Diagnostics

When I run regression analysis I find it important to run some model diagnostics, such as detection of outliers, influential observations, multi-collinearity (much like these examples ...
1
vote
1answer
20 views

How to create scaled scores using Likert Data with N/A options?

I have 7 survey questions that I asked students about regarding their online learning experience. Each question is on a 5 point Likert scale including -1 for not applicable. Now that I have the ...
0
votes
1answer
27 views

Self-Organising Maps and missing data (NAs) in R

The SOM algorithm should be able to deal with some datapoints containing NAs: to find their Best Matching Units, it would be possible to compute Euclidean distances with the neurons ignoring the ...
2
votes
2answers
135 views

Machine learning feature encoding

I'm new to Machine Learning. I've just finished the Coursera course. :) And for my first practical attempt I wanted to "analyse" a local used cars selling website in order to compose a modal that ...
3
votes
2answers
74 views

How to deal with values that don't exist, as opposed to are missing?

I am working with a dataset where the dependent variable is $y$ (level of use of a line of credit) and the key independent variables are $x_1$ and $x_2$ (two different types of interest rates). Some ...
2
votes
1answer
13 views

Using variables that are only available for part of the data-set in a classification model

I have Data X1, X2, and y. X1 has the same variables as X2, + some extra variables that X2 does not have. I want to use the data X2 to predict binary variable y. I suspect the extra variables In ...
0
votes
0answers
26 views

Model selection and performance evaluation using cross-validation for time series with missing values

So my task is to select and evaluate a statistical model (random forest, boosted trees, neural networks etc.) for a time series with missing values around 10 years long. One of the goals of that is to ...
0
votes
0answers
27 views

Missing data in Multilevel Longitudinal Model with Stata

I normally use xtmixed in Stata to test hierarchical linear models (e.g. performance of students nested in schools). Now it's the first time I need to test a ...
0
votes
0answers
10 views

An observation too short / missing data in panel

I have a panel data set with 7 lines or concepts from 1948 to 2013. However there is an 8th concept that I need that is only from 1993-2010. Is there a way in which I could estimate this variable's ...
1
vote
0answers
11 views

Strength of Little's MCAR

My Little MCAR is significant, but i use a dataset of N=10.000, so almost everything is significant. Is it possible to see if the Missings are really not random, or that its just becouse of the huge ...
0
votes
0answers
30 views

Itemfit to IRT model with missing data

I have matrix of dichotomus correct/false answers with many random missing data. (The data comes from an ability test where questions were randomly drawn from an item bank.) I am trying to find out ...
1
vote
3answers
39 views

How to compare 2 predictive models where one uses predictor with missing values

I am developing a model to predict y from a dataset (N=20,000) that contains x1, x2. Say I ...
2
votes
1answer
24 views

Hidden Markov Model to fill missing elements in a sequence

In my project I have a set of sequences (elements are letters from English alphabet) and some of the sequences have missing elements. I need to fill them with the most probable elements. I've been ...
0
votes
1answer
82 views

“The EM algorithm failed to converge in 25 iterations”

When I Replace Missing Values - Expectation-Maximization in SPSS, I receive the following message: The EM algorithm failed to converge in 25 iterations. Should the algorithm be able to converge? Can ...
3
votes
2answers
52 views

Survey analysis with missing data by design

I have a survey with 400 responses looking at the satisfaction of customers with a company's service overall, as well as on various specific aspects (website, account manager, invoicing, etc.). ...
0
votes
0answers
22 views

Imputing missing values of predictor for use in Regression Models

I have a panel data set that extends from January 2013 to July 2014. The response variable is complete for the entire period, however all of the predictor variables have values only up to June 2014. ...
0
votes
0answers
44 views

R MICE imputation failing

I am really baffled about why my imputation is failing in R's Mice 2.22 package. I am attempting a very simple operation with the following data frame: ...
0
votes
1answer
24 views

Principled way of combining time series with different spans and granularity into an econometric model

I want to forecast the price of something given various time series as inputs. The problem is that they are of different frequency (annual, quarterly, monthly, daily) and time periods (the more ...
1
vote
0answers
31 views

Find Weibull parameters from incomplete sample

I have samples with known number of missing elements (expressed as NaN's) that range from 10%-90% missing. The same function, Weibull, works for all cases. My question is then, how can I ...
1
vote
1answer
41 views

Dealing with missing values where the question was not asked

I have a question about missing values. We used 3 versions of a questionnaire were the possible answers were numeric (0-10) with 0 = no pain and 10 = worst possible pain. We had 4 pain questions ...
0
votes
0answers
65 views

fitting a model for time series data

Folks, I am working on time series traffic data where the waiting times are indexed over time, with 288 observations for 24 hour time period (interval of 5 minutes). I am trying to cleanse the data, ...
3
votes
1answer
36 views

Is there any rule of thumb to delete a variable in a large data set?

I'm working with a large set as a project for the business analytic course with $10^5$ observations and 170+ variables, some of which come with a missing value proportion of larger than 20%, even more ...
0
votes
1answer
8 views

GLM - X.intercept equal to NA [duplicate]

What does it mean X.Intercept equal to NA as a result of glm summary ? Thanks. Coefficients: (1 not defined because of singularities) ...
0
votes
1answer
26 views

identify nature of missingness for categorical variables

could you please give me some hints for identifying the nature of missingness for categorical variables' missing value? I mean, I gave a fast search on google scholar but I didn't find anything ...
0
votes
0answers
36 views

Handling Sparse Data Frames - algorithm selection

I am new to machine learning/statistical modelling. I am trying to run a classification on a highly sparse dataset with 100 features, most of which are categorical (TRUE/FALSE) with the remaining ...
0
votes
0answers
30 views

Missing Values in Dependent Variable and the Question on Tobit model (Panel Data)

I would appreciate some help and guidance on this issue. I have a panel data that looks like this. What I want to do is use the FDIflow variable as a dependent ...
0
votes
1answer
24 views

How do I deal with missing data for a repeated measure collected over time?

I have a real-life data set, with only one "measured" variable (i.e., patient waiting time). This single variable is collected weekly, in different clinics, for different providers, across time. I am ...
2
votes
0answers
59 views

Missing data not at random - Advice needed on method

I have been developing a logistic regression model based on retrospective data from a national trauma database of head injury in the UK. The key outcome is 30 day mortality (denoted as "Survive" ...
1
vote
1answer
78 views

How to handle missing data in a small $n$ large $k$ machine learning scenario?

I have a sample size $N=130$ and $1000$ variables. I am using machine learning techniques (SVM) for analysing the data. Some variables in the dataset have values that are so huge that they must be ...
0
votes
0answers
17 views

Incomplete inhibition curve fitting with Graphpad Prism6

One of my data set shows incomplete inhibition (for high concentrations points, inhibition response goes up to 45%-50% (of control). I am fitting my data using log(inhibitor) vs response -Variable ...
0
votes
0answers
18 views

Expectation maximisation for right-censored iid data from Normal

This is the data (which are length of ropes), $\textrm{Data}=\{99, 70, o ,89, 88, o, 88,70, o ,o\}$, where $o$ are censored data with value above $100$. Assume that data are from $\textrm{iid} \sim ...
0
votes
2answers
132 views

Single EM imputation with R (using Amelia or other packages)

I am trying to impute missing values with R. I would like to use the EM algorithm for that. As it seems this algorithm is implemented in the ...
0
votes
0answers
50 views

Estimating NA values in WinBUGS model

I am trying to fit a WinBUGS model with missing (NA) values as part of the input data. I'd like to use WinBUGS to generate missing estimates for the NA values from the full conditional distributions ...
0
votes
0answers
23 views

Imputation for small cells?

I have a question about the data requirements for imputation. I am trying to calculate the price elasticity of demand for service X where there are four possible provider types for that services. In ...
6
votes
1answer
136 views

Should additional crime reports about someone change our level of doubt about an initial crime report?

Edit: Note that this question is not about multiple unreliable witnesses to the same incident, but rather multiple incidents with only one witness each. Should the accumulation of separate alleged ...
0
votes
1answer
36 views

Factor analysis with categorical reponses and missing data

I factor analyzing a measure with 55 categorical items (3 categories each). I am use CFA to test a 7 factor model. I have a very large sample (>10,000), but approximately 20% of the sample is missing ...
0
votes
0answers
13 views

Conditional Distribution of an Independent Variable for missing data

Let $X=[X_1 X_2 X_3 ... X_p] $be a matrix of p independent variables where $X_i=[x_{i1} ... x_{in}]'$ is a nx1 vector. Let W be a nxn weight matrix based upon queen contiguity (so zero's along the ...
0
votes
0answers
62 views

Obtain factor scores in data set with missing values

I would like to obtain factor scores after factor analyzing data that contain missing values. I'm using Stata 13 to run the analysis. Here is the basic code (borrowed from the UCLA site): ...
2
votes
1answer
146 views

What are the pros and cons of using median imputation to handle missing value?

I have to choose between median or mean imputation to handle missing values. I feel median imputation will work better because it is a number that is already present in the data set and is less ...
1
vote
1answer
42 views

Imputation of a binary variable by Bayesian logistic regression

In the book "Flexible Imputation of Missing Data" by Van Buuren, the following algorithm is presented I think I understand the algorithm as given, but I would like to know what is the "more ...
2
votes
1answer
138 views

Missing factor levels after logistic regression glm()

I am quite new in the R universe, so please excuse me if the question is too simple.. I would like to perform a logistic regression on a marketing data set (only categorical variables), of the form ...
0
votes
0answers
26 views

MAR Vs. MNAR: attribute's missingness determined by the value of another attribute

Suppose I have a training data set that predicts the whether a person is unemployed or not using a decision tree. I have a certain set of attributes that I use to predict this. But for all samples ...
6
votes
3answers
88 views

Managing 'prefer not to says' in sensitive questionnaires

Consider a questionnaire where we ask someone about their sexuality. The five options, for simplicity, are: Heterosexual Homosexual Bisexual Other 'Prefer not to say' Assume we ask the population. ...
3
votes
3answers
259 views

How do I handle nonexistent or missing data?

I tried a forecasting method and want to check if my method is correct or not. My study is comparing different kinds of mutual funds. I want to use the GCC index as a benchmark for one of them but ...
0
votes
0answers
29 views

(binary) Matrix completion with less known data

Recently, I meet such problems, I call it matrix completion problem. For example, the row denotes the users and the column denotes items. And If one user like the ...
2
votes
0answers
48 views

Imputation of missing values for doing PCA in R [duplicate]

I have a dataset with approximately 4000 rows and 150 columns. I want to predict the values of a single column (= target). The data is on cities (demography, social, economic, ... indicators). A lot ...