Questions tagged [missing-data]

When the data present lack of information (gaps), i.e., are not complete. Hence, it is important to consider this feature when performing an analysis or test.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

How to replicate missing pattern for mice simulation

I have a dataset, the data has missing values, but some observations are complete. I have subsetted to make a new dataset only of the complete observations. I want to artificially create missingness ...
0
votes
0answers
5 views

Large amount of missing values in as input features for LSTM time series

I am using an LSTM to predict a time series chart from multiple other time series charts as input features. The problem is that some of these input charts have much ...
1
vote
2answers
33 views

Matching when there are missing data

In a treatment effects framework, I have a control and treatment group with some covariates on which to match (I'm using nearest-neighbor method). The control group has a no missing data, but some ...
1
vote
0answers
10 views

Factor analysis with missing data

I am looking for methods that allow me to do an exploratory factor analysis with missing data. The reason I need to account for missing data is that we will try to collect a large number of variables ...
1
vote
0answers
17 views
0
votes
0answers
31 views

Creating a composite variable with data that is missing by design

I have a dataset of 20 variables and over 2,000 observations. These variables are paired. A respondent is first asked if they endorse a goal. If they endorse that goal, a follow-up question asks how ...
1
vote
0answers
12 views

How do I handle missings with Full Information Maximum Likelihood in R?

I'm using R to perform a hierarchical multiple regression. To handle the missing values in my dataset, I'm supposed to use the Full Information Maximum Likelihood technique. I already looked at dozens ...
1
vote
0answers
12 views

Error with pool() during multivariage multiple missing impuation with mice package [closed]

Someone who is an expert! I am a new learner about missing data imputation using R and mice package. Thank you in advance for your response. I encountered an error with pool(), I cannot find any ...
1
vote
0answers
19 views

Forecasting with Irregularly Missing Data

Suppose I am supplying $N = 1000$ vendors, and I am looking for a way to predict their demand for my product over $T = 90$ days. Concretely, I hope to take some features for each vendor, such as their ...
0
votes
0answers
2 views

Backcasting or Not?

I'm doing a research about the relationship of Female Wage Gap and Internet from 1995-2018 (yearly) with fixed effect model. I found some missing data from 2 variables in 1995-2000. Should I do ...
0
votes
0answers
7 views

Change from listwise to pairwise deletion changes impact of the one variable with missings in otherwise unchanged SEM model - why?

I am puzzled and very much hope that somebody here can help me. I calculated a SEM model in which teenagers' grades were modelled as one predictor (among several) for receiving a recommendation for a ...
1
vote
0answers
21 views

How to identify whether my data follows “Missing At Random” (MAR) mechanism or not? [closed]

I was having two similar studies with two variables (anti-gE and anti-VZV (continuous variable)) linearly related to each other (with same relationship between both the studies in anti-gE and anti-VZV)...
0
votes
1answer
13 views

Interpolate or Extrapolate?

I've encountered a little problem with my research. My study is about the relationship between ICT Development and Service Trade from 1999-2018 (annually). Some of the missing data are located in the ...
0
votes
0answers
14 views

Invalid values during data imputation

I have a dataset in which several values are missing (NA). To fill the missing values, I used a couple of data imputation techniques like softImpute and MICE. While analyzing the imputed datasets, I ...
1
vote
1answer
39 views

How can I use logistic regression model to test data that is imputed in some cases?

I trained a logistic regression model and I want to test it. In some cases, I have missing features value. Is it still possible to test it? Or I must fill the missing values?
0
votes
1answer
21 views

How to calculate nullity correlation matrix? [closed]

I am able to print nullity correlation matrix using Using pandas- df.isnull().corr() (this is how it is done is missingno). But what is the maths behind it ?How is nullity matrix calculated when ...
0
votes
0answers
19 views

What is the best way to handle 2 (or more) missing values in same row?

one of the methods to handle missing data is using predictive model. If we have a row with 2 or more missing values, is it right (and accurate) to use ...
0
votes
0answers
7 views

ANOVA with random intercept - missing data

I have a basic anova model with one fixed effect (sampling type) and a random effect of site that I am running using a Bayesian approach in JAGS. I'd like to allow there to be both a random intercept ...
0
votes
0answers
24 views

Performing PCA in R with many NAs [duplicate]

I have a large dataset of 10 variables and 12,000 observations, coming from 3 types of distinct systems (200 from small ponds, 600 from rivers and 11200 from lakes). I have a lot of NAs in my ...
1
vote
1answer
30 views

How I generate more data to do analyses?

I have a datasets for 3 months, its about a vehicle movement and the gas consumption. The vehicle should refill gas after 750 km driving. But the data I have is not enough. The first month I have only ...
0
votes
1answer
28 views

Data pre-processing on test files

I am working on a classification problem. I have a training file with a label and a separate test file without a label field. I needed to remove some rows that contained missing values from the ...
0
votes
0answers
13 views

Modelling panel data - approach for dealing with missing values when analysing in wide format

My question concerns the appearance of missing values in panel data when it is converted from long to wide format. The model I am fitting (non linear distributed lag model using R package dlnm) ...
1
vote
0answers
25 views

Is there a reasonable way to do a regression analysis without knowing the correlations between correlated regressors?

I have several datasets, each with the same response variable but with different regressors. Unfortunately, I do not know the correlation between the regressors from the different datasets, but have ...
0
votes
2answers
30 views

Ranking theory for rainfall station

I have 10 rainfall stations. Each station has a rain day, no-rain day, and NA value, all in percentages. How can I rank these stations based on the condition that the best station is station that ...
0
votes
0answers
11 views

MCAR tests on mixed variable datasets [findings + discussion]

I have been spending a whole week on researching about testing on missingness mechanism in datasets and I thought it will be helpful to share what I found out about the 2 available MCAR test packages ...
0
votes
1answer
29 views

Dealing with missing data for linear mixed effects model (APIM) in SPSS

I plan to use SPSS for an actor-partner interdependence model (APIM). My sample will have a significant amount (possibly >20%) missing data on one of the predictor variables (edit: I'm emphasizing ...
1
vote
0answers
26 views

Getting an average rank when not every ranking covers the whole set

I am performing a survey study. In one question, responders were asked to select 5 factors that they consider most important out of a list of 17. They were then asked to rank those 5 selections in ...
0
votes
0answers
10 views

Handling missing target values in vector regression problem with Keras

I'm doing a vector regression problem and many of the target vectors have a few components which are missing (so these components are recorded as nans in the ...
1
vote
1answer
35 views

When is multiple imputation useful for multilevel models?

I am working on a longitudinal data set, with each person being measured 8 times on each dependent variable. Some of the dependent variables are continuous; some are counts (mostly with means between ...
0
votes
0answers
30 views

The data set missing values and percentage of data that would remain unaffected

Can not understand question and answer to it given on DS questions site (see link below): Q3. You are given a data set. The data set has missing values which spread along 1 standard deviation from ...
0
votes
0answers
10 views

cant seem to do random slope intercept model because I am missing values, any way around it?

I have a data set with 3 fixed effects categories region(2 levels), genus(2 levels), and food(5 levels). I am looking to see if sponges have different retention efficiency of the different food type ...
0
votes
0answers
10 views

Type of missing values in longitudinal data

In a longitudinal study (collecting data via Surveys), if some individuals are unwilling to give their data on specific occasions (for example at nights) then what type of missing-ness is there? ...
0
votes
0answers
26 views

When working with just one observation what would the variance be?

I am working with a data set where I am calculating the variance for the variable hours by gender, education group, agegrp (age group) and input. However, I am running into a situation where there ...
0
votes
0answers
34 views

How do you code missing values if 0 is meaningful?

Per this deep learning book I am reading: In general, with neural networks, it’s safe to input missing values as 0, with the condition that 0 isn’t already a meaningful value. The network will ...
22
votes
6answers
2k views

What are the disadvantages of using mean for missing values?

I have an assignment (Data Mining course) and there is a part which asks: "What are the disadvantages of using mean for missing values?" in Missing Value section. ...
0
votes
0answers
11 views

how to conduct Little’s MCAR tests in SPSS

The sample consists of 185 students. The data consists of task 2 score (the outcome) and some predictor variables. SPSS was used to test MCAR. I included three predictors that are moderately related ...
0
votes
1answer
34 views

Little's MCAR test Chi-Square =.000

I am trying to understand the results of my Little's MCAR test (SPSS 26). Chi-Square = .000, DF = 2113, Sig. = 1.000 As much as I have read articles reporting Little's MCAR test, nobody reports Chi-...
0
votes
0answers
20 views

Theoretical Panel Data Question - Take effects of a external variable

Bellow, I have some samples of income levels for some countries. I am using these levels only to filter my data when running a Panel Data Analysis. I read a lot and could not find any info about how ...
1
vote
1answer
29 views

How to deal with intentionally missing data

I have a dataset describing a vehicle's sensors. One of the sensors records the distance from cars in other lanes. Sometimes there are no cars to the right or the left of the vehicle and this is ...
0
votes
0answers
5 views

Solutions for a missing data on an aggregate continous variable for a regression

I want to make a differences on differences regression model. Let's say I have: Y = B0 + B1*D + B2*X + E Where (excuse my lack of knowledge on how to add i's and j's indicating observations across ...
0
votes
0answers
9 views

how PMM imputation works for categorical explanatory variable?

I'm currently using the PMM method in my dataset, which is composed of two categorical variables (such as family and genera, these are my explanatory variables) and one continuous variable for species ...
0
votes
0answers
15 views

unbalanced three way ANOVA with missing data from one level of a factor

I wanted to make sure I was doing everything right, I believe I am supposed to do a three-way ANOVA but correct me if there is a better way to go about it. I have data on the feeding retention of ...
0
votes
1answer
21 views

How to handle truncated or missing ranking data in a classification problem?

I'm preparing data for a classification problem that involves matches in a single-player sport. In each match, each competitor is either ranked and thus has a numeric rank; or unranked (rare but can ...
1
vote
1answer
57 views

Fill or not to fill? That's the question

I have a dataset which shows the expense of users in a specific expense category daily, along the time. I am building a time series in order to predict whether this person will buy this some product ...
0
votes
1answer
26 views

Is a questionnaire with a systematically missing item still usable for analysis?

Due to a glitch a single item of a validated questionnaire was not submitted to the participants. Instead of having 20 questions, only 19 were submitted. The literature about the questionnaire tells ...
0
votes
0answers
40 views

E-step of E-M algorithm with missing data

I am learning expectation-maximization (E-M) algorithm on Coursera and during the course the teacher says that it can be used to handle missing data when fitting Gaussian mixtures (GM) but did not ...
1
vote
1answer
56 views

Use Expectation-Maximization algorithm for obtaining maximal likelihood estimator

For $X = {(Z_{i}, Y_{i}) : i = 1, ... ,n}$, consider the model: $Y_{i} = \beta_{1} + \beta_{2}Z_{i} + \epsilon_{i}$ where $\epsilon_{1}, ... ,\epsilon_{n}$ are i.i.d $N(0,\sigma^2)$, $Z_{i},...Z_{i}$ ...
0
votes
0answers
12 views

Classifiy missing data

I have a data set for two years. In one year every level of factor A has been sampled, but two levels twice as much. In the other year only two levels have been sampled. In addition, due to technical ...
0
votes
0answers
29 views

How to treat large chunks of missing data when ROC-AUC scoring is used

I have a dataset with a majority of features for about one-third of both train and test data are missing. E.g. I have values A, B, and C for 66% of my data and only C's for the rest of the data. The ...
1
vote
0answers
23 views

python find the optimal # of cluster for K-Means algorithm

I have a data that contains 24 features and all features have some missing values. I want to use the impute-KNN algorithm from sklearn to fill the missing values. However, before I do that, I think I ...

1
2 3 4 5
25