Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Refers to a general class of methods used to "fill in" missing data. Methods used for doing this typically are related to interpolation (http://en.wikipedia.org/wiki/Interpolation) and require assumptions about why the data is missing (e.g. "missing at random")

1
vote
0answers
15 views

Shouldn't we consider larger standard errors for effect measures or outcomes that are converted in meta-analysis?

There are methods to convert effect measures in meta-analysis (pdf). There are also methods to convert outcomes; at least, I am aware of the conversion described in Furukawa et al. (2005) from ...
0
votes
0answers
11 views

what to do if the missing data in one column is based on some value/condition in another column in r?

I have a dataset with 20,000 observations and 19 variables. To start off with I have a gender column which has three levels namely 'M', 'F' and 'U' where U can be taken as not disclosed. Whenever ...
0
votes
0answers
12 views

How do I impute clustered data that is not time-series data?

The goal of my research is to understand whether MRI imaging characteristics can predict tumor pathology. The data consists of resected tumor samples, with multiple samples per patient. On the MRI, we ...
1
vote
0answers
13 views

Why are missing values MNAR harder to impute than MCAR or MAR?

Reading papers related to the imputation of missing values related to the -omics field, systematically imputation algorithms were less accurate when imputing MNAR compared to imputing MCAR. My ...
0
votes
1answer
23 views

Imputing values with linear regression, valid strategy or creating biases?

I am practicing on the titanic competition from kaggle. In the dataset the Age variable has a number of missing values and I am now left with the choice of what to ...
0
votes
0answers
19 views

Measurements to deterministic value

I have a number of measurements of two variables: the number of products, the weight. Sometimes the weight is missing and sometimes the number of products is missing. I want to use the given data to ...
0
votes
0answers
17 views

Imputing missing data with MICE where each observation has different levels

I have a set of observations that each consists of different levels. For example, I ask a $P$ individuals $N$ questions, each question with a possible $k_n$ discrete responses. This produces a table ...
0
votes
1answer
16 views

Imputing binary variable when no 0s, only 1s are available

I'm trying to impute missing values for a binary variable (values 0 and 1) with some challenging data (of about 1 million observations). The data can be divided into two groups: in group 1, we know ...
0
votes
1answer
62 views

Predictive Mean Matching as Single Imputation?

Multiple imputation is known to be advantageous compared to single imputation. However, in practice there are often non-statistical reasons why multiple imputation can not be used (e.g. the data ...
3
votes
2answers
91 views

data imputation of missing values in non-normally distributed explanatory variables

I have been told that mean imputation of missing values is inappropriate when the variables underlying distribution is non-normal. my variable is contiunous (but bound at 100) and most observations ...
0
votes
0answers
8 views

While performing label encoding or imputation, what should i do to the column with mostly 0's as values which is irrelevant to what column is about?

My DataFrame consists of 2919 rows. Now ,For example I have this column "2ndFlrSF" 2ndFlrSF: Second floor's Area in square feet and these are the values in it after i run my Pandas command ...
0
votes
0answers
12 views

Imputation in irregularly spaced time series data

I have irregular time series data containing missing values (called A). What I need is a regular time series with imputed values (called B). A spans roughly 3 years. Some days have multiple ...
0
votes
0answers
20 views

Imputing missing values in time series with Arima

I am using Arima and Kalman smoothing to impute missing values in univariate time series (similar way to this post: https://stats.stackexchange.com/questions/104565/how-to-use-auto-arima-to-impute-...
3
votes
2answers
55 views

Handling missing data for participants who have not completed any standardised measures and have only provided demographic answers

When managing missing data, how many questions should participant have completed, at a minimum, before imputing the remainder of their missing data? For example, a number of my participants only ...
0
votes
1answer
16 views

Imputation to Result in Known Total

I am using R and Amelia to impute missing data for the number of homeless children in several locations. There is information ...
0
votes
0answers
43 views

Extracting Factor Scores of Latent Variables after CFA in AMOS

I plan to extract the latent variables' factor scores after conducting Confirmatory Factor Analysis (CFA). I will use these factor scores as explanatory variables for my next statistical procedure - ...
0
votes
1answer
35 views

Predicting the probabilities of sales opportunities

I want to predict the probabilities of sales opportunities using a binary classification algorithmn. However after using logistic regression my results do not seem realistic. This could be due to ...
0
votes
1answer
31 views

Impute binary outcome variable for GLM using Stan in R

My outcome variable is a series of Bernoulli trials where some values are missing y $\in$ {0, 1, NA} How do you impute NA values for an outcome variable in rstan in the context of a GLM, assuming ...
1
vote
1answer
40 views

Imputing missing outcome data

I saw the other link (Multiple imputation for outcome variables) discussing missing outcome data imputation for complete case analysis. However, I have missing outcome data as well as missing ...
0
votes
0answers
12 views

Is it possible to preserve a imputation solution after adding a new variable with missing data?

I have imputed a dataset with missing values using MICE in R--all analyses have been completed. After the fact, I wanted to keep everything unchanged (so as to not repeat the analyses) but still add a ...
1
vote
0answers
34 views

Is there a way to estimate regression coefficients?

I'm currently working on a simulation study (based on empirical data) and for this simulation I created a model with multiple interaction terms. The interaction terms are between categorical variables,...
1
vote
0answers
36 views

Space-Time Principal Component Analysis with Missing Lat/Long Data

Thank you for your help, I am looking to run a space-time Principal Component Analysis on Shotspotter data from Brockton, MA: http://justicetechlab.org/data/. Shotspotter sensors record the timing, ...
2
votes
0answers
88 views

Overcome NA's in Random Forest and SVM?

I am using clinical data for prediction purposes with SVM and RF. Two of my columns are as following: ...
0
votes
0answers
30 views

Theoretical question about missing data in time series for prediction

I work with an ARIMA model with external regressors on a data set with data of two years at daily level. A central and important variable (let's call this variable "X") I use as an external regressor (...
2
votes
1answer
63 views

Forecast (impute) missing discrete values in multiple time series

I'm looking to forecast (impute) missing discrete values in multiple time series in order to reach a target volume in a consolidated time serie. The context: I have salesmen that are selling ...
1
vote
1answer
36 views

Drop data Vs fill data. Which one least hurts the integrity of the data?

I have a dilemma for an analysis I'm currently on. I doing some GARCH modelling of bitcoin and a fiat currency. There are some null values with the fiat datasets in comparison with bitcoin data as ...
0
votes
1answer
63 views

How does imputation work? I'm struggling to understand it

I have a short question. I am implementing Scikit-Learn in Typescript and currently blocked at understanding & implementing imputer (mean and regression strategies). Based on the example given ...
1
vote
0answers
49 views

Troubles with weekly data: reconciling weeks with months and years

I am working with weekly data and I am running into troubles when analyzing them because of their inconsistent nature. By inconsistent nature, I mean that not every month is made of the same number of ...
1
vote
0answers
17 views

Modeling when missingness will go away?

Are there any guidelines for how to create a model using data whose features have some missing values that will predict well when data quality improves and there is no longer missing values? I have ...
1
vote
0answers
31 views

Mice: partial imputation using where argument failing [closed]

I encounter a problem with the use of the mice function to do multiple imputation. I want to do imputation only on part of the missing data, what looking at the help seems possible and straightworward....
0
votes
0answers
16 views

Handle missing features [duplicate]

I am participating in a Kaggle competition and I would like to know what is the best way to handle missing attribute values in test data set. For example, if the train data set contains the attributes ...
0
votes
0answers
7 views

Repeated imputed values effect on distribution

In comparing two samples, one of which (red line) shows a consistent number of repeated imputed values creating a separate peak, would it make sense to use a parametric test? What are potential ...
0
votes
0answers
55 views

Is there a way to impute missing values by clustering method?

For personal knowledge, I've been trying out different imputation methods other than the mean/median/mode. I was able to try out KNN, MICE, median imputational methods so far. I was told that ...
0
votes
0answers
9 views

Error Detection and missing data imputation in Wireless Sensor Networks

I'm working on wireless sensor networks and I wish to be capable of detecting if there are any outliers in the sensed data as well as imputing missing ones. I read a lot of articles which made me more ...
0
votes
0answers
16 views

Is it valid to use Random Forest imputation in blocks and combine results into a final dataset?

I have an extremely large dataset (26 variables and 105,556 observations) with missing values and I would like to use Random Forest imputation to impute some missing values. Since the dataset is so ...
0
votes
0answers
108 views

Balanced bootstrap

I am using aregImpute to impute data but run into the following problem: ...
1
vote
0answers
226 views

Imputing missing values and SVD

Similar questions have been asked a lot of times but I have not found an answer that gives an intuitive explanation as to why this works. For reference I have read the answers here and here. As I ...
0
votes
0answers
86 views

Can I use the 'mice' library in R to impute missing data separately for each of two groups in the same dataset?

I would like to use the 'mice' library in R to impute data from a clinical trial, in which I have two groups (i.e. var="group" [0=control; 1=intervention]). I want to impute the missing data ...
2
votes
0answers
25 views

To impute or not - community consensus for reporting accuracy of an imputed model

I have a model generated using an imputed data set with imputation accuracy of 75%. If the model using imputed data has an accuracy of 80% What would be the community consensus to report the ...
0
votes
0answers
18 views

testing the accuracy of imputation methods

Hey I have a dataset of wind speeds with known data points. I am doing a project which test the accuracy and reliability of imputation methods for wind energy. The dataset consist of 14,376 data ...
0
votes
0answers
19 views

In risk prediction models, should development and validation be separated prior to imputation

Suppose you want to develop and (internally) validate a risk prediction model on a data set with missing data. Should you: 1) Separate your development and validation cohorts from the beginning, and ...
2
votes
0answers
32 views

Impute missing population values in Census data

I have population data from Census.gov: Total US population by age by year from 1940 through 2010 Depending on the range of decades, the data is missing discrete population values for ages greater ...
1
vote
0answers
23 views

Multiple imputation when have more than 1 outcome variable

Is there a good paper or reference for doing multiple imputation when there is more than one outcome variable? Anything that specifically addresses building the imputation model or software to use for ...
1
vote
1answer
47 views

One problem with imputing the missing value by the sample mean

Today I read an interesting statement, which I am not sure about its correctness. Assume we are looking at a data set with one column $H$ that is numeric. It has a bunch of missing values. Now let's ...
1
vote
0answers
27 views

Choosing, evaluating, and reporting data imputation

I have read about model checking for multiple imputation MI (https://ete-online.biomedcentral.com/articles/10.1186/s12982-017-0062-6), but I am not sure how one can check their model for a general ...
0
votes
0answers
66 views

Is any variable off limits for multiple imputation

I've read elsewhere that (despite common beliefs on the topic) that one should impute outcome variables (though this may have little benefit under MAR). My question is related, but I think, distinct: ...
2
votes
1answer
101 views

Using regression for imputing missing data

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I ...
1
vote
0answers
26 views

Multiple Imputation query

I am using multiple imputation to deal with around 49 missing observations for my outcome variable from my 324 observation panel dataset. I used Stata to perform 10 imputations for this, using ...
1
vote
1answer
2k views

K-Nearest Neighbor imputation explanation

I have a dataframe with some missing data in it. I need to deal with those missing data before trying anything. I've seen that knnImputation in R is a good ...
1
vote
2answers
63 views

How to check if the imputed value from caret is reliable? (predicting missing values)

I have a dataset which has some missing values and I tried to predict them by using caret. My data set looks like this ...