Questions tagged [censoring]

The process of censoring yields data w/ only partial information. The most common example of censoring is *right censoring* in survival analysis, where the time until the event occurred is only known to be longer than some duration because the event had not occurred when the study ended.

Filter by
Sorted by
Tagged with
26
votes
3answers
6k views

How to model this odd-shaped distribution (almost a reverse-J)

My dependent variable shown below doesn't fit any stock distribution that I know of. Linear regression produces somewhat non-normal, right-skewed residuals that relate to predicted Y in an odd way (...
10
votes
1answer
8k views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
24
votes
3answers
2k views

Unbiased estimation of covariance matrix for multiply censored data

Chemical analyses of environmental samples are often censored below at reporting limits or various detection/quantitation limits. The latter can vary, usually in proportion to the values of other ...
33
votes
2answers
25k views

What is the difference between censoring and truncation?

In the book Statistical Models and Methods for Lifetime Data , it is written : Censoring: When an observation is incomplete due to some random cause. Truncation: When the incomplete nature of ...
6
votes
1answer
2k views

Problem with informative censoring

I am reading "Monte Carlo Statistical Methods" by Robert and Cassella, and problem 1.3 asks In example 1.1, the distribution of the random variable $Z=\min(X,Y)$ was of interest. Derive the ...
14
votes
2answers
4k views

Layman's explanation of censoring in survival analysis

I have read about what censoring is and how it needs to be accounted for in survival analysis but I would like to hear a less mathematical definition of it and a more intuitive definition (pictures ...
14
votes
5answers
14k views

What exactly are censored data?

I have read different descriptions of censored data: A) As explained in this thread, unquantified data below or above a certain threshold is censored. Unquantified means data is above or below a ...
12
votes
1answer
829 views

Modelling when the dependent variable has a “cut-off”

Apologies in advance if any of the terminology I use is incorrect. I'd welcome any correction. If what I describe as a "cut-off" goes by a different name, let me know and I can update the question. ...
17
votes
1answer
4k views

What is “Targeted Maximum Likelihood Expectation”?

I'm trying to understand some papers by Mark van der Laan. He's a theoretical statistician at Berkeley working on problems overlap significantly with machine learning. One problem for me (besides ...
6
votes
1answer
17k views

Difference between independent and non-informative censoring

I was wondering if I could get a third opinion to settle a discussion on the distinction between independent and non-informative censoring. My definitions: 1) In independent censoring, the event ...
7
votes
1answer
6k views

How should I handle a left censored predictor variable in multiple regression?

I have a dataset (N=350) for which I would like to regress a neuropsychological test score (continuous) on age, education, symptom severity (continuous), and diagnosis (binary). Symptom severity is ...
10
votes
3answers
26k views

Right censoring and Left censoring

Wikipedia gives the following definitions: Right censoring: a data point is above a certain value but it is unknown by how much. Left censoring: a data point is below a certain value but it is ...
5
votes
0answers
116 views

Censored logit transform for (ad hoc) exploratory data analysis

In my work I commonly have to analyze binary composition data, expressed as a fraction $f\in[0,1]$. The data $f[x]$ is spatially distributed ($x\in\mathbb{R}^n$, $n=1,2,3$), and typically comes in the ...
3
votes
1answer
209 views

Weighted normal errors regression with censoring

I have some data which I would model via standard multiple regression except: There is censoring (left-censored, fixed but varying censoring points which are known) The errors are assumed independent ...
2
votes
2answers
77 views

Calculate E[X] from incomplete data?

The exercise I'm doing describes the random variable $X$ as the following ...
54
votes
3answers
2k views

Do we have a problem of “pity upvotes”?

I know, this may sound like it is off-topic, but hear me out. At Stack Overflow and here we get votes on posts, this is all stored in a tabular form. E.g.: post id voter id vote type ...
11
votes
2answers
8k views

How should I model a continuous dependent variable in the $[0, \infty]$ range?

I have a dependent variable that can range from 0 to infinity, with 0s actually being correct observations. I understand censoring and Tobit models only apply when the actual value of $Y$ is partially ...
9
votes
3answers
384 views

If $Z_i =\min \{k_i, X_i\}$, $X_i \sim U[a_i, b_i]$, what is the distribution of $\sum_iZ_i$?

Assume the following set up: Let $Z_i = \min\{k_i, X_i\}, i=1,...,n$. Also $X_i \sim U[a_i, b_i], \; a_i, b_i >0$. Moreover $k_i = ca_i + (1-c)b_i,\;\; 0<c<1$ i.e. $k_i$ is a convex ...
7
votes
2answers
334 views

Informative censoring caused by cesarean section

I study labor and delivery as an epidemiologist. It is well established that a large fetus has a higher risk of causing maternal birth trauma. But a large baby is also likely to be delivered by ...
5
votes
1answer
679 views

Fitting distributions on censored data

My question deals with fitting distributions on censored data; for the purposes of clarity, we can consider a continuous distribution which is both left and right-censored. In such a case, the ...
4
votes
1answer
3k views

Can a Cox proportional hazards model be used on general left-censored data (i.e. non-survival data)?

A familiar problem in applied science is having censored data, in particular left-censored data arising due to an assay or piece of equipment having a lower limit of detection (LOD). Assuming a ...
9
votes
1answer
375 views

What are the mean and variance of a 0-censored multivariate normal?

Let $Z \sim \mathcal N(\mu, \Sigma)$ be in $\mathbb R^d$. What are the mean and covariance matrix of $Z_+ = \max(0, Z)$ (with the max computed elementwise)? This comes up e.g. because, if we use the ...
8
votes
5answers
510 views

How can I estimate the time at which 50% of a binomial variable will have transitioned?

I have the following data, representing the binary state of four subjects at four times, note that it is only possible for each subject to transition $0\to 1$ but not $1\to 0$: ...
6
votes
2answers
700 views

How to cope with missing data in logistic regression?

I'm investigating optimal bidding in auctions, and am using logistic regression to predict the probability of winning an auction given a set of explanatory variables (e.g. the price I bid, number of ...
5
votes
1answer
3k views

Fit a Weibull distribution to…right-censored data?

Premise: I know next to nothing about survival analysis, I just started. I have a vector of failure times for some machines. Most of the machines (3362 vs 2694) are still running today, so I know they ...
4
votes
0answers
328 views

Estimating right-censored data

I am VERY new to stats. I have a large amount of life-time data (delay in arrival since start of experiment) from repeat experiments. Some data is missing, but essentially represents a delay longer ...
4
votes
1answer
495 views

Simple case of MNAR missing data

I believe I have a very simple problem with missing data, but I'm a bit lost because all the materials I read seem to be focused on much more complicated cases. I have a random variable $X$ which has ...
3
votes
1answer
126 views

Deriving the distribution of the sum of censored variables

I want to be able to calculate the distribution of $$Y = \sum_{i=1}^n\max\{0,X_i\}$$ where the random variable $X_i\sim N(\mu_i,\sigma_i)$. Is the calculation of $f_Y(y)$ possible and if so what is ...
2
votes
1answer
741 views

LASSO or other regularized regression with censored (missing) data

Here is my problem. I am looking at various time series curves. Let's call them total spend aggregated over all customers on various products versus time. At any given time, I want to predict the ...
2
votes
0answers
632 views

Statistics for rounded data

I wonder about the existence of "standard" statistical procedures for rounded (log)normal data. Indeed, in my work I often encounter rounded data which potentially cause some problems: "awful" qqplots,...
2
votes
2answers
1k views

Finding the mean of right-censored values

I wanted to check how fast an average colleague of mine is able to complete a puzzle, so I ran some experiments. The problem is they weren't done in exactly 50 minutes, I always got bored and moved on ...
2
votes
1answer
315 views

Learning a continuous model from binned data

A very similar question has been asked before, but it didn't get a real answer. Background I would like to develop a probability model for a continuous, ratio-scale random variable $Y$. Let's say it ...
2
votes
2answers
4k views

How to get R squared/goodness of fit for Tobit model in R?

I do a Tobit regression to analyze censored data. To measure the goodness of fit the authors of these papers use Efron's $R^2$. So my idea is to use this one as well. To realize my Tobit regression, ...
0
votes
1answer
99 views

Handling survival data where people join at various stages

My question is about survival analysis. According to this censoring is a condition in which the value of a measurement or observation is only partially known. For example, suppose a study is ...
5
votes
2answers
993 views

Inference with only left-censored data

Suppose I have a data set that is only left-censored data, ex: <5, <5, <5, <10, <10, <10 A technique to handle left-censored data is the Kaplan Meier estimate, see page 5 of this ...
4
votes
2answers
172 views

Should one use a censored survival model when an event is only observed at death?

A colleague of mine is trying to estimate how neutron radiation exposure changes cancer incidence rates (in mice). He has autopsy data that reports whether a cancer was observed at the time of death, ...
4
votes
1answer
745 views

How to estimate parameters of a distribution with left-truncated and right-censored data?

I have been trying to find the best way of estimating parameters for a known pdf from a data-set that is left-truncated and right-censored. More precisely, I have lifetimes for a system where there ...
3
votes
1answer
42 views

Poisson regression on devices that fail during observation period?

I am observing N devices over a period of time $[0,T]$ and counting the number of certain events $y_j$, for each device, during that period of time. I also have some specifications $x_j$ for each ...
3
votes
0answers
688 views

Right-censored independent variable in Cox/logistic regression

I have a right-censored continuous independent variable that I want to include in a Cox regression. The variable is a physiologic test which is capped at a certain time, say 120 seconds, due to safety ...
3
votes
2answers
61 views

Inference for a distribution with different number of samples due to censoring

I have the following problem in case someone has an idea about how to solve this. Assume three experiments that refer to the same population for a random variable $X$. In the first experiment, I ...
2
votes
1answer
509 views

Poisson regression for ordered variables

I have three waves of data, and I am trying to estimate group-based trajectories of binge drinking across the three waves. The question asked (at all three waves was): “Over/During the past 12 months, ...
2
votes
1answer
1k views

Method for fitting censored data in R

I have 320 data points - each has a redshift and a turnover-frequency, and I want to fit a correlation between them (a linear fit). However, 120 of the turnover-frequency values are upper limits. As ...
2
votes
1answer
323 views

Version of Mann-Whitney U test, for right-censored data

Is there an equivalent of the Mann-Whitney U test, but for right-censored data? I have right-censored numerical data from two different groups. The mean for one group is higher than the other. I'd ...
2
votes
1answer
3k views

Best way to handle na values in regression predictor variable

I am running a multinomial logistic regression model in R (using the multinom function from the nnet package) with a set of 12 ...
1
vote
1answer
104 views

Maximum Likelihood Estimator for Censored Data

Let $X^n=(X_1,X_2,...,X_n)$ denote a sample where (1) $X_i=\mathbf 1_{(\epsilon_i + \mu \geq 0)}(\mu+\epsilon_i)+\mathbf 1_{(\epsilon_i + \mu \leq 1)}(\mu+\epsilon_i)+\mathbf 1_{(\epsilon_i + \mu &...
1
vote
1answer
315 views

Likelihood of censored data

Let $X_1,X_2,\ldots, X_{n_1}$ be IID with PDF $f(x-\theta) $, for $-\infty<x<\infty$ and $-\infty<\theta<\infty$. Denote the CDF of $X_i$ by $F(x-\theta)$. Let $Z_1,Z_2, \ldots, Z_{n_2}$ ...
1
vote
0answers
371 views

Is it possible to model BOTH censoring and truncation in BUGS?

Survival times are often right censored and left truncated. From my experience, it does not seem like OpenBUGS allows for both. Truncation is denoted as T( , ) and censoring as C( ,). For instance, a ...
1
vote
2answers
250 views

Time-to-event data with low censoring

I have data on individuals, consisting of an individual identifier column, starting date column and length date column. This indicates an individual being in financial distress for a specific period ...
0
votes
0answers
16 views

Aggregating (taking the mean of) a censored variable

This post is related to this post. I have survey data, with a (censored) range from 0-100, that looks as follows: ...