Questions tagged [censoring]

The process of censoring yields data w/ only partial information. The most common example of censoring is *right censoring* in survival analysis, where the time until the event occurred is only known to be longer than some duration because the event had not occurred when the study ended.

Filter by
Sorted by
Tagged with
25
votes
3answers
6k views

How to model this odd-shaped distribution (almost a reverse-J)

My dependent variable shown below doesn't fit any stock distribution that I know of. Linear regression produces somewhat non-normal, right-skewed residuals that relate to predicted Y in an odd way (...
9
votes
1answer
7k views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
24
votes
3answers
2k views

Unbiased estimation of covariance matrix for multiply censored data

Chemical analyses of environmental samples are often censored below at reporting limits or various detection/quantitation limits. The latter can vary, usually in proportion to the values of other ...
14
votes
2answers
4k views

Layman's explanation of censoring in survival analysis

I have read about what censoring is and how it needs to be accounted for in survival analysis but I would like to hear a less mathematical definition of it and a more intuitive definition (pictures ...
14
votes
5answers
13k views

What exactly are censored data?

I have read different descriptions of censored data: A) As explained in this thread, unquantified data below or above a certain threshold is censored. Unquantified means data is above or below a ...
6
votes
1answer
2k views

Problem with informative censoring

I am reading "Monte Carlo Statistical Methods" by Robert and Cassella, and problem 1.3 asks In example 1.1, the distribution of the random variable $Z=\min(X,Y)$ was of interest. Derive the ...
12
votes
1answer
768 views

Modelling when the dependent variable has a “cut-off”

Apologies in advance if any of the terminology I use is incorrect. I'd welcome any correction. If what I describe as a "cut-off" goes by a different name, let me know and I can update the question. ...
32
votes
2answers
23k views

What is the difference between censoring and truncation?

In the book Statistical Models and Methods for Lifetime Data , it is written : Censoring: When an observation is incomplete due to some random cause. Truncation: When the incomplete nature of ...
15
votes
1answer
4k views

What is “Targeted Maximum Likelihood Expectation”?

I'm trying to understand some papers by Mark van der Laan. He's a theoretical statistician at Berkeley working on problems overlap significantly with machine learning. One problem for me (besides ...
7
votes
1answer
6k views

How should I handle a left censored predictor variable in multiple regression?

I have a dataset (N=350) for which I would like to regress a neuropsychological test score (continuous) on age, education, symptom severity (continuous), and diagnosis (binary). Symptom severity is ...
6
votes
1answer
14k views

Difference between independent and non-informative censoring

I was wondering if I could get a third opinion to settle a discussion on the distinction between independent and non-informative censoring. My definitions: 1) In independent censoring, the event ...
10
votes
3answers
24k views

Right censoring and Left censoring

Wikipedia gives the following definitions: Right censoring: a data point is above a certain value but it is unknown by how much. Left censoring: a data point is below a certain value but it is ...
5
votes
0answers
110 views

Censored logit transform for (ad hoc) exploratory data analysis

In my work I commonly have to analyze binary composition data, expressed as a fraction $f\in[0,1]$. The data $f[x]$ is spatially distributed ($x\in\mathbb{R}^n$, $n=1,2,3$), and typically comes in the ...
3
votes
1answer
169 views

Weighted normal errors regression with censoring

I have some data which I would model via standard multiple regression except: There is censoring (left-censored, fixed but varying censoring points which are known) The errors are assumed independent ...
2
votes
2answers
76 views

Calculate E[X] from incomplete data?

The exercise I'm doing describes the random variable $X$ as the following ...
52
votes
3answers
2k views

Do we have a problem of “pity upvotes”?

I know, this may sound like it is off-topic, but hear me out. At Stack Overflow and here we get votes on posts, this is all stored in a tabular form. E.g.: post id voter id vote type ...
9
votes
2answers
7k views

How should I model a continuous dependent variable in the $[0, \infty]$ range?

I have a dependent variable that can range from 0 to infinity, with 0s actually being correct observations. I understand censoring and Tobit models only apply when the actual value of $Y$ is partially ...
9
votes
3answers
373 views

If $Z_i =\min \{k_i, X_i\}$, $X_i \sim U[a_i, b_i]$, what is the distribution of $\sum_iZ_i$?

Assume the following set up: Let $Z_i = \min\{k_i, X_i\}, i=1,...,n$. Also $X_i \sim U[a_i, b_i], \; a_i, b_i >0$. Moreover $k_i = ca_i + (1-c)b_i,\;\; 0<c<1$ i.e. $k_i$ is a convex ...
7
votes
2answers
328 views

Informative censoring caused by cesarean section

I study labor and delivery as an epidemiologist. It is well established that a large fetus has a higher risk of causing maternal birth trauma. But a large baby is also likely to be delivered by ...
4
votes
1answer
3k views

Can a Cox proportional hazards model be used on general left-censored data (i.e. non-survival data)?

A familiar problem in applied science is having censored data, in particular left-censored data arising due to an assay or piece of equipment having a lower limit of detection (LOD). Assuming a ...
9
votes
1answer
320 views

What are the mean and variance of a 0-censored multivariate normal?

Let $Z \sim \mathcal N(\mu, \Sigma)$ be in $\mathbb R^d$. What are the mean and covariance matrix of $Z_+ = \max(0, Z)$ (with the max computed elementwise)? This comes up e.g. because, if we use the ...
8
votes
5answers
507 views

How can I estimate the time at which 50% of a binomial variable will have transitioned?

I have the following data, representing the binary state of four subjects at four times, note that it is only possible for each subject to transition $0\to 1$ but not $1\to 0$: ...
4
votes
1answer
454 views

Simple case of MNAR missing data

I believe I have a very simple problem with missing data, but I'm a bit lost because all the materials I read seem to be focused on much more complicated cases. I have a random variable $X$ which has ...
4
votes
0answers
323 views

Estimating right-censored data

I am VERY new to stats. I have a large amount of life-time data (delay in arrival since start of experiment) from repeat experiments. Some data is missing, but essentially represents a delay longer ...
3
votes
1answer
114 views

Deriving the distribution of the sum of censored variables

I want to be able to calculate the distribution of $$Y = \sum_{i=1}^n\max\{0,X_i\}$$ where the random variable $X_i\sim N(\mu_i,\sigma_i)$. Is the calculation of $f_Y(y)$ possible and if so what is ...
2
votes
1answer
198 views

Learning a continuous model from binned data

A very similar question has been asked before, but it didn't get a real answer. Background I would like to develop a probability model for a continuous, ratio-scale random variable $Y$. Let's say it ...
2
votes
2answers
3k views

How to get R squared/goodness of fit for Tobit model in R?

I do a Tobit regression to analyze censored data. To measure the goodness of fit the authors of these papers use Efron's $R^2$. So my idea is to use this one as well. To realize my Tobit regression, ...
2
votes
1answer
701 views

LASSO or other regularized regression with censored (missing) data

Here is my problem. I am looking at various time series curves. Let's call them total spend aggregated over all customers on various products versus time. At any given time, I want to predict the ...
2
votes
0answers
573 views

Statistics for rounded data

I wonder about the existence of "standard" statistical procedures for rounded (log)normal data. Indeed, in my work I often encounter rounded data which potentially cause some problems: "awful" qqplots,...
5
votes
1answer
3k views

Fit a Weibull distribution to…right-censored data?

Premise: I know next to nothing about survival analysis, I just started. I have a vector of failure times for some machines. Most of the machines (3362 vs 2694) are still running today, so I know they ...
5
votes
2answers
872 views

Inference with only left-censored data

Suppose I have a data set that is only left-censored data, ex: <5, <5, <5, <10, <10, <10 A technique to handle left-censored data is the Kaplan Meier estimate, see page 5 of this ...
4
votes
1answer
667 views

How to estimate parameters of a distribution with left-truncated and right-censored data?

I have been trying to find the best way of estimating parameters for a known pdf from a data-set that is left-truncated and right-censored. More precisely, I have lifetimes for a system where there ...
4
votes
2answers
162 views

Should one use a censored survival model when an event is only observed at death?

A colleague of mine is trying to estimate how neutron radiation exposure changes cancer incidence rates (in mice). He has autopsy data that reports whether a cancer was observed at the time of death, ...
3
votes
1answer
37 views

Poisson regression on devices that fail during observation period?

I am observing N devices over a period of time $[0,T]$ and counting the number of certain events $y_j$, for each device, during that period of time. I also have some specifications $x_j$ for each ...
3
votes
0answers
590 views

Right-censored independent variable in Cox/logistic regression

I have a right-censored continuous independent variable that I want to include in a Cox regression. The variable is a physiologic test which is capped at a certain time, say 120 seconds, due to safety ...
3
votes
2answers
61 views

Inference for a distribution with different number of samples due to censoring

I have the following problem in case someone has an idea about how to solve this. Assume three experiments that refer to the same population for a random variable $X$. In the first experiment, I ...
2
votes
1answer
1k views

Method for fitting censored data in R

I have 320 data points - each has a redshift and a turnover-frequency, and I want to fit a correlation between them (a linear fit). However, 120 of the turnover-frequency values are upper limits. As ...
2
votes
1answer
253 views

Version of Mann-Whitney U test, for right-censored data

Is there an equivalent of the Mann-Whitney U test, but for right-censored data? I have right-censored numerical data from two different groups. The mean for one group is higher than the other. I'd ...
2
votes
1answer
445 views

Poisson regression for ordered variables

I have three waves of data, and I am trying to estimate group-based trajectories of binge drinking across the three waves. The question asked (at all three waves was): “Over/During the past 12 months, ...
2
votes
1answer
2k views

Best way to handle na values in regression predictor variable

I am running a multinomial logistic regression model in R (using the multinom function from the nnet package) with a set of 12 ...
1
vote
1answer
281 views

Likelihood of censored data

Let $X_1,X_2,\ldots, X_{n_1}$ be IID with PDF $f(x-\theta) $, for $-\infty<x<\infty$ and $-\infty<\theta<\infty$. Denote the CDF of $X_i$ by $F(x-\theta)$. Let $Z_1,Z_2, \ldots, Z_{n_2}$ ...
1
vote
2answers
210 views

Time-to-event data with low censoring

I have data on individuals, consisting of an individual identifier column, starting date column and length date column. This indicates an individual being in financial distress for a specific period ...
1
vote
0answers
360 views

Is it possible to model BOTH censoring and truncation in BUGS?

Survival times are often right censored and left truncated. From my experience, it does not seem like OpenBUGS allows for both. Truncation is denoted as T( , ) and censoring as C( ,). For instance, a ...
0
votes
1answer
92 views

Handling survival data where people join at various stages

My question is about survival analysis. According to this censoring is a condition in which the value of a measurement or observation is only partially known. For example, suppose a study is ...