Questions tagged [censoring]

The process of censoring yields data w/ only partial information. The most common example of censoring is *right censoring* in survival analysis, where the time until the event occurred is only known to be longer than some duration because the event had not occurred when the study ended.

191 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7 votes
0 answers
113 views

COVID-19: Do epidemiologists use censoring methods to calculate case fatality rates with undercount?

In case of the coronavirus COVID-19 (or 2019-nCoV), it seems that there are a lot of mild cases which do not require medical intervention, see here: For every person who is sick enough to come to ...
user avatar
5 votes
0 answers
129 views

Censored logit transform for (ad hoc) exploratory data analysis

In my work I commonly have to analyze binary composition data, expressed as a fraction $f\in[0,1]$. The data $f[x]$ is spatially distributed ($x\in\mathbb{R}^n$, $n=1,2,3$), and typically comes in the ...
user avatar
  • 12.1k
5 votes
0 answers
48 views

Inferences about a distribution given running maximum values

Here is a question inspired by this question from StackOverflow. Suppose you have observations of a variable which is measured once a minute, but the values are only recorded if they are greater than ...
user avatar
5 votes
0 answers
346 views

Estimating right-censored data

I am VERY new to stats. I have a large amount of life-time data (delay in arrival since start of experiment) from repeat experiments. Some data is missing, but essentially represents a delay longer ...
user avatar
  • 51
4 votes
1 answer
95 views

How to fit a distribution with an "10 and more" category at the bottom?

I want to fit a distribution to some data to sample from it in a subsequent simulation. There are I got a dataset that looks somehwat like this: ...
user avatar
4 votes
0 answers
65 views

Censored regression but dependent variable is sum of two censored variables

Suppose we have two censored variables: $$y_1 = \begin{cases} 0, & y_1^*\leq 0\\ y_1^*, & 0< y_1^* < 1000 \\ 1000, & 1000<y_1^* \end{cases}. $$ $$y_2 = \...
user avatar
  • 2,149
4 votes
0 answers
252 views

Sign and size of OLS bias for Tobit models

I have a question related to the sign and size of the OLS bias in the case of a Tobit model. Consider the following model (1) Sample of observations $\{X_i,Y_i\}_{i=1}^n$, i.i.d., $X_i$ is a vector ...
user avatar
  • 266
4 votes
0 answers
115 views

Appropriate probability distribution for censored ratio data

I have two bacterial markers, I'll just call them X and Y. X codes for virulence, whereas Y just indicates the bacteria is present. Consequently, X will not show up without Y although not all bacteria ...
user avatar
  • 1,126
4 votes
0 answers
443 views

Factor analysis with severly skewed ordinal data and censored ordinal data

I am aiming to run initial exploratory factor analysis in one sample and then confirmatory in another sample. My indicators are ordinal and so I planned to generate a polychoric correlation matrix and ...
user avatar
4 votes
0 answers
84 views

literature on small samples and parametric survival models

I have an abundance of small data sets with right-censored data. There are different groups in each data set and I'd like to get confidence intervals for the regression parameters. Each data set has 3-...
user avatar
  • 5,860
4 votes
0 answers
2k views

Finite mixture models with bounded data

I am trying to fit a finite mixture model to a dependent variable which is bounded (practically) between -0.594 and 1 (theoretically, the latent variable is bounded between -Inf - 1). The data are ...
user avatar
  • 123
4 votes
0 answers
52 views

What could be the described method for treating time-outs in algorithm benchmarks

As a computer scientist, you often face the problem to empirically analyse the improvements in run time of some algorithm. I stumbled over the following text in a paper: These run times are averages ...
user avatar
  • 1,504
3 votes
1 answer
51 views

Left censored index date during survival analysis

I am trying to do a Time To Event analysis, looking at patients with Multiple Sclerosis, which can lead to wheelchair use. My intended study is to look at the time from MS diagnosis to first ...
user avatar
  • 767
3 votes
0 answers
96 views

Multiple regression for left-censored independent and dependent variables

I am interested in developing a predictive multiple regression model which predicts a concentration of one compound based on the measured concentrations of several other compounds. Both the dependent ...
user avatar
3 votes
0 answers
32 views

variance estimator for a symmetrical two-sides censored normal distribution

Suppose to draw a sample of $n$ observations from $X \sim \mathcal{N}(0,\sigma)$, with observations outside the interval $(-c,+c)$ censored; $c$ is known and one can conveniently set $c=1$, for ...
user avatar
  • 1,020
3 votes
0 answers
84 views

Which duration model to use with a fully right censored database

Currently I'm examining the duration of residence of households. I have a database at my disposal that indicates how long a specific household resides in its current home. I want to explain their ...
user avatar
  • 31
3 votes
0 answers
46 views

How can we deal with limit of detection independent variables in regression

Suppose that you have a regression in which one or more of the independent variables has a lower limit of detection (e.g. some result of blood tests). How best to deal with this? Any pointers to the ...
user avatar
  • 94.5k
3 votes
0 answers
143 views

Likelihood with random censoring

Suppose to observe a random sample from a r.v. $Y_i=\min(T_i,C_i)$ where $T$ and $C$ are iid absolutely continuous distribution. I would like to inference about a parameter of $T$ (for example, $\...
user avatar
  • 125
3 votes
0 answers
243 views

Variable selection with right-censored data?

I have the typical linear regression model: $$y_i = x_i^T\beta + e_i,$$ where $e_i\sim N(0,\sigma^2)$, iid. However, in my case, some (not all of them, only around 1/3 of them) response variables $...
user avatar
  • 31
3 votes
0 answers
164 views

Likelihood function for a Weibull model

I have an assignment for my course Microeconometrics. Currently I am stuck on the following question You are asked to help out with the statistics of a medical study. We are interested in the time ...
user avatar
3 votes
0 answers
772 views

Right-censored independent variable in Cox/logistic regression

I have a right-censored continuous independent variable that I want to include in a Cox regression. The variable is a physiologic test which is capped at a certain time, say 120 seconds, due to safety ...
user avatar
3 votes
0 answers
664 views

Handling Informative Censoring in Survival Analysis

In a survival study with informative censoring (for example, studying the effects of cigarettes on mortality and smokers are more likely to be Lost to Follow Up). This causes the censored data to be ...
user avatar
  • 196
3 votes
0 answers
948 views

Censored logistic regression

I have the following problem: We have data with a 0-1 outcome which can occur precisely once. It can occur at any time within a certain time period (say 3 years). For this data set, for some ...
user avatar
3 votes
0 answers
30 views

estimate the pdf from mutually censored (competing) observations

Imagine we have three independent random variables, distributed according to pdfs $a(t), b(t), c(t)$, with cdfs $A(t), B(t), C(t)$. In my case the distributions are lognormal. However, our ...
user avatar
  • 1,272
3 votes
0 answers
142 views

R: Technique for lasso with left-censored data?

I am searching for an R function that uses lasso for variable selection that includes a method to account for left-censored data. Is there an R function available for this? Thanks for any input.
user avatar
  • 31
3 votes
0 answers
178 views

Evaluating survival models in the presence of covariate-dependent censoring

I have a censored survival analysis problem with the following characteristics: Failure times are discretized The censorship distribution depends on certain covariates I don't have a properly-...
user avatar
  • 5,423
3 votes
1 answer
277 views

Weighted normal errors regression with censoring

I have some data which I would model via standard multiple regression except: There is censoring (left-censored, fixed but varying censoring points which are known) The errors are assumed independent ...
user avatar
  • 83
3 votes
0 answers
519 views

Estimate of censored poisson process

I have a set of processes, each of which has number of events and the total length of time. I'm trying to model them as independent Poisson processes with there own rates. The rate of the ith process ...
user avatar
  • 444
3 votes
0 answers
74 views

Comparison of cell counts with a right censoring

I have cell counts related to the action of different microorganisms and I want to compare their distribution. It's supposed they follow a normal distribution after a log transformation, but I can't ...
user avatar
3 votes
0 answers
279 views

Accelerated Failure Time scale parameter excessively large when data is right censored

I am using an accelerated failure time model with the Weibull distribution to predict failure times. My failure times range from 1 - 365, with many (80%) data points that are right censored (no ...
user avatar
  • 1,392
3 votes
0 answers
765 views

Broken Tobit regressions

I have a dataset with 43,422 observations and a left-censored (at 0) dependent variable. Of the $n$ observations, 42,536 are left-censored and 886 are not. I plan on analyzing this with a Tobit ...
user avatar
3 votes
0 answers
79 views

Is a snapshot of last login times a type of censored data?

I have a database of users with a field of the last time they logged in. Would this kind of data be considered "censored", and if so is there anything I can do with it beyond a histogram/density of ...
user avatar
  • 20.1k
2 votes
0 answers
68 views

OLS regression when dependent variable's measurement is capped

Let's say you have $y=a+xb$ but your $y$ variable is measured imperfectly such that the measurement never exceeds 99 but in reality it should. We know it's a measurement issue. Is the best technique ...
user avatar
2 votes
0 answers
40 views

Is it Sufficient to Truncate a Left Censored Distribution?

A colleague explained their approach to dealing with left censored data in an analysis, and while I don't think it is the best approach, I am not sure if it is insufficient or not. My colleague has ...
user avatar
2 votes
0 answers
28 views

Bound on Error from Censoring

Suppose I have $N$ Gaussian random variables $X_1,...,X_N$ where $X_i \sim \mathcal{N}(0,\sigma)$. Suppose I censor each $X_i$ to lie in the range $[-t,t]$. Denote the censored values by $Z_1,...,Z_N$....
user avatar
  • 33
2 votes
0 answers
39 views

Parametric estimator for straightforward interval-censored data

$X_i$ is iid from some distribution, such as $N(\mu, \sigma^2)$. All I want is to estimate the parameters of the distribution. However, I don't observe $x_i$, instead, I observe $(a_i, b_i)$ such that ...
user avatar
  • 1,051
2 votes
0 answers
94 views

Distribution of shifted and censored random variable

Suppose I have a non-negative random variable $X$ with finite mean $\mu$. Let's for example assume it denotes demand, and we denote by $X(t)$ demand in period $t$, demand in different periods are iid. ...
user avatar
  • 149
2 votes
0 answers
426 views

Censored Regression/Tobit: coefficients decrease drastically when non-normal distribution is assumed

I am trying to use a Tobit/ censored regression model to estimate the effect of a certain political condition on a tax. Our dependent variable is zero-inflated because in most of our observations this ...
user avatar
2 votes
0 answers
48 views

Piecewise integration

I am trying to estimate residential demand for electricity in a country where electricity is sold (to all households (HH)) at an increasing two-part tariff. By choosing marginal prices as my key ...
user avatar
2 votes
0 answers
81 views

Appropriate loss function for unobservable failure time

I'm trying to construct a forecast for the failure time of a system. I have a model for this failure time built from physical understanding of the system. I'd like to be able to quantify the accuracy ...
user avatar
2 votes
0 answers
43 views

MCMC for model with cumulative hazard involving integrals?

Let $h(t\vert \theta, x)$ be a hazard function with parameter vector $\theta$ and covariate vector $x$, and let $H(t\vert \theta, x) = \int_0^t h(s\vert \theta, x)ds$ be the corresponding cumulative ...
user avatar
2 votes
0 answers
106 views

Right-censored cash flows and survival analysis

I am doing survival analysis on a disability claims dataset - each row tells me how long a person has been on claim and whether they have gone off claim yet. Ultimately I am interested in predicting ...
user avatar
  • 9,933
2 votes
0 answers
130 views

Dealing with censored count data from a limited pool of potential customers

I have daily data of reservations for a restaurant with a certain capacity. I also know about the population size that will possibly visit this restaurant. I would like to do some analysis to get more ...
user avatar
  • 31
2 votes
0 answers
268 views

Manually right-censoring (top-coding) data for survival modeling: justified?

Say we have survival-time data that has a long right tail. Does it make analytical/statistical sense to set a time limit and then manually right-censor (top-code) all times-to-event that exceed this ...
user avatar
  • 20.1k
2 votes
0 answers
304 views

Probability model for duration, how to write log-likelihood function for Weibull distribution to fit aggreate data

Background: I have a basic to moderate knowledge on probability and statistics. I have familiarity with R programming language and optimization routines. I'm reading an excellent article "Jobs, ...
user avatar
  • 7,399
2 votes
0 answers
70 views

Censoring? Am I training my model on an event that might not have happened yet?

I'm working on a paper at the moment, and I have gone into one of those self-doubt loops/spirals. My supervisors assure me that this is the right way, but i'm sceptical. I am identifying doctors at ...
user avatar
  • 21
2 votes
0 answers
80 views

Appropriate Model for inflated variable at non zero value

I have a question regarding the appropriate regression model for a continuous variable that ranges from zero to 10. This variable has a large concentration of observations in the "10" value (almost 50 ...
user avatar
2 votes
0 answers
352 views

Partly truncated and not truncated data

In some places 100 km races stop to count time if a certain time limit has been passed, for example 15 hours. It is difficult, respective, not possible to distinguish which finisher or site was ...
user avatar
  • 879
2 votes
0 answers
900 views

Variance-Covariance matrix of Weibull Distribution for right-censored data

The probability distribution function, cumulative distribution function and survival function of Weibull distribution are given by respectively, \begin{equation} f(t;\alpha, \beta)= \dfrac{\beta}{\...
user avatar
  • 630
2 votes
0 answers
54 views

how to quantify variation in continuous outcome for samples matched on continuous predictor

I have a data consisting of about 2 million cases. Cases are scored on some (overdispersed) predictor we'll call "employee performance." I would like to match or pool cases with the same score or ...
user avatar
  • 359