Skip to main content

Questions tagged [truncation]

Truncation is a process that results in the omission of data that are beyond a threshold.

Filter by
Sorted by
Tagged with
0 votes
0 answers
29 views

Which log-likelihood is to be maximized for left-truncated count data?

What is to be done, if the count data is missing the counts on the zeros (i.e. left truncated data)? Say one wants to estimate a Poisson regression and the goal is to derive the log-likelihood to be ...
Marlon Brando's user avatar
1 vote
1 answer
35 views

Rejection sampling to obtain a random sample from a truncated version of a multivariate probability density

Suppose I have a multivariate probability density $f(\mathbf{y}|\boldsymbol{\theta})$ with support $\mathbb{R}^d$ that is analytically tractable, and I know how to randomly sample from $f(\mathbf{y}|\...
Ron Snow's user avatar
  • 2,103
1 vote
1 answer
89 views

Difference between truncated and unseen data

I have 2 related questions. Assume that we want to build a model to study of some random discrete variable $x$ that follows some known distribution with PMF $P(x)$, yet with unknown parameters that we ...
Geo's user avatar
  • 47
4 votes
0 answers
80 views

Truncated response variable in boosted regression trees

I was thinking about the differences in approaches between parametric and non-parametric statistics in regression. I am working with a non-negative integer response $N\in\mathbb{N}_{0}$. Let's imagine ...
statsplease's user avatar
  • 2,842
3 votes
1 answer
41 views

How do right censoring and right truncation interact?

I am working with data related to time of event for some physical product. the data is current status data, also sometimes called interval censored type II data, which mean i only have one inspection ...
Nikolaj Pedersen's user avatar
2 votes
2 answers
224 views

Can I estimate the mean of a dataset if I have its standard deviation and a portion of the full data that is higher than some threshold?

I have a partial set of measurement data that is limited due to my tool's sensitivity. I know that the data is approximately normally distributed and I have a standard deviation from another data set ...
Shuesh's user avatar
  • 31
0 votes
1 answer
21 views

Left Truncated Data survival data

I am working with a data that is I believe left truncated. I am looking at how Veterans with MPNs have a different timing of arterial thrombosis (AT) compared with Civilians with MPNs. In the data ...
Dnz857's user avatar
  • 25
0 votes
1 answer
138 views

On the difference of truncated Gaussian and a new definition

Given a r.v. $X \sim N(0, 1)$, what is the density of $Z = X I(\lvert x \rvert < \lambda)$. I am confused with the truncated Gaussian $Y = X$ if $\lvert X \rvert < \lambda$ otherwise $Y = 0$. My ...
Chia's user avatar
  • 43
0 votes
1 answer
55 views

Estimating a censored distribution curve

I have a sample of only 142 numbers from a distribution of 3852 numbers ranging from 0 to 53, but it is censored below 35 (The values ​​exist, but I don't have access.), so I have only the values in ...
Silvio Duarte's user avatar
4 votes
0 answers
116 views

Distribution closed under convolution and truncation followed by convolution

Let $D(\theta)$ denote an absolutely continuous distribution on $\mathbb{R}$. (The finite dimensional vector $\theta$ collects the parameters of the distribution.) Assume that the p.d.f. of $D(\theta)$...
cfp's user avatar
  • 535
1 vote
1 answer
418 views

Left truncation or left censoring?

I am conducting a study in which I analyse the time to opt out after the free trial is ending. I have individuals who sign up for a free membership-trial period. At some point the period is ending and ...
JohnDK's user avatar
  • 11
1 vote
0 answers
56 views

Likelihood contribution of censored data - why conditional?

Let $x_i$ be a sample of survival time from right-truncated data with right-truncation time $Y_R$. I would like to find the likelihood contribution from the sample. I thought I have to find the ...
Kaira's user avatar
  • 299
1 vote
1 answer
83 views

Parameter estimation on exponential distribution from a bounded subset of that distribution

I have a random variable that is exponentially distributed with some $\lambda$. I'm sampling observations from this variable, but I'm limited to observing only those that are less than some maximum ...
Eric Pauley's user avatar
2 votes
0 answers
85 views

Distribution of a constrained Gaussian distribution in frequency domain

We know that a Gaussian Distribution is not limited and it spans from $-\infty$ to $+\infty$ . However, practically if we sample the Gaussian with a finite sampling frequency, the maximum frequency is ...
CfourPiO's user avatar
  • 235
1 vote
0 answers
51 views

Is my data right censored or right truncated?

I have a dataset of the percent of pavement area that is cracked and time as the explanatory variable for a city's streets. Engineering principles and logic tell us that as that as time progresses, ...
Sampat Kedarisetty's user avatar
0 votes
0 answers
16 views

How to model an uncertain (or erroneous) truncation effect on a random variable?

Let $ V_a $ be a random variable which is truncated at a value $v_c$. Therefore, the updated density function of truncated $V_a$ is given by, $$ f(v_a| V_a\leq v_c) = \frac{g(v_a)}{F(v_c)}$$ where $g(...
Shihab Khan's user avatar
2 votes
1 answer
156 views

Survival analysis when event cannot happen after timepoint

People are contacted for a survey for a fixed number of days. For simplicity, let's say this happens over 5 days. People can complete their survey on day 1, day 2, ..., up to the end of day 5. After ...
Eli's user avatar
  • 2,672
2 votes
1 answer
746 views

Prediction metrics for left-truncated and right-censored data

I am trying to assess (out-of-sample) predictive performance of survival analysis models with left-truncated and right-censored data. Assume the training and test data, respectively, consist of ...
njirons's user avatar
  • 61
2 votes
0 answers
359 views

Formula for standard deviation of a normal distribution from truncated data [closed]

I have some samples taken from a normal distribution of unknown $\mu$ and $\sigma$, and I know someone took away the top and bottom $p$ percent of the original samples ($p$ is known). Is there a ...
relatively_random's user avatar
1 vote
1 answer
2k views

How to deal with left truncation in cox model in R

I have a longitudinal data with 8 follow-ups. My aim is to see the effect of a disease on ability decline at old age. In my study, I have sibling pairs, one with disease (1) and one without disease (0)...
user358238's user avatar
0 votes
0 answers
41 views

Expected value of a random variable with truncation

Let $f:[0,\infty)\to \mathbb R_+$ denote the PDF of a random variable $X$ and $c>0$ a constant. I want to evaluate the following integral: $$I(c)=\int_0^\infty{\min(x,c)f(x)dx}.$$ This can be ...
clueless's user avatar
  • 113
3 votes
1 answer
1k views

Distribution of a sample of uniformly distributed points in the 2D

Let there be a rectangle in the plane and a set of points distributed in the rectangle by a uniform distribution. I select a random point on the top and right border and draw the red line. The blue ...
M a m a D's user avatar
  • 157
3 votes
2 answers
285 views

Simulation of a truncated normal distribution over two intervals

Given $X$ a random variable with a normal distribution, what is the best procedure to simulate $X|X\in[a;b]\cup[c;d]$, i.e. we want to simulate the truncated normal distribution only on the intervals $...
KACEFMA.'s user avatar
  • 131
2 votes
1 answer
753 views

Better understanding of censoring and truncation (in particular right truncation) in survival analysis

I am taking a course in survival analysis where we follow the book "Survival Analysis: Techniques for Censored and Truncated Data" by John P. Klein and Melvin L. Moeschberger. Although I ...
Jacobiman's user avatar
  • 147
2 votes
2 answers
159 views

Is it possible to calculate or find out what the original distribution was of a dataset? [duplicate]

Is it possible to calculate or find out what the original distribution was of a dataset? For example: I have (part of) a dataset with 800 weights and I know that the original dataset contained 1000 ...
user avatar
2 votes
1 answer
324 views

What is the correlation of a truncated bivariate Normal distribution?

$X$ and $Y$ are independent standard normal variables. If we generate $n$ samples $(x,y),$ what is the correlation for samples $(x, y)$ with $x+y\gt0$?
qwerty1010's user avatar
2 votes
1 answer
142 views

Is trial with accrual period left truncated?

I am reading Dirk Moore's Applied Survival Analysis 11.2. "...However, patients actually enter over an accrual period of length a, and then, after accrual to the trial has ended, they are ...
user45765's user avatar
  • 1,445
1 vote
1 answer
2k views

Censoring, truncated and missing data in Survival Analysis

Can someone please explain the difference between censored, truncated and missing data in survival analysis? Suppose I have following information. ...
Dushi Fdz's user avatar
  • 145
1 vote
1 answer
34 views

Baseline adjustment when both pre- and post values are subject to left-censoring

Assume $X$ is a binary treatment variable, $Y$ is a continuous variable measured pre- and post-treatment $(Y_{pre}$, $Y_{post})$, and $Z$ represents the remaining covariates. Both $Y_{pre}$ and $Y_{...
andrjens's user avatar
4 votes
1 answer
2k views

Dealing with left-, right- and interval-censoring and left- and right-truncation

I have a survival data set with left-, right- and interval-censoring and left- and right-truncation. Now I want to fit a Cox proportional hazard and an AFT model to these data. What is the best way to ...
Tino's user avatar
  • 87
1 vote
1 answer
108 views

How to use logistic regression with original values from a dependent variable that is a ratio?

I have a dependent variable that is a ratio, i.e. it takes the values between 0 and 1. Some 30% of values are 1s. The dependent variable measures the distribution of funds, i.e. it is calculated just ...
Ken Lee's user avatar
  • 351
3 votes
1 answer
897 views

Why is it problematic to use OLS for a bounded dependent variable (e.g. a ratio variable)?

I have a question that emerged from my previous post. If you look at my previous post, the dependent variable there is a ratio variable and thus is bounded, i.e. it can only take the values between 0 ...
Ken Lee's user avatar
  • 351
6 votes
1 answer
267 views

Flawed multiple linear regression in academia? Heteroscedasticity's effect on p-value?

I believe I have found a paper in academia that has used a flawed multiple linear regression. I have downloaded the data set and replicated their regression results. I have done some diagnostics and ...
Ken Lee's user avatar
  • 351
4 votes
0 answers
118 views

truncation of bivariate normal under quadratic condition

Consider a complex normal variable $Z \sim \mathcal{CN}(\mu,2\sigma^2)$ with real component $X \sim \mathcal{N}(\mu,\sigma^2)$ and imaginary component $Y \sim \mathcal{N}(0,\sigma^2)$. We can write ...
Ahwaq's user avatar
  • 121
4 votes
0 answers
95 views

Bivariate normal and truncated expectation [duplicate]

What is the expectation $$\mathbb E[X_1 \lvert X_1 > X_2]$$ assuming that $$(X_1,X_2) \sim \mathcal MVN(0,\Sigma),$$ with $\mathcal{MVN}$ being the multivarite normal. I would expect this to have ...
Jesper for President's user avatar
3 votes
1 answer
437 views

Central Limit Theorem for Truncated observations

Consider a random variable $X$ with values in $\left[0,\infty\right)$ such that $E\left[X\right]=\infty$. Given $M > 0$ I want to estimate the expected value of $X$ truncated at $M$. That is I want ...
steveo'america's user avatar
0 votes
1 answer
136 views

Truncating variable in survival analysis

this is my first time developing a survival analysis model so bare with me if the nomenclature is not on point. Basically, I'm running a Cox PH model for the length a contract is active, where 1 is ...
amestrian's user avatar
  • 265
0 votes
1 answer
41 views

Partially left truncated data in bivariate time series, one series is missing data what to do?

I have two time series of futures Settlement Prices where I am to model the systematic volatility of the settlement prices for these two time series. My plan is to: Calculate the logged returns. Fit ...
Jens Kramer's user avatar
0 votes
0 answers
23 views

Truncation based on two variables

I have some data, which has been constructed from multiple API pulls. Observations have an expiry date, $x_i$, which means that observations which expire before the API pull is done do not appear in ...
Oska Fentem's user avatar
2 votes
0 answers
57 views

Is it Sufficient to Truncate a Left Censored Distribution?

A colleague explained their approach to dealing with left censored data in an analysis, and while I don't think it is the best approach, I am not sure if it is insufficient or not. My colleague has ...
Dave Bapst's user avatar
5 votes
1 answer
3k views

Understanding left-truncated data in survival analysis

I am having a hard time grasping the concept of left truncation. According to what I understand, left truncation occurs when we observe individuals only if their event of interest takes place after ...
Bergson's user avatar
  • 69
2 votes
1 answer
43 views

How do we obtain the probability density of a truncated regression with an upper and lower bound

I know my density for $y$ is supposed to be something of this form $$g(y|x_{i},t)=\frac{f(y|x'\beta, \sigma^{2})}{F(t|x' \beta' \sigma^{2}}$$ where the numerator is the density of the normal ...
Alex's user avatar
  • 31
1 vote
1 answer
229 views

How to handle truncated or missing ranking data in a classification problem?

I'm preparing data for a classification problem that involves matches in a single-player sport. In each match, each competitor is either ranked and thus has a numeric rank; or unranked (rare but can ...
yurnero's user avatar
  • 205
4 votes
2 answers
841 views

Moments of truncated Student's $t$-distribution

I performed random sampling on a Student's $t$-distribution. I used SciPy to calibrate my parameters and then truncated my allowable values to the maximum and minimum observation in the data for ...
Mild_Thornberry's user avatar
0 votes
0 answers
168 views

Truncation versus censoring with Kaplan-Meier

I am trying to run Kaplan-Meier on a rather odd dataset and am having difficulty determining whether I should be truncating or censoring my data. I have looked at the other feeds, including this very ...
Janth's user avatar
  • 1
4 votes
1 answer
688 views

Fitting a gamma distribution to truncated data

I am faced with the following truncation problem: $$ X_i \sim \Gamma(\alpha, \beta) \\ \delta_i = \chi(X_i \le \tau_i) $$ I can observe only $(X_i, \tau_i)$ where $\delta_i = 1$ and I have no a-...
AlexR's user avatar
  • 799
2 votes
1 answer
328 views

What is a sensible way to truncate data to a region that fits a model?

I want to use an exponential decay model in python to relate the flow rate in a device to the mass left inside it, in particular $flow=a−b×e^{−c×mass}$ where a, b and c are the parameters of the model....
st210's user avatar
  • 33
4 votes
2 answers
457 views

Distributions with simple truncated expectations

For a project I'm looking for continuous distributions which have a somewhat simple closed form for upper-truncation expectation ($E[x|x>c]$). Here are two examples I've found so far: Exponential ...
0 votes
0 answers
131 views

Truncation of distance matrix

Given a sample $\textbf{X}_1$,...,$\textbf{X}_n\in\mathbb{R}^p$ from an arbitrary distribution with distribution function $F$ we can calculate the pairwise Mahalanobis distances between the sample ...
ricsirke's user avatar
2 votes
1 answer
192 views

How do to deal with data missing NOT at random?

It seems that because values are missing from a specific range of my target variable, my model performs poorly when predicting samples that are actually in that range. My target variable is ...
George's user avatar
  • 155