Questions tagged [truncation]
Truncation is a process that results in the omission of data that are beyond a threshold.
155
questions
2
votes
0
answers
30
views
Distribution of a constrained Gaussian distribution in frequency domain
We know that a Gaussian Distribution is not limited and it spans from $-\infty$ to $+\infty$ . However, practically if we sample the Gaussian with a finite sampling frequency, the maximum frequency is ...
- 99
1
vote
0
answers
30
views
Is my data right censored or right truncated?
I have a dataset of the percent of pavement area that is cracked and time as the explanatory variable for a city's streets. Engineering principles and logic tell us that as that as time progresses, ...
0
votes
0
answers
11
views
How to model an uncertain (or erroneous) truncation effect on a random variable?
Let $ V_a $ be a random variable which is truncated at a value $v_c$. Therefore, the updated density function of truncated $V_a$ is given by,
$$ f(v_a| V_a\leq v_c) = \frac{g(v_a)}{F(v_c)}$$
where $g(...
- 173
2
votes
1
answer
26
views
Survival analysis when event cannot happen after timepoint
People are contacted for a survey for a fixed number of days. For simplicity, let's say this happens over 5 days. People can complete their survey on day 1, day 2, ..., up to the end of day 5. After ...
- 2,284
2
votes
1
answer
144
views
Prediction metrics for left-truncated and right-censored data
I am trying to assess (out-of-sample) predictive performance of survival analysis models with left-truncated and right-censored data. Assume the training and test data, respectively, consist of ...
- 21
2
votes
0
answers
41
views
Formula for standard deviation of a normal distribution from truncated data [closed]
I have some samples taken from a normal distribution of unknown $\mu$ and $\sigma$, and I know someone took away the top and bottom $p$ percent of the original samples ($p$ is known).
Is there a ...
0
votes
0
answers
12
views
Discrete choice models: truncated or censored model
I am reviewing some practice.
Suppose we have a random sample of households reporting their share of financial wealth invested in stocks (alpha). The minimum alpha in the sample is 0.15, and the ...
- 1
1
vote
1
answer
158
views
How to deal with left truncation in cox model in R
I have a longitudinal data with 8 follow-ups. My aim is to see the effect of a disease on ability decline at old age. In my study, I have sibling pairs, one with disease (1) and one without disease (0)...
- 61
0
votes
0
answers
36
views
Expected value of a random variable with truncation
Let $f:[0,\infty)\to \mathbb R_+$ denote the PDF of a random variable $X$ and $c>0$ a constant. I want to evaluate the following integral:
$$I(c)=\int_0^\infty{\min(x,c)f(x)dx}.$$
This can be ...
- 113
3
votes
1
answer
253
views
Distribution of a sample of uniformly distributed points in the 2D
Let there be a rectangle in the plane and a set of points distributed in the rectangle by a uniform distribution. I select a random point on the top and right border and draw the red line. The blue ...
- 155
0
votes
0
answers
10
views
DAGs and missingness/truncation
Being sort of new to the DAG way of thinking, I have a hard time wrapping my head around this question: What's the best way to represent the following problem as a DAG?
Consider a simple regression ...
- 1
0
votes
0
answers
10
views
Converting untruncated Poisson regression effects to truncated effects - theory behind formulae
I am running a zero-truncated Poisson regression in R and the output provides untruncated mean lambdas for effects (i.e. includes values < 1). I have been advised to consider applying one of the ...
0
votes
1
answer
32
views
Simulation of a truncated normal distribution over two intervals
Given $X$ a random variable with a normal distribution, what is the best procedure to simulate $X|X\in[a;b]\cup[c;d]$, i.e. we want to simulate the truncated normal distribution only on the intervals $...
- 101
2
votes
1
answer
110
views
Better understanding of censoring and truncation (in particular right truncation) in survival analysis
I am taking a course in survival analysis where we follow the book "Survival Analysis: Techniques for Censored and Truncated Data" by John P. Klein and Melvin L. Moeschberger.
Although I ...
- 147
2
votes
2
answers
113
views
Is it possible to calculate or find out what the original distribution was of a dataset? [duplicate]
Is it possible to calculate or find out what the original distribution was of a dataset?
For example: I have (part of) a dataset with 800 weights and I know that the original dataset contained 1000 ...
Ken
2
votes
1
answer
136
views
What is the correlation of a truncated bivariate Normal distribution?
$X$ and $Y$ are independent standard normal variables. If we generate $n$ samples $(x,y),$ what is the correlation for samples $(x, y)$ with $x+y\gt0$?
- 21
2
votes
1
answer
51
views
Is trial with accrual period left truncated?
I am reading Dirk Moore's Applied Survival Analysis 11.2.
"...However, patients actually enter over an accrual period of length a, and then, after accrual to the trial has ended, they are ...
- 1,169
1
vote
1
answer
417
views
Censoring, truncated and missing data in Survival Analysis
Can someone please explain the difference between censored, truncated and missing data in survival analysis? Suppose I have following information.
...
- 85
1
vote
1
answer
26
views
Baseline adjustment when both pre- and post values are subject to left-censoring
Assume $X$ is a binary treatment variable, $Y$ is a continuous variable measured pre- and post-treatment $(Y_{pre}$, $Y_{post})$, and $Z$ represents the remaining covariates. Both $Y_{pre}$ and $Y_{...
- 11
2
votes
1
answer
698
views
Dealing with left-, right- and interval-censoring and left- and right-truncation
I have a survival data set with left-, right- and interval-censoring and left- and right-truncation. Now I want to fit a Cox proportional hazard and an AFT model to these data. What is the best way to ...
- 67
1
vote
1
answer
64
views
How to use logistic regression with original values from a dependent variable that is a ratio?
I have a dependent variable that is a ratio, i.e. it takes the values between 0 and 1. Some 30% of values are 1s. The dependent variable measures the distribution of funds, i.e. it is calculated just ...
- 341
3
votes
1
answer
251
views
Why is it problematic to use OLS for a bounded dependent variable (e.g. a ratio variable)?
I have a question that emerged from my previous post. If you look at my previous post, the dependent variable there is a ratio variable and thus is bounded, i.e. it can only take the values between 0 ...
- 341
5
votes
1
answer
195
views
Flawed multiple linear regression in academia? Heteroscedasticity's effect on p-value?
I believe I have found a paper in academia that has used a flawed multiple linear regression. I have downloaded the data set and replicated their regression results. I have done some diagnostics and ...
- 341
4
votes
0
answers
107
views
truncation of bivariate normal under quadratic condition
Consider a complex normal variable $Z \sim \mathcal{CN}(\mu,2\sigma^2)$ with real component $X \sim \mathcal{N}(\mu,\sigma^2)$ and imaginary component $Y \sim \mathcal{N}(0,\sigma^2)$. We can write ...
- 111
4
votes
0
answers
87
views
Bivariate normal and truncated expectation [duplicate]
What is the expectation
$$\mathbb E[X_1 \lvert X_1 > X_2]$$
assuming that
$$(X_1,X_2) \sim \mathcal MVN(0,\Sigma),$$
with $\mathcal{MVN}$ being the multivarite normal.
I would expect this to have ...
- 5,099
3
votes
1
answer
198
views
Central Limit Theorem for Truncated observations
Consider a random variable $X$ with values in $\left[0,\infty\right)$ such that $E\left[X\right]=\infty$. Given $M > 0$ I want to estimate the expected value of $X$ truncated at $M$. That is I want ...
- 513
0
votes
1
answer
66
views
Truncating variable in survival analysis
this is my first time developing a survival analysis model so bare with me if the nomenclature is not on point.
Basically, I'm running a Cox PH model for the length a contract is active, where 1 is ...
- 233
0
votes
1
answer
24
views
Partially left truncated data in bivariate time series, one series is missing data what to do?
I have two time series of futures Settlement Prices where I am to model the systematic volatility of the settlement prices for these two time series.
My plan is to:
Calculate the logged returns.
Fit ...
- 15
0
votes
0
answers
17
views
Truncation based on two variables
I have some data, which has been constructed from multiple API pulls. Observations have an expiry date, $x_i$, which means that observations which expire before the API pull is done do not appear in ...
- 11
2
votes
0
answers
41
views
Is it Sufficient to Truncate a Left Censored Distribution?
A colleague explained their approach to dealing with left censored data in an analysis, and while I don't think it is the best approach, I am not sure if it is insufficient or not.
My colleague has ...
- 21
4
votes
1
answer
666
views
Understanding left-truncated data in survival analysis
I am having a hard time grasping the concept of left truncation.
According to what I understand, left truncation occurs when we observe individuals only if their event of interest takes place after ...
- 157
2
votes
1
answer
35
views
How do we obtain the probability density of a truncated regression with an upper and lower bound
I know my density for $y$ is supposed to be something of this form $$g(y|x_{i},t)=\frac{f(y|x'\beta, \sigma^{2})}{F(t|x' \beta' \sigma^{2}}$$ where the numerator is the density of the normal ...
- 21
0
votes
1
answer
106
views
How to handle truncated or missing ranking data in a classification problem?
I'm preparing data for a classification problem that involves matches in a single-player sport. In each match, each competitor is either ranked and thus has a numeric rank; or unranked (rare but can ...
- 195
3
votes
2
answers
404
views
Moments of truncated Student's $t$-distribution
I performed random sampling on a Student's $t$-distribution. I used SciPy to calibrate my parameters and then truncated my allowable values to the maximum and minimum observation in the data for ...
- 161
0
votes
0
answers
119
views
Truncation versus censoring with Kaplan-Meier
I am trying to run Kaplan-Meier on a rather odd dataset and am having difficulty determining whether I should be truncating or censoring my data. I have looked at the other feeds, including this very ...
- 1
4
votes
1
answer
434
views
Fitting a gamma distribution to truncated data
I am faced with the following truncation problem:
$$
X_i \sim \Gamma(\alpha, \beta) \\
\delta_i = \chi(X_i \le \tau_i)
$$
I can observe only $(X_i, \tau_i)$ where $\delta_i = 1$ and I have no a-...
- 759
2
votes
1
answer
106
views
What is a sensible way to truncate data to a region that fits a model?
I want to use an exponential decay model in python to relate the flow rate in a device to the mass left inside it, in particular $flow=a−b×e^{−c×mass}$ where a, b and c are the parameters of the model....
- 33
4
votes
2
answers
328
views
Distributions with simple truncated expectations
For a project I'm looking for continuous distributions which have a somewhat simple closed form for upper-truncation expectation ($E[x|x>c]$).
Here are two examples I've found so far:
Exponential ...
0
votes
0
answers
84
views
Truncation of distance matrix
Given a sample $\textbf{X}_1$,...,$\textbf{X}_n\in\mathbb{R}^p$ from an arbitrary distribution with distribution function $F$ we can calculate the pairwise Mahalanobis distances between the sample ...
2
votes
1
answer
133
views
How do to deal with data missing NOT at random?
It seems that because values are missing from a specific range of my target variable, my model performs poorly when predicting samples that are actually in that range. My target variable is ...
- 135
1
vote
0
answers
95
views
Estimate population variance / mean from truncated distribution with known cutoff, but without parametric assumptions
Suppose you have a sample of $N$ iid random variables $X_i$ drawn from an unknown (but finite variance) distribution but with a known upper-cutoff $K$ and therefore support $[0,1,2,...,K]$ but un-...
- 1,001
0
votes
0
answers
763
views
Right truncation and right censoring
Is is possible for a survival data to be **right truncated and right censored **. If so, please leave an example for better understanding.
Thanks in advance!
0
votes
0
answers
118
views
Right censored or right truncation and Cox model?
Suppose, a list of targeted population is invited to participate in a health program. Invitation date could vary for every individual based on eligible criteria (this criterion is the same for all). ...
- 611
1
vote
1
answer
36
views
Can sampling from a truncation of a random variable, rather than the original variable be more Blackwell-informative?
Suppose you are interested in finding the mean of a random variable. You have some prior belief of it and before sampling 1 observation, you can decide whether to sample from the original random ...
- 141
9
votes
1
answer
3k
views
Compute truncated normal distribution with specific mean and variance
I have a simple setting: I simulate demand patterns that are distributed according to a truncated normal distribution with a given mean and variance after truncation. The truncation is from the left ...
- 159
1
vote
0
answers
96
views
Survival analysis: how to account for immortal time with time-dependent exposure
I am working on a survival analysis to look at time to preterm birth (birth before 37 weeks). I have a time-dependent exposure that can occur anytime at or after 28 weeks, defined using a heaviside ...
- 33
0
votes
0
answers
131
views
Working out the true mean of a distribution from a truncated set of observations
Suppose people receive a random draw which represents the probability of some event. The draws are taken randomly from a normal distribution with true mean $\mu$ and standard deviation $\sigma$.
...
7
votes
1
answer
3k
views
Distribution of sum of independent exponentials with random number of summands
Let $\tau_i\sim\exp\left(\lambda\right)$ be independent and identically distributed exponentials with parameter $\lambda$. Then, for given $n$, the sum of these values
$$T_n := \sum_{i=0}^n \tau_i$$
...
- 1,162
1
vote
0
answers
44
views
Truncated count model -- including information about the number of unobserved realisations
Background
Suppose we have a model such that $Y \sim \mathcal{M}(\theta)$ is a discrete random variable taking values in $[0, 1, \ldots]$. We would like to make inference about $\theta$ from a ...
6
votes
2
answers
636
views
Efficient random generation from truncated Laplace distribution
We have several ways of drawing random samples from Laplace distribution. Is there any efficient way of sampling from left truncated Laplace distribution? Inverse transform sampling is an obvious ...
Tim♦
- 121k