0
$\begingroup$

Imagine I am measuring a bounded variable (with a maximum possible value above which the data doesn't make sense) and I end up with the following dataset with my measurements and measurement errors as follows:


set.seed(123)
dataset <- data.frame(Measurements = rnorm(n = 100, mean = 0.9, sd = 0.1),
                      Standard.Error = rnorm(n = 100, mean = 0.1, sd = 0.02))

Now consider I have prior knowledge of this dataset and I know that the maximum possible value each measurement can take is 1. However, due to measurement error and perhaps due to technical artefacts in the equipment used to make these measurements, I end up with some values above this limit:

hist(dataset$Measurements)
abline(v=1,col="red")

enter image description here My question is: is there any principled way of using my prior knowledge of the data boundary (max allowed value = 1), as well as my measurements and measurement errors, to "correct" the outlier data such that they fall inside the allowed range? Is my only option to set them all to the maximum allowed value (1) or is there anything else I could do?

Thanks in advance!

EDITS to reflect additional information I was asked about in the comments

EDIT 1: I'm measuring a variable that takes any value between 0 and 1 (a proportion or a fraction). Because of how I'm measuring the data, I sometimes get values above 1 (although these are theoretically impossible). Further down the line, I want to logit-transform the data because that is the space in which the different datapoints interact additively with each other, but I cannot logit-transform anything above 1.

EDIT 2: I'm trying to measuring a proportion between events A and B. However, if I measure all my datapoints at once, due to technical reasons I can only measure event A. What I can do, however, is measure the real proportions for a selection of datapoints, and then use these accurate measurements to calibrate all of my other datapoints. Because I am not directly measuring the proportions for all my data points, and the calibration line may be off for some of my datapoints, I sometimes end up with "impossible" proportions. This is a problem intrinsic to the measurement method I'm using and is something I cannot avoid.

Three things I want to do in my downstream analysis:

  • compare the proportions among the different datapoints in my experiment.
  • compare the proportion distribution across different experiments (here I described just one experiment and one such distribution)
  • The datapoints are associated with a number of variables X, Y and Z. A linear model to predict the datapoints from X, Y and Z does not work because the data is bounded. However, the data behaves linearly in logit space, so what I have been doing in the past is to logit-transform the datapoints and then bulid a linear model. I think cannot do this for datapoints with a fraction > 1.
$\endgroup$
4
  • $\begingroup$ Could you please say why you want to "correct" the observed data in this way? The "corrected" data would, for example, have a bias in their mean value from the actual value of 0.9. If you could say more about what you are trying to do, at a higher level of your study, there might be ways to accomplish what you wish without such a "correction." $\endgroup$
    – EdM
    Commented May 15, 2019 at 13:46
  • $\begingroup$ Thanks for replying. I'm measuring a variable that takes any value between 0 and 1 (a proportion or a fraction). Because of how I'm measuring the data, I sometimes get values above 1 (although these are theoretically impossible). Further down the line, I want to logit-transform the data because that is the space in which the different datapoints interact additively with each other, but I cannot logit-transform anything above 1. $\endgroup$
    – Ender
    Commented May 15, 2019 at 13:54
  • $\begingroup$ It's often better to analyze the raw data behind such proportions rather than the proportions themselves. For example, with the raw data you have information on the numbers of cases contributing to each proportion observation and can thus weight observations correspondingly. Please say more about the nature of the raw data, why you sometimes get "impossible" proportions, and what downstream analysis you want to perform on the data. It's best to edit the question itself to add this information (and that in your prior comment) as comments can sometimes be deleted. $\endgroup$
    – EdM
    Commented May 15, 2019 at 14:05
  • $\begingroup$ I didn't know that comments could be deleted! I have added the information you asked about to the main text. $\endgroup$
    – Ender
    Commented May 15, 2019 at 14:24

1 Answer 1

0
$\begingroup$

What you face is a missing data problem, which you have tried to overcome with a particular calibration method. This is data imputation, on which there are over 400 questions on this site. Data imputation is often a good choice and can improve the power of a study provided that the reasons for missingness are completely at random or depend completely on data that are at hand. There are, however, some better ways to handle the imputation than what you have tried so far.

First, your calibration method is leading to predicted ratios that exceed a theoretical limit of 1. That suggests that you might need to try a different calibration method. Although I'm still a bit sketchy on just what you mean by "a proportion between events A and B," I'll take that to mean that over some observation interval you have events of both type A and type B and that the number of type B events can never exceed that of the type A events. For example, perhaps all events are type A but only a portion of those have additional characteristics that make them type B.

In that case your calibration/imputation method itself should enforce that requirement. You can do that with a logistic regression calibration (thus introducing your logit transformation at an earlier stage of your analysis). Although logistic regression is often used for analysis of binary outcomes, it also can accommodate counts of different types that can be thought of as generalizations of successes and failures. For example, in the scenario where all events are A but only a fraction are B, you could count B events as successes and non-B events as failures and directly use the glm() function in R to model the proportions in cases where you have data, and then use predictions from the logistic regression to impute the missing B fractions from the A observations in other cases. Then you would never have a predicted B/A ratio that exceeds 1.

The second issue is that imputation necessarily introduces uncertainties into the analysis: the imputed data aren't as reliable as the actual observations. The standard way to deal with this is with multiple imputation. You make several separate imputations of the missing data with a probabilistic approach, analyze each of the imputed data sets separately, and then combine the information from all the analyses to take into account the variability introduced by the imputation. That might sound complicated but there is a widely used R package called mice that can do all this for you while using extremely flexible ways to do the imputations that might well perform better than the simple logistic regression I suggested above.

$\endgroup$
6
  • $\begingroup$ Many thanks for the detailed explanation! This makes a lot of sense and I'll try to do this. You were right in asking for more details about my problem as I was going about it in a very roundabout way! Have a nice day :) $\endgroup$
    – Ender
    Commented May 15, 2019 at 15:21
  • $\begingroup$ @Ender it's very easy to get caught up in difficulties with one particular type of solution when the project actually requires a different type of solution. That often happens with questions on this site. $\endgroup$
    – EdM
    Commented May 15, 2019 at 15:35
  • $\begingroup$ I have been thinking about this yesterday and today. Although I understand what you wrote, I'm not sure I understand what to do in my case. For the sake of making things clear I will re-define B as successes and A as failures, and the total number of events for a sample = A + B. I have counts of B, but not counts of A. For a subset of samples I have both A and B. So, I was previously using a linear model to calibrate all samples. Since B is be linearly related to B/(A+B), I was imputing A by modeling B/(A+B) ~ B using a linear model. How would I use a logistic regression to impute A? $\endgroup$
    – Ender
    Commented May 16, 2019 at 10:26
  • $\begingroup$ @Ender model the A/B ratio based on the available A and B data and on other covariate data. Otherwise you are assuming a relationship between A and B counts that doesn't change with experimental conditions. For a logistic model you could model cases with A and B information as: logit(A/B) ~ B+ f(covariates), that is, as a function of covariate values too. Then, in cases without data on A, use the A/B ratio predictions from corresponding covariate values to estimate the A values. Themice package does this in a more sophisticated and generally better way. $\endgroup$
    – EdM
    Commented May 16, 2019 at 12:14
  • $\begingroup$ OK this is very reasonable thank you very much $\endgroup$
    – Ender
    Commented May 16, 2019 at 12:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.