3
$\begingroup$

I'd like to be able to use a multiple linear regression model even when I don't have all the independent variables. Ideally, I'd also be able to calculate some indicator of confidence too.

Using the following example (shamelessly borrowed), we could imagine that I'd want to predict blood pressure without the "Age" factor.

| Independent Variable       | Regression Coefficient | T     | P-value |
|----------------------------|------------------------|-------|---------|
| Intercept                  | 68.15                  | 26.33 | 0.0001  |
| BMI                        | 0.58                   | 10.30 | 0.0001  |
| Age                        | 0.65                   | 20.22 | 0.0001  |
| Male gender                | 0.94                   | 1.58  | 0.1133  |
| Treatment for hypertension | 6.44                   | 9.74  | 0.0001  |

Omitting the age * 0.65 element of the regression equation would be the same as predicting for age 0, which has obvious problems. I suppose I could plug in the average age from the original dataset, but that would imply a greater precision than is true.

I'm leaning towards a "brute force" approach, whereby I'd calculate a multiple regression for each combination of factors, then select the appropriate one depending on the available data. Whilst I think this would work, it seems inelegant and I'm sure there must be a better way.

Is there a way to square this circle?

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ I don't totally understand what you're asking. You fit a model on X1, X2, and X3, but now only X1 and X2 are available to make out of sample predictions? $\endgroup$
    – shadowtalker
    Commented Jun 23, 2020 at 16:58
  • $\begingroup$ @shadowtalker That's exactly it. $\endgroup$
    – Tom Wright
    Commented Jun 23, 2020 at 17:39
  • $\begingroup$ How is it that you don't have the age variable? Can't you ask the patient how old they are? What do you want the prediction for? Are you trying to estimate the population average for some specified sex, BMI & Tx combination? $\endgroup$
    – gung - Reinstate Monica
    Commented Jun 23, 2020 at 17:47
  • $\begingroup$ @gung-ReinstateMonica The blood pressure example is just illustrative. Without boring you all with the details, the actual problem is very niche, highly dimensional, and (unfortunately) has patchy data available. $\endgroup$
    – Tom Wright
    Commented Jun 24, 2020 at 7:53
  • $\begingroup$ @gung-ReinstateMonica I missed you question on the purpose of the prediction... Within the fictional example, we're interested in the likely "blood pressure" of individuals, but only for the purposes of evaluating risk (i.e. not diagnostic). Accuracy is somewhat important, but characterising the confidence of the prediction is more so. $\endgroup$
    – Tom Wright
    Commented Jun 24, 2020 at 7:56

1 Answer 1

Reset to default
3
$\begingroup$

I don't know if it is more elegant, but I imagine multiple imputation will give better results than your brute force approach. The process requires coming up with a stochastic imputation model. This model will predict the value of age based on BMI, gender, etc., and it will also incorporate a random component. For example, you might fit the model on a random subset of the data, or add a noise term to your estimate analagous to the residual error.

You can then apply this imputation model repeatedly to predict your missing value until you have multiple predicted values for your missing variable. You can then predict blood pressure from your data using each imputed value for age. The central tendency and spread of your final predictions will give you an idea of the best estimate and its uncertainty.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I like this idea. The imputation model could be a simple as fitting a Gaussian distribution to your training set, or something more sophisticated. $\endgroup$
    – shadowtalker
    Commented Jun 23, 2020 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.