# Regression with different probability measure

Say we have a set of observations $$(y,X)$$ where the response variable is assumed to follow some conditional distribution under a particular probability measure $$Q_1$$, say, $$y \sim N(\mu(X),\sigma)$$. In this case, we could perform a simple OLS to get an estimate of $$E^{Q_1}[y]$$ (assuming other conditions of exogenity and linearity are also satisfied).

But what if we wanted to estimate $$E[y]$$ under some different probability measure $$Q_2$$? Can we still use OLS and apply some weights to the observations to adjust for the change of measure? Something like in importance sampling for variance reduction in MC? References on the subject would be welcome.

Following whuber's comments, here is some background to the problem I am looking at. We have, say, $$n=2^{15}$$ observations of a function of $$m$$ random variables generated by some stochastic process under some measure $$Q$$

$$y_i = f(x_{1,i}(t),...,x_{m,i}(t)) = f(\mathbf{x}_i(t))$$

$$d\mathbf{x}(t) = \mu(\mathbf{x},t)dt + \sigma(\mathbf{x},t)d\mathbf{W}^Q(t)$$

and we need to calculate expectation of $$y_i$$ at some time $$s using for covariates the observed values of $$\mathbf{x}_i(s)$$, but the expectation should be under a diferent measure $$P$$ which imposes a change in the drift of the stochastic processes

$$d\mathbf{x}(t) = \nu(\mathbf{x},t)dt + \sigma(\mathbf{x},t)d\mathbf{W}^P(t)$$

Actually, we need not just to estimate the expectation $$E^P[y|\mathbf{X}(s)]$$, but we then need to make predictions of that expectation for different values of covariates. And this has to be done many thousands of times for different functional forms $$f$$ and different covariates, so that the procedure has to be quick and fairly general.

The current approach used is to simply apply OLS, ignoring the change of measure:

$$E^Q\left[y|\mathbf{X}\right] = E^Q\left[\mathbf{X\beta + \epsilon}\right|\mathbf{X}] = \mathbf{X\beta}$$

So, since

$$E^P[y] = E^Q\left[y\frac{dP}{dQ}\right]$$

where $$\frac{dP}{dQ}$$ is the Radon–Nikodym derivative of $$P$$ with respect to $$Q$$, we have

$$E^P\left[y|\mathbf{X}\right] = E^Q\left[\left(\mathbf{X\beta + \epsilon}\right)\frac{dP}{dQ}|\mathbf{X}\right]$$

If $$\frac{dP}{dQ}$$ depends only on $$\mathbf{X}$$ then we get

$$E^P\left[y|\mathbf{X}\right] = \mathbf{X\beta}\frac{dP}{dQ}$$

So, it seems that we only need to apply some weights based on R-N derivative. But I am not sure if the above is correct, in particular, if the change of measure affects $$E^P[\epsilon|X]$$.

• This is called a "generalized linear model," or GLM. That gives you an extensive literature to search. – whuber Nov 28 '18 at 18:47
• @whuber Thank you for your comment. I am not sure that I have seen a change of measrue explcitly being delat with in GLM literature. – Confounded Nov 28 '18 at 19:13
• Could you clarify the distinction you seem to be making between "change of measure" and "arbitrary distribution"? – whuber Nov 28 '18 at 19:15
• @whuber The "distinction" I am making is that a) as far as I know, GLMs don't deal with arbitrary distributions, but those coming from a sub-class of exponential family; b) I am not aware of GLMs using Radon–Nikodym derivative to weight observations, as is, for example, done in importance sampling. – Confounded Nov 29 '18 at 9:40
• You are correct about GLM procedures being limited to exponential-class distributions, although one might argue that the approach is not limited. But the remark about the R-N derivative seems to confound the model with the procedure, which may be a potential source of confusion. Once we understand clearly what model you have in mind we can discuss procedures to fit it, so could you explain how an R-N derivative might be used to express a statistical model for a regression problem? – whuber Nov 29 '18 at 14:03