Consider a random variable $Y$ with mean: $y = \alpha + \beta x$ and variance $\sigma^{2}$. Assume $n$ independent values of $Y$ are observed and are split into two samples such that the samples can be labelled: $n_{1}$ and $n_{2}$ ($n_{2} = n-n_{1}$); furthermore, assume that the observations in both groups share a slope/gradient ($\beta$), but have differing intercepts ($\alpha$). How can it be shown that:

$$\hat{\beta} = \frac{w_{1}\hat{\beta_{1}} + w_{2}\hat{\beta_{2}}}{w_{1}+w_{2}}$$

where $\hat{\beta_{1}}$ and $\hat{\beta_{2}}$ are the estimates for $\beta$ derive from each sample and $w_{1}$ and $w_{2}$ are the sums of squares of the $x$ values (independent variables) about their mean for each sample.

In addition, what form does $Var(\hat{\beta})$ have?

Finally, for clarity, following on from the above model, is it correct to assume that, the intercept ($\alpha$) for each sample will be given by:

$$\alpha_{j} = \bar{Y}^{(j)}-\beta_{j}\bar{X}^{(j)} \text{, for } j=1,2$$

And the gradient/slope ($\beta$) is given by:

$$\hat{\beta_{j}} = \frac {\sum_{i=1}^{n} X_{i}^{(j)}Y_{i}^{(j)}-n\bar{X}^{(j)}\bar{Y}^{(j)}} {\sum_{i=1}^{n} X_{i}^{(j)2}-n\bar{X}^{(j)2}} = \frac{S_{XY}}{S_{XX}} \text{, for } j=1,2$$

(Note: The $(j)$ notation in the above ($X^{(j)}$, $Y^{(j)}$, etc.) is intended to denote sample to which each data point belongs -- It is not exponential notation.)


I should probably mention that, originally, I had assumed that I could use the method of least squares to sum the $SS_{E}$ terms from each sample, set the partial derivative w.r.t. $\beta$ equal to zero and solve for $\beta$; such that:

$$ \sum_{j=1}^{2} \frac{\partial SS_{E}^{(j)}}{\partial \beta} = \frac{\partial SS_{E}^{(1)}}{\partial \beta} + \frac{\partial SS_{E}^{(2)}}{\partial \beta} = 0$$

$$ = \frac{\partial} {\partial \beta} [\sum_{i=1}^{n - n_{1}} (y_{i}^{(1)}-\hat{\alpha_{1}}-\hat{\beta}x_{i}^{(1)})^{2} + \sum _{i=n+1-n_{1}}^{n} (y_{i}^{(2)}-\hat{\alpha_{2}}-\hat{\beta}x_{i}^{(2)})^{2}] = 0$$

$$ = -2 \sum_{i=1}^{n - n_{1}} [(y_{i}^{(2)}-\hat{\alpha_{1}}-\hat{\beta}x_{i}^{(2)})(x_{i}^{(2)})] -2 \sum_{i=n+1-n_{1}}^{n} [(y_{i}^{(2)}-\hat{\alpha_{2}}-\hat{\beta}x_{i}^{(2)})(x_{i}^{(2)})] = 0$$

However, this does not appear to be equivalent to the form given above, so I'm assuming this is not a valid assumption.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.