# What are the consequences of "copying" a data set for OLS?

Suppose I have a random sample $\lbrace X_i, Y_i\rbrace_{i=1}^n$. Assume this sample is such that the Gauss-Markov assumptions are satisfied such that I can construct an OLS estimator where

$$\hat{\beta}_1^{OLS} = \frac{\text{Cov}(X,Y)}{\text{Var(X)}}$$ $$\hat{\beta}_0^{OLS} = \bar{Y} - \bar{X} \hat{\beta}_1^{OLS}$$

Now suppose I take my data set and double it, meaning there is an exact copy for each of the $n$ $(X_i,Y_i)$ pairs.

# My Question

How does this affect my ability to use OLS? Is it still consistent and identified?

• This is a linear transformation of the data, why don't you try seeing how this affects the estimator directly? Commented Jun 2, 2016 at 17:33
• @JohnK It's difficult to conceive of this as a "linear transformation" when it actually introduces additional data!
– whuber
Commented Jun 2, 2016 at 17:35
• @whuber Linear in the sense that $x_i \mapsto 2x_i$. Commented Jun 2, 2016 at 17:38
• @Johnk The OP isn't doubling the values in the data set but stacking two copies of the same data set. Commented Jun 2, 2016 at 17:39
• $x_i \mapsto (x_i,x_i)$ is still linear ... Commented Jan 5, 2019 at 10:46

Do you have a good reason to do the doubling (or duplication?) It doesn't make much statistical sense, but still it is interesting to see what happens algebraically. In matrix form your linear model is $$\DeclareMathOperator{\V}{\mathbb{V}} Y = X \beta + E,$$ the least square estimator is $$\hat{\beta}_{\text{ols}} = (X^T X)^{-1} X^T Y$$ and the variance matrix is $$\V \hat{\beta}_{\text{ols}}= \sigma^2 (X^t X)^{-1}$$. "Doubling the data" means that $$Y$$ is replaced by $$\begin{pmatrix} Y \\ Y \end{pmatrix}$$ and $$X$$ is replaced by $$\begin{pmatrix} X \\ X \end{pmatrix}$$. The ordinary least squares estimator then becomes $$\left(\begin{pmatrix}X \\ X \end{pmatrix}^T \begin{pmatrix} X \\ X \end{pmatrix} \right )^{-1} \begin{pmatrix} X \\ X \end{pmatrix}^T \begin{pmatrix} Y \\ Y \end{pmatrix} \, =\, \\ (X^T X + X^T X)^{-1} (X^T Y + X^T Y ) \, = \, (2 X^T X)^{-1} 2 X^T Y = \\ \hat{\beta}_{\text{ols}}$$ so the calculated estimator doesn't change at all. But the calculated variance matrix becomes wrong: Using the same kind of algebra as above, we get the variance matrix $$\frac{\sigma^2}{2}(X^T X)^{-1}$$, half of the correct value. A consequence is that confidence intervals will shrink with a factor of $$\frac{1}{\sqrt{2}}$$.

The reason is that we have calculated as if we still have iid data, which is untrue: the pair of doubled values obviously have a correlation equal to $$1.0$$. If we take this into account and use weighted least squares correctly, we will find the correct variance matrix.

From this, more consequences of the doubling will be easy to find as an exercise, for instance, the value of R-squared will not change.

• And in fact, you can show that this is equivalent to a weighted least squares with weight 2 on each observation. Commented Jun 2, 2016 at 19:21
• Surprisingly, there are areas of statistics in which there is (arguably) sometimes a reason to do this. It goes by the name "data cloning". Commented Jun 2, 2016 at 23:17
• Interesting! datacloning.org/dc.html Commented Jun 3, 2016 at 7:30
• In Python, the $\sigma^2$ is estimated using $\hat{\sigma}^2$ which is computed using $\frac{1}{N - p - 1}$ factor rather than $\frac{1}{N}$, this actually makes the factor of $\frac{1}{\sqrt{2}}$ slightly bigger.
– 24n8
Commented May 6, 2021 at 19:29
• @kjetilbhalvorsen I think I was confused by the terms $rank$. In this case, it means the column-rank. If $X$ is full column rank, then $X^TX$ is invertible. And I see your point, adding new rows does not change the column rank. Commented Mar 16, 2022 at 23:31

I am not yet familiar enough with the theory to give you a very mathematical answer, but intuitively, OLS only cares about proportions in which different cases are present. This makes sense when you recall that OLS chooses the coefficients that minimize the mean of the squared residuals, and the mean reflects purely the proportions of its inputs (in the sense that the mean of (1, 3, 3) is the same as the mean of a dataset with a million 1s and two million 3s). So, doubling the dataset will get you the identical model.

Here's an R example, where we generate a random regression problem and notice that the coefficients are unchanged when doubling the data:

nc = sample(1:10, 1, replace = T)
n = sample(11:500, 1, replace = T)
x = as.matrix(replicate(nc, rnorm(n)))
coef = rnorm(nc)
sd.resid = runif(1, 0, 5)

y = x %*% matrix(coef) + rnorm(n, sd = sd.resid)

print(cbind(
coef(lm(y ~ x)),
coef(lm(c(y, y) ~ rbind(x, x)))))


One run gives me:

                   [,1]        [,2]
(Intercept) -0.10002238 -0.10002238
x1          -2.14801619 -2.14801619
x2           0.23120764  0.23120764
x3           0.05360792  0.05360792
x4           1.91972198  1.91972198
x5          -1.09887264 -1.09887264
x6           0.04248358  0.04248358

• As statisticians, we do far more than just estimate parameters. At the least, we evaluate how uncertain they are. Doubling the data will (typically and approximately) halve the variances, making the results look much more solid than they ought to. This error has huge ramifications for interval estimation, such as confidence and prediction intervals. Another way to characterize the mistake is to recognize that each doubled pair of responses is perfectly correlated, so applying OLS to them as if they were independent uses the wrong error structure.
– whuber
Commented Jun 2, 2016 at 17:32
• I agree, but to me, that is a matter of inferences about the model rather the model itself. Doing something like computing a confidence interval for a parameter is not part of OLS per se, even if the parameter comes from OLS. Commented Jun 2, 2016 at 17:41
• Yes, that's why I said inferences about the model. I would agree that OLS per se is a kind of algorithmic curve-fitting. Commented Jun 2, 2016 at 17:45
• In that case, what does one call the procedure of computing the coefficients of a linear model such that the sum of the squared deviations is minimized? Commented Jun 2, 2016 at 17:54
• @Iamanon You calculate the variance as if the errors in $(Y,Y)^\prime$ were iid--but they are not: they come in perfectly correlated pairs.
– whuber
Commented Aug 9, 2020 at 13:37