# What are the consequences of "copying" a data set for OLS?

Suppose I have a random sample $\lbrace X_i, Y_i\rbrace_{i=1}^n$. Assume this sample is such that the Gauss-Markov assumptions are satisfied such that I can construct an OLS estimator where

$$\hat{\beta}_1^{OLS} = \frac{\text{Cov}(X,Y)}{\text{Var(X)}}$$ $$\hat{\beta}_0^{OLS} = \bar{Y} - \bar{X} \hat{\beta}_1^{OLS}$$

Now suppose I take my data set and double it, meaning there is an exact copy for each of the $n$ $(X_i,Y_i)$ pairs.

# My Question

How does this affect my ability to use OLS? Is it still consistent and identified?

• This is a linear transformation of the data, why don't you try seeing how this affects the estimator directly? Jun 2, 2016 at 17:33
• @JohnK It's difficult to conceive of this as a "linear transformation" when it actually introduces additional data!
– whuber
Jun 2, 2016 at 17:35
• @whuber Linear in the sense that $x_i \mapsto 2x_i$. Jun 2, 2016 at 17:38
• @Johnk The OP isn't doubling the values in the data set but stacking two copies of the same data set. Jun 2, 2016 at 17:39
• $x_i \mapsto (x_i,x_i)$ is still linear ... Jan 5, 2019 at 10:46

Do you have a good reason to do the doubling (or duplication?) It doesn't make much statistical sense, but still it is interesting to see what happens algebraically. In matrix form your linear model is $$\DeclareMathOperator{\V}{\mathbb{V}} Y = X \beta + E,$$ the least square estimator is $$\hat{\beta}_{\text{ols}} = (X^T X)^{-1} X^T Y$$ and the variance matrix is $$\V \hat{\beta}_{\text{ols}}= \sigma^2 (X^t X)^{-1}$$. "Doubling the data" means that $$Y$$ is replaced by $$\begin{pmatrix} Y \\ Y \end{pmatrix}$$ and $$X$$ is replaced by $$\begin{pmatrix} X \\ X \end{pmatrix}$$. The ordinary least squares estimator then becomes $$\left(\begin{pmatrix}X \\ X \end{pmatrix}^T \begin{pmatrix} X \\ X \end{pmatrix} \right )^{-1} \begin{pmatrix} X \\ X \end{pmatrix}^T \begin{pmatrix} Y \\ Y \end{pmatrix} = \\ (x^T X + X^T X)^{-1} (X^T Y + X^T Y ) = (2 X^T X)^{-1} 2 X^T Y = \\ \hat{\beta}_{\text{ols}}$$ so the calculated estimator doesn't change at all. But the calculated variance matrix becomes wrong: Using the same kind of algebra as above, we get the variance matrix $$\frac{\sigma^2}{2}(X^T X)^{-1}$$, half of the correct value. A consequence is that confidence intervals will shrink with a factor of $$\frac{1}{\sqrt{2}}$$.

The reason is that we have calculated as if we still have iid data, which is untrue: the pair of doubled values obviously have a correlation equal to $$1.0$$. If we take this into account and use weighted least squares correctly, we will find the correct variance matrix.

From this, more consequences of the doubling will be easy to find as an exercise, for instance, the value of R-squared will not change.

• And in fact, you can show that this is equivalent to a weighted least squares with weight 2 on each observation. Jun 2, 2016 at 19:21
• Surprisingly, there are areas of statistics in which there is (arguably) sometimes a reason to do this. It goes by the name "data cloning". Jun 2, 2016 at 23:17
• Interesting! datacloning.org/dc.html Jun 3, 2016 at 7:30
• In Python, the $\sigma^2$ is estimated using $\hat{\sigma}^2$ which is computed using $\frac{1}{N - p - 1}$ factor rather than $\frac{1}{N}$, this actually makes the factor of $\frac{1}{\sqrt{2}}$ slightly bigger.
– 24n8
May 6, 2021 at 19:29
• It may be an obvious question. But after duplicating the data, say just add duplicate one row of $X$, then the new data matrix $X$ is not full rank, thus not invertible, and neither is $X^TX$. But how do we still have $\hat{\beta} = (X^TX)^{-1}X^TY$? That will imply the solution is unique(but it is not because $X$ is not full rank) Mar 16 at 22:26

I am not yet familiar enough with the theory to give you a very mathematical answer, but intuitively, OLS only cares about proportions in which different cases are present. This makes sense when you recall that OLS chooses the coefficients that minimize the mean of the squared residuals, and the mean reflects purely the proportions of its inputs (in the sense that the mean of (1, 3, 3) is the same as the mean of a dataset with a million 1s and two million 3s). So, doubling the dataset will get you the identical model.

Here's an R example, where we generate a random regression problem and notice that the coefficients are unchanged when doubling the data:

nc = sample(1:10, 1, replace = T)
n = sample(11:500, 1, replace = T)
x = as.matrix(replicate(nc, rnorm(n)))
coef = rnorm(nc)
sd.resid = runif(1, 0, 5)

y = x %*% matrix(coef) + rnorm(n, sd = sd.resid)

print(cbind(
coef(lm(y ~ x)),
coef(lm(c(y, y) ~ rbind(x, x)))))


One run gives me:

                   [,1]        [,2]
(Intercept) -0.10002238 -0.10002238
x1          -2.14801619 -2.14801619
x2           0.23120764  0.23120764
x3           0.05360792  0.05360792
x4           1.91972198  1.91972198
x5          -1.09887264 -1.09887264
x6           0.04248358  0.04248358

• As statisticians, we do far more than just estimate parameters. At the least, we evaluate how uncertain they are. Doubling the data will (typically and approximately) halve the variances, making the results look much more solid than they ought to. This error has huge ramifications for interval estimation, such as confidence and prediction intervals. Another way to characterize the mistake is to recognize that each doubled pair of responses is perfectly correlated, so applying OLS to them as if they were independent uses the wrong error structure.
– whuber
Jun 2, 2016 at 17:32
• I agree, but to me, that is a matter of inferences about the model rather the model itself. Doing something like computing a confidence interval for a parameter is not part of OLS per se, even if the parameter comes from OLS. Jun 2, 2016 at 17:41
• Inference is impossible without information about the potential inaccuracy. Otherwise, you have no valid response to the question "how sure are you of your answer?" That is why (at a minimum) computing a variance-covariance matrix of the residuals is an essential part of OLS. Anything less is just algorithmic curve fitting.
– whuber
Jun 2, 2016 at 17:42
• Yes, that's why I said inferences about the model. I would agree that OLS per se is a kind of algorithmic curve-fitting. Jun 2, 2016 at 17:45
• In that case, what does one call the procedure of computing the coefficients of a linear model such that the sum of the squared deviations is minimized? Jun 2, 2016 at 17:54