What are the consequences of “duplicating” a subset of data for OLS? Suppose I have a sample $\{X_i,Y_i\}_{i=1}^n$. Then the OLS estimator of the slope coefficient is given by $$\hat{\beta}=\frac{Cov(X,Y)}{Var(X)}$$
Now suppose I take my data set and replicate a subset of it, meaning there are $m_i$ copies for each $(X_i,Y_i)$ pairs. How does this affect OLS estimates? I suspect that it is a weighted OLS estimator, but I can't prove it.
 A: Suppose the original design matrix and response vector are
\begin{align*}
X = \begin{pmatrix}
x_1^T \\
x_2^T \\
\vdots \\
x_n^T
\end{pmatrix} \in \mathbb{R}^{n \times p},
\quad 
y = \begin{pmatrix}
y_1 \\
y_2 \\
\vdots \\
y_n
\end{pmatrix} \in \mathbb{R}^n
\end{align*}
respectively, then the "duplicated" design matrix and response vector, according to your description, are
\begin{align*}
X^\dagger = \begin{pmatrix}
e_1 \otimes x_1^T \\
e_2 \otimes x_2^T \\
\vdots \\
e_n \otimes x_n^T
\end{pmatrix} \in \mathbb{R}^{(m_1 + \cdots + m_n) \times p},
\quad 
y^\dagger = \begin{pmatrix}
e_1 \otimes y_1 \\
e_2 \otimes y_2 \\
\vdots \\
e_n \otimes y_n
\end{pmatrix} \in \mathbb{R}^{m_1 + \cdots + m_n},
\end{align*}
where $e_i$ is an $m_i \times 1$ column vector consisting of all ones, $i = 1, \ldots, n$, and "$\otimes$" stands for Kronecker product.
Given that, you can apply the OLS formula to calculate the new OLS estimate based on the duplicated data $\{X^\dagger, y^\dagger\}$ (by the way, what you wrote in your question is not the sample-level OLS estimate, the correct one is $\hat{\beta} = (X^TX)^{-1}X^Ty$.):
\begin{align*}
 &\tilde{\beta} = (X^{\dagger T}X^\dagger)^{-1}X^{\dagger T}y^\dagger \\
=& (m_1x_1x_1^T + \cdots + m_nx_nx_n^T)^{-1}(m_1y_1x_1 + \cdots + m_ny_nx_n) \\
=& (X^TWX)^{-1}X^TWy,
\end{align*}
where $W$ is the diagonal matrix $\mathrm{diag}(m_1, \ldots, m_n)$. In the above calculation, we used two properties of Kronecker product:
\begin{align*}
& (A \otimes B)^T = A^T \otimes B^T, \\
& (A \otimes B)(C \otimes D) = (AC)\otimes(BD).
\end{align*}
So your conjecture is correct -- $\tilde{\beta}$ does have a weighted LS (probably we shouldn't say "weighted OLS", because "weighted" implies it is not "ordinary") form.
