Changing a row of $A$ will give a rank-1 update of $A'A$, recomputing the inverse of a matrix following a rank-1 update can be achieved using the Sherman-Woodbury-Morrison formula. However this is numerically unstable, so it is better to perform the computation using a cholesky decomposition instead, which can be updated in a similar, but more numerically stable manner. IIRC MATLAB has some commands built in for this, but I can't remember the names ('lookfor cholesky' ought to find them). See also Matthias Seeger's software for Updating the Cholesky Decomposition for Low Rank Modifications. Matthias is a very bright guy, I suspect this is probably the best approach.
UPDATE - The command is cholupdate:
R1 = cholupdate(R,x) where R = chol(A) is the original Cholesky factorization of A, returns the upper triangular Cholesky factor of A + x*x', where x is a column vector of appropriate length. cholupdate uses only the diagonal and upper triangle of R. The lower triangle of R is ignored
so R1 = chol(A1'*A1); followed by R2=cholupdate(R1,x); ought to do the trick, where x is the differrence between the row of A1 that is replaced to get A2 and the new row in A2. I can't check as I don't have access to MATLAB at home, but it will be something of that nature. You can then get $\beta_2$ by the standard method for regression with Cholesy decomposition.