I am implementing an R-type summary() function in python with the restriction to exclude use of scientific libraries. (assignment) I found this https://www.nd.edu/~rwilliam/stats1/x91.pdf material explaining the calculation of standard errors of coefficients. As I already had a multiple linear model coded with gradient descent, I was able to code the standard error of coefficients, except the intercept. I just do not understand how to calculate $S{_b}_0$ based on the below equation:

Standard error of regression coefficient I can calculate $R^{2}{_X}_k{_G}_k$ for k>0, but not for k=0

The following code piece works already, but I need something for the (z=-1) below, which would fill the coeff_stderr[0] element:

        for z in range(len(X[0])):
            xk_vector = get_matrix_column(X, z)
            var_xk = variance(xk_vector)
            RRxk = coefficient_of_determination_Xk(X, z, num_iters, alpha)
            coeff_stderr[z+1] = RSE / sqrt((1-RRxk)*var_xk*(len(X)-1))

what is $X_0$? The intercept term - the column with 1s? Then it's variance is 0, and I would divide with 0...

  • 1
    $\begingroup$ @Isvan Orosz you might find this useful stats.stackexchange.com/questions/439966/… $\endgroup$ – PsychometStats Dec 9 '19 at 22:24
  • $\begingroup$ As I understand you are suggesting a matrix inversion along the line? I can't do that the moment with my program. $\endgroup$ – István Orosz Dec 10 '19 at 14:13
  • $\begingroup$ Yes, you will need to invert covariance matrix to calculate parameters $\endgroup$ – PsychometStats Dec 10 '19 at 14:22
  • $\begingroup$ if you check out the formula in the linked .pdf, it does not mention inverting covariance matrix. And as I mentioned, the formula works for the coefficients, except the intercept. My solution coefficient_of_determination_Xk() works with gradient descent. $\endgroup$ – István Orosz Dec 10 '19 at 14:59
  • $\begingroup$ You don't need to invert the matrix: you only need to compute one entry in its inverse. (That's where the formula you quote comes from, by the way: it gives diagonal entries of the inverse.) $\endgroup$ – whuber Dec 10 '19 at 19:02

I gave up finding the magic formula and coded the matrix inversion routine. this is the final piece, I calculate all the standard errors of the coefficients with this:

        # add intercept term to X (to the left)
        X = concatenate_matrices_by_column([[1.0]] * len(X), X)
        XX = multiply_matrices(transpose(X),X)
        vc = invert_matrix(XX)
        for k in range(len(vc)):
            coeff_stderr[k] = sqrt(vc[k][k] * SSE / df_residual)
| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.