Linked Questions

0
votes
0answers
54 views

Summation representation for multivariate regressions (or other time-saving techniques) [duplicate]

Possible Duplicate: Efficient online linear regression Is there a summation representation for multivariate regressions? For example, if I regress $y$ on $X$ instead of using $\hat \beta = (X'X)^...
2
votes
0answers
41 views

incremental $R^2$ update at every new sample [duplicate]

I am sampling from a random process $X$ and I would like to calculate $R^2$ for the cumulative sum of the samples: $$x_1,..x_n$$ $$y_n=\sum_0^n x_i$$ $$R^2_n=RSQ( [1,2,...n], [y_1,y_2,..,y_n])$...
32
votes
7answers
18k views

Are there algorithms for computing “running” linear or logistic regression parameters?

A paper "Accurately computing running variance" at http://www.johndcook.com/standard_deviation.html shows how to compute running mean, variance and standard deviations. Are there algorithms where the ...
16
votes
3answers
12k views

Computation of new standard deviation using old standard deviation after change in dataset

I have an array of $n$ real values, which has mean $\mu_{old}$ and standard deviation $\sigma_{old}$. If an element of the array $x_i$ is replaced by another element $x_j$, then new mean will be $\...
8
votes
2answers
963 views

Intuition for recursive least squares

The least squares formula, $\beta = (X'X)^{-1}X'Y$ can be recursively formulated as \begin{align} \beta_t &= \beta_{t-1} +\frac{1}{t}R_t^{-1}x_t'(y_t-x_t\beta_{t-1}),\\ R_t &= R_{t-1}+\frac{1}...
7
votes
1answer
1k views

How to calculate standard errors in OLS without inverting the X'X matrix?

This may seem a trivial question but I haven't found a satisfactory answer anywhere. I need to compute standard errors in a OLS regression $y = X\beta + u$ in R from scratch. How can I do this without ...
4
votes
1answer
757 views

Is it possible to seed RANSAC with a given line?

I am analyzing a stream of data and I want to seed every new instance with the best guess output (line) of the previous, so as to eventually converge. Given that Scikit Learn - RANSAC is an iterative ...
12
votes
1answer
310 views

Online, scalable statistical methods

This was inspired by Efficient online linear regression, which I found very interesting. Are there any texts or resources devoted to large-scale statistical computing, by which computing with ...
2
votes
1answer
60 views

Online update of Pearson coefficient

Suppose I have an online stream of data points $x_i,y_i$, where $i=1,2,\dots$. I want to compute the Pearson correlation coefficient between the vectors $\vec x$ and $\vec y$. But here is the catch. ...
1
vote
0answers
216 views

pattern classification with regression--singular matrix

For learning purposes I have been using a subset of the MNIST data--just the training set and just the digits 3 and 8. I want to try linear regression for doing this classification. (Yes I realise ...
0
votes
1answer
102 views

Combining multiple OLS Regressions

I have a single output $y$, and multiple inputs $x_1, x_2,\dots,x_n$. I am running online(streaming) regression, which would be complicated with many inputs. So, to go around it, I want to have $n$ ...
3
votes
0answers
139 views

Aggregate variance for ordinary least squares?

I am writing some MapReduce code to calculate ordinary least squares from a sample of data. I'd like to include standard error, but I am running into a problem in calculating the variance of the noise....
2
votes
0answers
134 views

Bounds on coefficients of linear regression when changing data

Are there any general results on bounds for linear regression model coefficients if we change the data set? For example, if I learn a model with a few data points less, what will happen to the model ...
1
vote
1answer
60 views

Online algorithm to compute rolling standard error for intercept in multiple regression

Suppose I'm fitting a multiple linear model $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2$ + .... My $n$ data vectors are arriving sequentially and I'm trying to get the $t$-statistic for the estimated ...
2
votes
0answers
82 views

Large ridge regression

The following is a question pertaining to a large scale ridge regression. I am stumped by this question, any one have an idea? Thanks Suppose the data for the ridge regression problem becomes ...

15 30 50 per page