Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

RMA, MA, and OLS Regression Line [closed]

When will RMA (reduced major axis), MA (major axis), and OLS regression line be the same? And how will RMA regression line be different from OLS regression other than differences in the treatment of ...
2
votes
1answer
17 views

What exactly is violated if your response variable exhibits autocorrelation in ordinary least squares regression?

I'm trying to understand the issues with using OLS regression when our data exhibits autocorrelation. Let's say you simulate a process where: ...
0
votes
0answers
33 views

Intercept with parameter estimate 0.0000 and p-value 1.0000. Interpretation/meaning?

I get the following information from an estimated model: ...
1
vote
0answers
17 views

Least Squares removing first $k$ observations Woodbury formula

Given the matrix $X_{n,p}$ from the least squares problem $$ \mathbf{X} \cdot \mathbf{\beta} = z $$ Where the normal equation is: $$ \mathbf{\hat{\beta}} = \left(\mathbf{X}^T \mathbf{X}\right)^{-1} ...
0
votes
0answers
7 views

Looking for a toy example of Damped Least Squares

I've implemented Gauss Newton for a homework problem, and the normal equations for that are pretty straight forward, however, while the math for DAMPED least squares is somewhat understandable, I've ...
1
vote
0answers
19 views

OLS regression in stata. Dummy dependent variable vs if== at the end of the coding

I have a question, I'm quite new to the field so apologies if it is silly question. I have the following regression. Y(tcs)= X1(tcs), X2(tcs), X3(tcs), u t= time, I have eleven years c= countries, ...
0
votes
0answers
21 views

What's the difference between the OLS, MLE and MOM estimates for AR model? Derivations of each would be MUCH appreciated

I'm in a time series class, however we have only been taught how do derive OLS, so comparisons would be useful, thank you.
0
votes
1answer
24 views

When using a gaussian link in GLM, what are the assumptions?

In R, when I am fitting a model glm(y~x, family = gaussian(link="log")), do I assume that $Y \stackrel{iid}\sim N(\mu, \sigma^2)$ or do I assume that $Y \stackrel{...
0
votes
0answers
29 views

Cross-sectional pooled regression

I have data from three surveys conducted in three different years (2012, 2014, 2016). Each survey was administered to public managers, but not necessarily the same ones (some retired or changed ...
13
votes
1answer
393 views

OLS as approximation for non-linear function

Assume a non-linear regression model \begin{align} \mathbb E[y \lvert x] &= m(x,\theta) \\ y &= m(x,\theta) + \varepsilon, \end{align} with $\varepsilon := y - m(x,\theta)$...
0
votes
1answer
23 views

Panel data OLS regression: plot and quadratic fitting line

I am conducting an OLS regression panel data analysis with package PLM in R. I use the following script to obtain a plot and fitting line of variables D and ...
0
votes
2answers
33 views

With Ordinary Least Squares, how do we know that when the partial derivative of RSS is 0, that is a minima?

To minimise the residual sum of squares, we take its derivative with respect to the beta parameters and set this to 0. But when a derivative is set to 0, this means it can be one of the minima or one ...
0
votes
0answers
8 views

Resolution Matrix and data subsets

If I solve a regularized linear least squares problem, $$ \min_x || y - A x ||^2 + x^T \Lambda x $$ then the solution is $$ x^{est} = (A^T A + \Lambda)^{-1} A^T y $$ Now the resolution matrix is the ...
2
votes
0answers
13 views

Robust regression for autocorrelation and heteroskedasticity - coefficients do not change, only standard errors change?

When using Newey-West robust standard errors to deal with heteroskedasticity and autocorrelation: http://support.sas.com/kb/40/098.html is it correct to state that the coefficients are not different ...
1
vote
1answer
34 views

VAR MODEL: Error in solve.default(Sigma) : system is computationally singular: reciprocal condition number

I am using R vars package to implement VAR model in a multivariate time series model. I tried to run: VAR(foo_ts, p = 6) but I ...
1
vote
1answer
33 views

Determin the determination coefficient R square

I have the following question and I couldn't figure out how to solve it. The given is the following: The model is: Yi = B0 + B1 Xi + Ei I have the following dependent variables 2, 4 and 8 (3 ...
1
vote
0answers
136 views

What’s the relationship between the variables in $E[(Y-(b_0+b_1X))^2]$ and $\frac{1}{n}\sum_{i=1}^{n}(y_i-(b_0+b_i x_i))^2$ in linear regression?

From Section 1.2 and 1.3 of these notes, for some r.v. $X$ and $Y$, if $MSE(b_0, b_1) = E[(Y-(b_0+b_1X))^2]$, it was found that $\beta_1 = \frac{Cov[X,Y]}{Var[X]}$ and $\beta_0 = E[Y]-\beta_1E[X]$ are ...
0
votes
0answers
15 views

Regression as function of another variable

I'm trying to run a simple OLS where I have housing prices (in log prices) as the dependent variable and surface area (in log surface area), wall type, roof type and other characteristics as ...
1
vote
1answer
24 views

Question regarding cointegration and superconsistency

I am reading this PDF: https://warwick.ac.uk/fac/soc/economics/staff/gboero/personal/hand2_cointeg.pdf where on pages 4 and 5 it says that if the residuals are stationary, the OLS regression is ...
2
votes
0answers
22 views

How do I properly include systematic uncertainty of x and y values correctly into fitting parameters (y=ax+b)?

I am doing a simple experiment that involves measuring the resistance of a wire. To do this, we measure the voltage across a wire as we increase the the current going through it with two Fluke Digital ...
2
votes
1answer
38 views

can zero covariance and zero expectation imply zero conditional expectation?

$x$ and $\epsilon$ are two random variables. If $Cov(x, \epsilon)=0$ and $E[\epsilon]=0$, can that lead to $E[\epsilon|x]=0?$
0
votes
0answers
3 views

How to derivate the sd(resid) of Least Squares Method for Estimating Vasicek Model Parameter

So, I learned about estimating Vasicek Model parameter from https://www.statisticshowto.datasciencecentral.com/wp-content/uploads/2016/01/Calibrating-the-Ornstein.pdf I tried to derive the $sd(\...
0
votes
1answer
16 views

Apply same regression model on multiple time points

i have a vector of time series data (1020x30; an index of neural activation measured at 1020 timepoints in 30 participants). I also have assessed 5 covariates of interest at a group level (eg age). I ...
0
votes
0answers
12 views

What to use: linear glm wls etc?

This is probably a basic enough question. What I want to achieve is a regression analysis of lapse rates on savings type policies. Say in one year person A withdraws 10 out of 100, and person B ...
1
vote
0answers
30 views

Why do robust linear models give smaller standard errors?

I've always read and been told that for heteroskedastic errors a normal OLS fit will generate standard errors that are too small, leading to a false degree of confidence in coefficient estimates. ...
0
votes
0answers
33 views

Should I drop variable highly correlated with independent variable? (Causal inference)

I estimate OLS: Y = c + b1*x1 + ....+ bn*xn +err corr (Y, x1) = 0.8, Corr(x1,err)>0 Should I drop variable x1? What kind of biases would I have in both cases: with and without x1.
1
vote
0answers
33 views

Regression changing of dependent variable [duplicate]

In regression model with random regressors $$y = a + bx + e$$ can I change the equation to $$x = (-a/b) + (1/b)y + (-1/b)e$$ and consistently estimate $(1/b)$ with OLS?
0
votes
0answers
31 views

OLS - Non stationary variables but stationary residuals - is this OK or not?

I am running an OLS on which the dependent variable (Y) and the independent variables (X1, X2, X3, ...) are non-stationary. But the residuals are found to be stationary. Does this mean my regression ...
1
vote
1answer
45 views

Is the linear combination of least square estimators in linear regression normally distributed? [duplicate]

I am reading Mathematical Statistics with Applications by Wackerly et al. (7th edition). In Chapter 11, the book discusses linear models and least squares, specifically $Y = \beta_0+\beta_1x+\epsilon$ ...
0
votes
0answers
27 views

Issue about confidence interval on OLS intercept

Let us assume this simple linear model: $Y|X=\beta_0+\beta_1X+\epsilon $ where $X \sim N(\mu,\sigma^2)$ and $\epsilon \sim N(0,\sigma_{\epsilon}^2)$ Suppose also that $X$ and $\epsilon$ have all ...
1
vote
0answers
14 views

Analysis of varying scenarios in least-squares regression?

First question: Is there a name for the type of analysis that is described below? (the second question is italicized in sentence below). I have a dependent variable $y$ that is related to the ...
3
votes
2answers
96 views

Distribution of $\hat{y}=Hy$

I wish to find the distribution of $\hat{y}=Hy$ where $H$ is the hat matrix $X(X'X)^{-1}X'$ in which a dash represents the transpose. Also, $\epsilon$ is $N(0, \sigma^2)$ distributed. Thanks
2
votes
2answers
75 views

Distinguishing Bad Leverage Points from Vertical Outliers

Going through Regression class notes (written mostly following Kutner`s book, I believe), there was a brief display of how, in some cases, robust residual plots (such as standardized LTS residuals), ...
1
vote
0answers
17 views

How to interpret standardized coefficients for an interaction effect between continuous and categorical variables?

It is conventional, in some circles, to standardize all continuous variables before conducting OLS regression. It is argued that this actions makes it possible to rank the effects observed, and hence ...
2
votes
2answers
63 views

What does the true level of significance mean in data mining?

There is a formula α*=1-(1-α)^c/k, a* - true level of significance. a - nominal level of significance. c - the number of candidate regressors. k - the number of finally selected regressors. I ...
0
votes
1answer
40 views

How do you prove that OLS Estimator must pass through mean of X and Y [duplicate]

This is regarding the simple case of y=mx+b. It's my understanding that the OLS estimator must necessarily pass through the mean of X and Y. How do you prove that this is always so?
0
votes
0answers
19 views

How can the endogeneity occur if the assumption E(Y|X)=0 assumption is satisfied?

According to the assumption of OLS, x and error term is expected to be independent. However, I am wondering given the E(Y|X)=0 assumption is satisfied, how can x and error term be correlated? Suppose ...
3
votes
1answer
166 views

Expectation in Linear Regression with Least Squares

We know that the best predictor of $\beta$ using the least squares criteria for linear regression is $\hat{\beta} = (X^TX)^{-1}X^Ty$ and I can derive this equation by minimizing the squared error in ...
18
votes
5answers
1k views

What regression/estimation is not a MLE?

I just rigorously learned that OLS is a special case of MLE. It surprises me because the popular and "reliable" sources such as researchgate and this do not mention this most important connection ...
0
votes
0answers
29 views

Why can a biased estimate still be statistically significant?

For example, I conduct an OLS regression and a regressor turns out to be statistically significant. When I conduct the same regression but with a GMM to account for serial correlation - I get a ...
0
votes
0answers
37 views
0
votes
0answers
33 views

weighted linear regression and outliers

I am fitting a regression model with weights because without weights I have heterocedasticity. Suposse $\epsilon \sim N(0,x_i \sigma^2)$ Then check the weights through aux model Model $Y=X \beta + \...
0
votes
0answers
10 views

hyp test: constant return to scale with Squared and interaction terms

I want to test if constant return to scale applies to my production function (modelA): log(y) = log(x1) + log(x2) + log(x1)^2 + log(x1)*log(x2) I have tried the following in r, but iam not sure if ...
0
votes
0answers
11 views

Interpreting a Dependent Variable that is Standard Deviation of Regression Residuals

I have a regression (OLS) output and was hoping to get some suggestions on how I might be able to interpret the coefficient using Stata or SAS. The dependent variable is ‘standard deviation of ...
1
vote
1answer
28 views

How can I randomly draw a set of parameters from a regression given a model fit and covariance matrix?

I have built a GLM and have the fitted parameters and covariance matrix. I'd like to generate a set of random beta-parameters using the covariance matrix to estimate what the confidence bounds and ...
0
votes
0answers
48 views

Interpreting patterns in the residual plot from OLS regression

Below are some residual plots from an OLS regression. The dependent variable is quality of life in patients measured on the 0-1 scale and independent variables are a mix of continuous and categorical ...
0
votes
1answer
22 views

Jacobian for function including cubic spline

I am trying to fit a measured spectrum with a linear combination of end-member spectra which are approximated by cubic spline functions ($f_1$ and $f_2$). I also need to incorporate terms that account ...
0
votes
0answers
12 views

Why does the marginal effects plot of OLS appear non-linear despite OK diagnostic plots?

the title says it all. I am plotting marginal effects with ggpredict in R and all of the lines exhibit some curvature. The diagnostic plots of the model look fine and I have log-transformed the ...
0
votes
1answer
37 views

OLS, IV applied to basic macro model

I am preparing for my final in Econometrics but I am confused over a new problem I encountered. I think I have solved it but I am unsure whether I am not making any gross mistakes. This is the study ...
0
votes
0answers
19 views

OLS Regression Project - Halloween Sales Cannibalization

I have marked up code in R to loop through a bunch of customer data with the goal of measuring the cannibalization between items selling in the stores. I feel really good about my R, but I would ...

1 2 3 4 5 37