Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
1answer
15 views
0
votes
0answers
9 views

Recursive least squares with forgetting factor - parameter covariance

The recursive least-squares algorithm equipped with forgetting factor is summarized as \begin{array}{l} \hat \theta \left( t \right) = \hat \theta \left( {t - 1} \right) + L\left( t \right)\left[ {y\...
0
votes
0answers
29 views

What could be the reason for a coefficient's change in magnitute and/or sign for identical specifications of OLS and Random Intercept Models?

What could be the reason(s) for a coefficient's change in magnitute and/or sign for identical specifications of OLS and Random Intercept Models? Further, does the Random Intercept Model, controlling ...
0
votes
0answers
8 views

How to deal with model misspecification - linktest

I am running an OLS model to examine to what degree is the physical quality of life (SF-12 scale) is associated with depression (HADS scale), adjusting by age and the presence of an acute illness (yes/...
3
votes
2answers
62 views

Confusion about when to use least-squares regression analysis

I am going through an article titled On the misuse of regression in earth science. On page 65, the author say as follows about the least-squares method. It is usual to require that the ...
1
vote
1answer
81 views

Too many variables and multicollinearity in OLS regression

After reading material related to my topic, I understood that multicollinearity among predictors would result in singular matrix $X'X$, and that leads to noninvertible matrix. Thus, the solution will ...
2
votes
2answers
41 views

Calculate interaction sum of squares by hand

I'm trying to manually calculate the sums of squares (SSs) in a two-way ANOVA in Python. However, I cannot seem to get the correct result for the interaction SS. I'm using the ...
2
votes
1answer
11 views

Is OLS Unbiased on Count Data in the Positive, Real Domain?

On the domain of positive real number is OLS on count data a consistent estimator of coefficients? I am trying to understand if the estimator is inefficient or produces biased SEs but is nevertheless ...
4
votes
0answers
60 views

LASSO with $\lambda = 0$ and OLS produce different results in R glmnet [migrated]

I expect LASSO with no penalization ($\lambda=0$) to yield the same (or very similar) coefficient estimates as an OLS fit. However, I get different coefficient estimates in R putting the same data (x,...
1
vote
2answers
55 views

Arima and lm not giving same coefficients in R

I'm fitting an arima(1,0,0) model using the forecast package in R on the usconsumption ...
0
votes
0answers
22 views

Best fitting linear transformation (not quite regression)

This is related to a previous question: Doing a "linear regression" with 2d points under a linear transformation Actually it is the 1D version of that question. This problem is similar to ...
4
votes
3answers
112 views

In linear regression, is there any meaning for the term $X^Ty$?

Recently, I was wondering about this question. In a standard linear regression problem ($y=X\beta$ and we solve for $\beta$), the solution is $\beta = X^{-1}y$ when $X$ is square and invertible, and $...
0
votes
0answers
19 views

Understanding the Matrix Operations for computing Leverage

I'm learning about OLS regressions and I want to learn how the matrix operations that yield the hat matrix from the design matrix connect to my understanding from the non-matrix equation. I think ...
5
votes
2answers
110 views

Using HAC standard errors although there might be no autocorrelation

I'm running a couple of regressions and, as I wanted to be on the safe side, decided to use HAC (heteroskedasticity & autocorrelation consistent) standard errors throughout. There might be a few ...
4
votes
2answers
130 views

Residuals analysis: interpretation of a scatter plot

I have problems with the interpretation of a scatter plot in a multiple linear regression (OLS method). I have posted an image below of the scatter plot of the standardized residuals vs the predicted ...
5
votes
1answer
130 views

Is it possible to compute RMSE iteratively?

I am working on continuous evaluation of a regression model on streaming data from sensors. I think that Mean Absolute Error (MAE) can be found out iteratively similar to this link for averaging. $$ ...
0
votes
0answers
13 views

Using nlsLM() for importance sampling for pricing european optoins

I am trying to implement a method for finding an optimal drift in pricing European options with importance sampling using this article. The article in pages 489-490 suggests to use Levenberg-...
0
votes
1answer
20 views

Determining two most similar objects based on multiple variables

I am trying to determine the two most similar years based on a number of environmentally variables. For example, I would like to choose the most similar year to 2015, from the set 1989 to 2016, based ...
1
vote
0answers
54 views

MLE of heteroscedastic model

I'm doing some practice questions for an upcoming exam and am unsure whether I've understood the problem correctly. Can anyone confirm what I've done or point out where I've gone wrong? My final ...
0
votes
1answer
39 views

Ways to approximate multiple samples of same function in R

Example dataset (simplified): ...
1
vote
0answers
64 views

Reducing variable candidates for multivariate regression step by step

I have a set of possible candidates that I want to use in a multivariate regression. I am trying to reduce this set by the following procedure (using Stata): Step 1: univariate regression (if ...
3
votes
1answer
139 views

Things that I am not sure about “LASSO” regression method

I have read the chapters that are related to "LASSO" regression in: The elements of statistical learning (Tibshirani et al.) Statistical Learning with Sparsity: The Lasso and Generalizations. (...
1
vote
0answers
19 views

Why coefficients are different when pooled ols and random effect applied?

Does anybody know why the coefficients of an equation which is estimated by Random effect estimators and Pooled OLS is different from each other?
0
votes
0answers
22 views

VAR and coefficients with OLS

I have a model like $Y_{it}=\beta_{0it} +\beta_1x_{1it}+\beta_2x_{2it}+\beta_3x_{3t}+\beta_4x_{4t}+e_{it}$ I have the data for all the variables and I know that $x_3$ and $x_4$ follow an AR(1) process....
1
vote
2answers
88 views

Analytical solution of a simple regression with fixed intercept

I would like to know how to find out the analytical solution of a simple linear regression with fixed intercept = 0: $$ s = e^{-ht}$$ $$ y = -ln(s) = h\cdot t$$ Here ist the background: I have ...
1
vote
1answer
36 views

Standard error of residuals v.s. standard error of regression

We know that in simple linear regression the variance of the regression error, $\sigma^2$, is estimated by $\frac {\sum_{i=1}^{n} (y_i - \hat y)^2} {n-2}$, i.e., the Mean Squared Error of the errors. ...
0
votes
0answers
16 views

interaction continuous*dummy: is it possible to treat the continuous variable as the moderator?

This is my first post here so please be kind to me! :-) For my thesis my aim is to run a moderation analysis between organizational commitment and training. My dependent variable is job satisfaction. ...
0
votes
0answers
8 views

How should I exclude results using standard error info - Levenberg-Marquardt

I have a 3D data map of estimated physiological values in the brain. i.e., 1 value for each 'voxel'. These values are that of a parameter resulting from a non-linear curve fit, using the Levenberg-...
10
votes
1answer
187 views

Simple linear regression, p-values and the AIC

I realise this topic has come up a number of times before e.g. here, but I'm still unsure how best to interpret my regression output. I have a very simple dataset, consisting of a column of x values ...
3
votes
1answer
36 views

What does strict exogeneity condition of OLS really mean?

In Hayashi's Econometrics, it is stated that one of the assumption of classical OLS is: $$\mathbb{E}(\epsilon_i\lvert\mathbf{x_1}, \mathbf{x_2}, \ldots, \mathbf{x_n}) = 0 \text{, for } i=1, \ldots, n. ...
8
votes
1answer
134 views

Weighted least square weights definition: R lm function vs. $\mathbf W \mathbf A\mathbf x=\mathbf W \mathbf b$

Could anyone tell me why I am getting different results from R weighted least squares and manual solution by matrix operation? Specifically, I am trying to ...
1
vote
2answers
89 views

What is parameter identification in the context of OLS?

Can someone explain what identification means in the context of an OLS model? I have a fair grasp of the derivation using either the method of moments or by minimizing the squares, but am failing to ...
1
vote
1answer
42 views

How to show the least square estimator of $b$ has the minimum variance in the class $\sum a_iy_i$

Consider the regression model: $$ y_i=bx_i+e_i,1\leq i\leq n.$$ where $x_i$'s are fixed non-zero real numbers and $e_i$'s are independent random variables with mean zero and equal variance. $(a)$...
0
votes
0answers
13 views

Random sampling as a Gauss-Markov assumption [duplicate]

In Wooldridge's Introductory Econometrics it is stated that random sampling is a Gauss-Markov assumption. As such it is a necessary condition for the unbiasedness of OLS estimators. While this can ...
2
votes
2answers
267 views

What are the consequences of “copying” a data set for OLS?

Suppose I have a random sample $\lbrace X_i, Y_i\rbrace_{i=1}^n$. Assume this sample is such that the Gauss-Markov assumptions are satisfied such that I can construct an OLS estimator where $$\hat{\...
0
votes
0answers
22 views

How to interpret non significant F-test for main effect (genotype), but significant differences in the means of the genotypes?

I am analyzing an experimental data set based on the trait values under drought condition. The experiment was carried out on three separate drought environments. First, I have done a single ...
13
votes
4answers
767 views

Why is Ordinary Least Squares performing better than Poisson regression?

I'm trying to fit a regression to explain the number of homicides in each district of a city. Although I know that my data follows a Poisson distribution, I tried to fit an OLS like this: $log(y+1) = ...
0
votes
0answers
6 views

Comparison between default, robust unclustered and cluster robust standard errors

I regress from my data lnhr on lnwg, first with the default OLS (POLSiid), second with the robust unclustered option (POLShet), third with the cluster robust option (POLSpanel). I understand with the ...
1
vote
1answer
46 views

Manually compute the regression coefficients of a multiple regression model with numerical and categorical variables

I am going to explain my question using a reproducibile toy example. I would like to regress a numerical variable using a multiple regression model with either numerical and categorical variables. I ...
0
votes
0answers
16 views

Relationship Between Correlations and Contour Plots for OLS

In the paper, "Simultaneous Regression Shrinkage, Variable Selection and Supervised Clustering of Predictors with OSCAR" (Bondell, Reich), the authors state: "As the contours are in terms of $X^TX$ ...
0
votes
0answers
17 views

Auto-correlation Assumption

I am testing the auto-correlation assumption of OLS. My study is conducted on the most active companies on the Egyptian stock exchange over a period of 5 years. Not all companies included in the ...
1
vote
1answer
19 views

Mixed effect - Pooled ols Different results interpretation

I have a question. I have collected data regarding the performance of companies and their board structure. I want to find the effect of the Board structure upon the performance and I am using pooled ...
0
votes
1answer
26 views

Simplification in proof of OLS inconsistency

I'm a little confused right now regarding the LLN "jump" from probability limits to expectations and variances/covariances: Say we have a linear regression model of the form with $S$ observations: $$...
0
votes
0answers
35 views

OLS, phenomenon { alpha = - mean(beta_2*x_orig)} : coincidence?

as suggested in the title, when with some data I perform this model: y ~ alpha + beta_1 * x_1 + beta_2 * (x_1)^2 + error term with OLS I SOMETIMES fall into the ...
1
vote
0answers
14 views

How to interpret and constrain the 'bias' from an OLS multiple regression?

I'm trying to solve a linear system with OLS and understand how the output coefficients deviate from the input values of mock data. The basic ideas are as follows. For the linear system ...
0
votes
0answers
16 views

Quadratic fitting raw time series data vs linear fitting its derivative

I have time series data $f_i(t_i)$. Is there a difference between the following two strategies: Fitting $\hat{f}(t)=at^2+bt+c$ to the original data Fitting $\hat{g}(t)=2at+b$ to the time derivative ...
4
votes
1answer
36 views

$Y=\epsilon$ in GLM?

In general linear model $$Y=X\beta +\epsilon $$ the LSE for $\beta$ is $$\hat \beta=(X^TX)^{-1}X^TY$$ and so $$\hat Y=X\hat \beta=X(X^TX)^{-1}X^TY=HY$$ where $H=X(X^TX)^{-1}X^T$. Then the ...
1
vote
1answer
53 views

Can I apply OLS (multiple regression) to panel data to identify significant variables?

I have panel data for a 5-year period and want to explore the determinants of car prices (number of doors, house power, etc.). Is it appropriate to use OLS or multiple regression to explore the ...
1
vote
2answers
66 views

How to compare results from two regressions?

We have performed two linear regressions (OLS), one with data from 2009 and one with data from 2014. All the variables are the same, both the dependent and the six independent variables. The sample ...
1
vote
0answers
18 views

What are some motivations for using nonnegative least squares?

I'm having a hard time understanding the reasoning behind it. Imagining the case of a single independent variable, if the correlation between it and the dependent is very negative, a nonlinear least ...