Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

0
votes
0answers
14 views

How to check if i have strong linear relationship between dependent variable and independent variables in ols?

I want compare the out of sample prediction from an ols model and a regression tree. I read that ols outperforms regression tree if the relationship between the dependent variable and independent ...
0
votes
0answers
7 views

Consistent estimation and valid inference when performing regressions on data with differing levels of granularity

Imagine that a dataset has a combination of variables of differing levels of granularity (e.g. an international sample of firms containing both firm-level and country level information). There are $K$ ...
1
vote
0answers
9 views

OLS - Predeterminedness and moment condition

I'm having trouble validating if following procedure to test for predeterminedness is plausible. Given the linear model: $y_t=\beta_1+\beta_2x_{1t}+\beta_3x_{2t}+\epsilon_t$ Having $x_{2t} = y_{t+1|...
0
votes
0answers
27 views

What to do if random forest still overfit after grid tuning?

I have a random forest and an ols regression. Both models i want use for an out of sample prediction. Before tuning the parameters of the random forest the default settings of the random forest yield ...
0
votes
1answer
29 views

Is quantile regression a special case of OLS?

Quantile regression is often advertised as a way of "predicting change in the dependent variable that is not the mean." It seems like one can do this with linear regression, however. Am I correct? ...
1
vote
0answers
28 views

Does bias in regression coefficients affect the prediction?

Goal is to create ols model for out of sample prediction for log(wages). Theory say I could have a sample selection bias. So I choose the heckit method to correct for it. The correction term lambda (...
1
vote
0answers
31 views

OLS as estimator

we've been given following question, but have some trouble getting started, can anyone help out? $\pi_{t}=\alpha_{1}+\alpha_{2} u_{t}+\alpha_{3} \pi_{t+1 | t}^{e}+\eta_{t}, \quad t=1,2, \ldots, T$ ...
1
vote
1answer
32 views

Interpretation of removed continuous variables in regression due to linear dependence

I have created a standard OLS regression model to estimate the House Price and one group of variables describe the age group percentage of population in a particular neighborhood (ranging 0 to 100). ...
0
votes
3answers
22 views

Can a prediction be better with insignificant coefficients than with significant coefficients?

I have two OLS models and want to do an out of sample prediction for wages on a test set. In the first model I excluded the insignificant coefficent. The second model has the insignificant coefficient....
2
votes
0answers
21 views

Different error weighting for positive and negative residuals for OLS?

For OLS-estimators in multivariate regression analysis, it logically doesn't matter whether an error is positive or negative. I was wondering if in some situations it might make sense to weight a ...
0
votes
0answers
10 views

Time Series Econometric

i want to ask about Time Series Regression. I have data from 5 variables, 4 of which have 34 series, while one data only has 23 series. would it be good if I use OLS Time Series for that data ?, or I ...
0
votes
1answer
10 views

Can i use an independent variable in % units in a probit regression

I want to do a probit regression and one explanatory variable is given in % units. Do i have to transform it in decimal units or can i use it in % units in my probit regression?
0
votes
1answer
15 views

A question about the Least Squares Estimation: what motivates its definition in the general case?

Let $Y_{1},Y_{2},\ldots,Y_{n}$ be independent random variables with expected values $\mu_{1},\mu_{2},\ldots,\mu_{n}$, respectively. Suppose that the $\mu_{i}$'s are functions of the parameter vector ...
0
votes
0answers
25 views

OLS basic doubt

In a multivariate OLS model : $ Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \epsilon$ My estimator for $\beta_1$ is given by which expression: $\hat \beta_1 = [X_1'X_1]^{-1} X_1'Y$ OR $\hat \...
0
votes
1answer
54 views

Multiple linear regression: am I interpreting the methodology right?

This is a follow-up question to 1 and 2. So we have the normal linear model \begin{align*} \textbf{Y} = \textbf{X}\beta + \epsilon \end{align*} where $\epsilon\sim\mathcal{N}(\textbf{0},\sigma^{2}\...
1
vote
1answer
38 views

regression basic doubt

If have a simple bivariate regression model: $ Y_i= x_i \beta + \epsilon_i $ where $i$ are the number of observations. How do I test for the hypothesis that the OLS coefficient $\beta$ does not ...
3
votes
1answer
51 views

Why do we need to determine the distribution of $\textbf{Y}$ in the multiple linear regression problem?

Once again, here I am. Given the multiple linear regression model \begin{align*} \textbf{Y} = \textbf{X}\beta + \epsilon \end{align*} where $\epsilon\sim\mathcal{N}(\textbf{0},\sigma^{2}\textbf{I})$ ...
1
vote
1answer
28 views

If the first order conditions of MLE and OLS are identical, is MLE as efficient as OLS i.e. are they both BLUE

If the first order conditions of MLE and OLS are identical is MLE as efficient as OLS? It seems that they should be equal in terms of efficiency however if OLS is the best linear unbiased estimator ...
0
votes
0answers
25 views

OLS regression with multiple dependent variables that are correlated with each other

Suppose I want to see the impact of an explanatory variable $X$ on two different dependent variables: $Y_1$ and $Y_2$. Suppose also that I find that $Y_1$ and $Y_2$ are correlated. Assuming that all ...
0
votes
1answer
28 views

Vector autoregressions [on hold]

Suppose running a VAR system featuring the central bank interest rate, inflation and GDP. Start by estimating each equation by OLS. How should you interpret the OLS residuals to these equations? My ...
0
votes
0answers
16 views

Interpretation of a log-level regression in its associated 'level' form

Is it conventional to interpret a least-squares regression with a log-transformed dependent variable (log-linear model) in its "level" form? In other words, running a model with the outcome in 'log ...
0
votes
0answers
41 views

Event studies: Panel OLS vs cross sectional OLS vs FE

I'm currently working on my bachelor's thesis, where I'm investigating the effect of surprise interest rate decisions on stock prices using an event study approach similar to Bernanke and Kuttner (...
0
votes
1answer
21 views

OLS - regression: how to interpret it? [duplicate]

I'm running an OLS and was wondering if the 'Estimate' in my SPSS output is the same as the beta coefficient in a linear regression? Are there specific assumptions required to run an OLS? I have age, ...
0
votes
0answers
22 views

Maximum likelihood and OLS estimation of ARDL model under nonstationarity

Consider the simple ARDL(1,1) model $y_t=\beta_0+\beta_1y_{t-1}+\beta_2x_{t}+\beta_3x_{t-1}+\epsilon_t$ If $y_t$ and $x_t$ are non-stationary can I fit the model with OLS? If not, is assuming that $\...
1
vote
0answers
26 views

Plotting Residuals vs Predicted Values

In textbooks, residual plots are described as have predicted (fitted) values on the x-axis, with the y-axis being the difference between the predicted and observed values. However, I'm having trouble ...
0
votes
0answers
12 views

Replicating partial least squares (NIPALS) results using ordinary least squares regression in Tensorflow?

I have multivariate variables that I want to regress to a single target label. For some reason, using partial least squares regression (projected to a single component) gives much better prediction ...
0
votes
0answers
13 views

Non-normal regression errors: consequences and solution

Doing an empirical econometrics project, I have managed to confuse myself on one of the basics and wonder if you could lead me on the right track. What I wonder about is the normality assumption of ...
0
votes
0answers
6 views

Regress in two variables against basis functions when one of them might be “sparse”

I'm tasked with the problem of fitting a series of functional forms $\{f_t(S,K)\}$ along the time axis. At each time $t=0,1,\cdots,T$, there are $N_t$ samples $$(F_{t,i}, S_{t,i}, K_{t,i}),\quad i=1,2,...
1
vote
1answer
17 views

Updating regression solutions for removing a regressor without the original dependent variable

Note: This question is analagous to the question I asked here except instead of adding a column, I am removing it. I am interested in a linear regression on the model; $Y= X\beta + \epsilon$ And I ...
2
votes
1answer
61 views

Updating regression solutions for a new regressor without the original dependent variable

Note: This question is analagous to the question I asked here except instead of a removing column, I am adding it. I am interested in a linear regression on the model; $Y= X\beta + \epsilon$ And I ...
0
votes
0answers
15 views

Statistical significance of co-efficient derived through several regressions for panel data

For the panel data that I have, I'm trying to run multiple one-variable regressions (y=ax+e) across time periods - i.e. one for each time period. Naturally, the a's that I find from each regression ...
0
votes
0answers
28 views

Testing interaction terms individually and simultaneously

I am currently testing two interaction terms individually in OLS regressions. Both interaction terms are significant (p < .5) when tested individually (i.e., two regression equations, one for each ...
0
votes
0answers
20 views

What is a computationally tractable way to infer statistical power in the context of OLS with robust clustered standard errors?

I have clustered data and am trying to scale up OLS with robust clustered standard errors (implemented in Python Statsmodel) as the default analytical framework going forward. It is important that I ...
0
votes
0answers
22 views

Finding inverse of $X'X$ in the case of two regressors [duplicate]

Variance of OLS etimator in matrix form look like this: $Var(\hat{\beta_j})=\sigma^2(X'X^{-1})$ I'm struggling to derive inverse matrix for the case with two independent variables. $X'X$ $=$ $\...
0
votes
0answers
24 views

MLE estimate for least squares if features have Gaussian noise

We have come across the problem of MLE estimate for least squares if errors are normally distributed, eg, $Y_i=\beta x_i+\epsilon$, where $\epsilon$ ~ $N(0,\sigma^2)$. The estimate for the above case ...
1
vote
0answers
8 views

Interaction in logit

I am estimating a model where interaction terms play a role, and I am wondering which specification I should use, and how to interpret the results. More specifically, I regress a binary variable, say ...
1
vote
1answer
31 views

OLS loss function 3-d surface plot

I was trying to plot the OLS loss function as a function of coefficients $\beta_0$, $\beta_1$. As far as I know it should be a convex function with one local minimum which is also a global minimum. I'...
0
votes
0answers
11 views

Direction of bias when changing from OLS to IV

If when using an instrumental variable it increases the size of the coefficient and changes the direction of the relationship compared to OLS --> what direction of bias does it suggest in the OLS ...
0
votes
0answers
18 views

Heteroskedasticity in linear probability models

I have a class on linear probability models. We want to estimate a model $y=\beta x$ where both $y$ and $x$ can be either $0$ or $1$, so that the conditional expectation function can be expressed in ...
1
vote
0answers
20 views

Fixed Effects model

I am trying to understand the fixed effects model. $ Y = D\alpha + X\beta + \epsilon$ Where $\alpha$ is the fixed time invariant effect. If we model this using Frisch Waugh lovell partitioning ...
1
vote
0answers
21 views

hypothesis testing doubt

I have the OLS regression model: $Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_3 + \beta_4X_4 + \epsilon$ I want to check the hypothesis: Ho : $\beta_2*\beta_3$ = 1 Will I use the Delta ...
0
votes
0answers
20 views

OLS Heteroskedasticity correction

I have a data and was trying to correct for heteroskedasticity (which is significant as per Breush Pagan test). However, after using the robust command in stata, my standard errors of almost all the ...
1
vote
1answer
29 views

Validity of the linearity assumption

I have to research unaided recall of commercials given a set of variables. So, I formulated the following model: $unaided = \beta_0 + \beta_1duration + \beta_2blocksize + \beta_3position + \beta_4 ...
0
votes
0answers
9 views

F test, Model selection test

How to compare which of the following two models is a better fit: M1 $y$ = $\alpha$$X$ + $\epsilon$ M2 $ln(y)$ = $\alpha$$X$ + $\epsilon$ Can we run a test statistic to compare the two models? ...
0
votes
0answers
11 views

How to test for sample selection bias in heckit estimation?

I'm doing a heckit two step estimation by hand where i first estimate the labor force participation, compute the inverse mills ratio by using the linear predictors of the model and add it as an ...
0
votes
1answer
16 views

Prove that the direction in Least Angle Regression makes equal angle with all predictors [closed]

Least Angle Regression iteralively adds predictors according to the procedure described here : Writing by hand first steps in Least Angle Regression (LARS) We note $A_{k}$ the active set of variables ...
0
votes
0answers
21 views

Singular values for a latent-factor model

Suppose we build a latent-factor model using alternating least squares (ALS) or stochastic gradient descent (SGD). Can we calculate weights for each latent factor, in a similar way to how the singular-...
2
votes
1answer
144 views

In the presence of heteroskedasticity, is quantile regression more appropiate than OLS?

..for understanding the relationship between a dependent and independent variables, given that quantile regression makes no assumptions about the distribution of the residual.
0
votes
0answers
26 views

How to interpret plot residuals vs fitted values?

I run a ols regression and want now check the linearity assumption. I found out that i have to plot the residuals vs the fitted values and if there is no non linear pattern the linearity assumption ...
0
votes
0answers
11 views

Proof for multicollinearity consequence

I came across this statment "Even extreme multicollinearity (so long as it is not perfect) does not violate OLS assumptions. OLS estimates are still unbiased and BLUE (Best Linear Unbiased Estimators)"...