Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
1answer
27 views

How does simple linear regression minimize least squares [on hold]

Intuitively, how does simple linear regression minimize the squared residuals?
4
votes
1answer
67 views

Proof that the coefficients in an OLS model follow a t-distribution with (n-k) degrees of freedom

Background Suppose we have an Ordinary Least Squares model where we have $k$ coefficients in our regression model, $$\mathbf{y}=\mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}$$ where $\mathbf{\beta}$ ...
1
vote
0answers
18 views

Least Squares Regression - Error [duplicate]

In standard least squares regression, we find constants $\beta_1$ and $\beta_2$ such that the square of the average error, $\epsilon = y_i - (\beta_1 + \beta_2x_i)$, is minimized, and so the 'line of ...
0
votes
0answers
29 views

Help in calculating OLS estimators under certain constraints/modifications

I am new to these forums and to econometrics as a whole. I was hoping someone would be able to give me a nudge in the right direction with this problem. I've done extensive research both online and in ...
2
votes
0answers
49 views

OLS standard error that corrects for autocorrelation but not heteroskedasticity

Question: By mapping the OLS regression into the GMM framework, write the formula for the standard error of the OLS regression coefficients that corrects for autocorrelation but not ...
0
votes
0answers
15 views

analytic problem on the R^2 and LSE [on hold]

I'm stuck in #7 and #9 for hours. how can I solve these problems?
2
votes
2answers
75 views

Different output for R lm() and python statsmodel OLS for linear regression

I'm exploring linear regressions in R and Python, and usually get the same results but this is an instance I do not. I added the sum of Agriculture and ...
0
votes
1answer
50 views

What am I missing in basic OLS? [closed]

I must first admit that I haven't done stats in a loooong time, but I need to do some rudimentary analyses. My issue is that my findings just don't "look right," so I want to see if anyone can spot ...
0
votes
0answers
17 views

multidimensional time series nonlinear parameter estimation

I am trying to fit time series data for performing parameter estimation of a nonlinear multidimensional dynamical model (grey-box). At the moment I'm successfully using MATLAB's ...
1
vote
1answer
89 views

OLS vs IV estimates - Sign and Significance

Assume I have an equation with 1 endogenous variable, and many other exogenous variables. Also assume I have 2 valid instruments for the endogenous variable for IV estimation. If I were to estimate ...
1
vote
0answers
19 views

Orthogonal projection of a vector which is already orthogonal to part of the basis

Context This question emerged from trying to solve problem 5.1. of Wooldridge, Econometric Analysis of Cross Section and Panel Data. The problem asks to show the equivalence of the estimators ...
0
votes
1answer
79 views

How can you prove that the naive estimator is less efficient than the OLS estimator

The "naive estimator" is an estimate of the slope obtained by joining the first and last observations and dividing the increase in the height by the horizontal distance between them. Given that the ...
0
votes
0answers
10 views

Arbitrary least squares fitting with an initial model

I have a problem that seems like a straightforward implementation of linear least-squares, but I can't quite figure out how it will work. I have a (noisy) dataset $f(x,y)$ where $f$ is is non-linear ...
1
vote
1answer
46 views

Regressing a differenced variable on a lagged variable. How can I fix the error in R?

I have a time series (std) of 324 observations with no missing values, starting from January 1987 and ending in December 2013. I want to regress via OLS the one in the question. In R, the code: ...
0
votes
1answer
18 views

Why do we use fixed effects and controls at the same time?

When we run a regression model (say OLS for simplicity) of y ~ x, we might have to use several control variables say z1 and z2. Now our model is y ~ x + z1 + z2, we may believe that z1 and z2 are not ...
1
vote
0answers
27 views

How much does the value of a car depreciate due to time (only), and mileage (only)?

I have a database of second hand cars. It contains (among other things) the asking price, the mileage, the fuel consumption and the year it was built. I would like to know how much the value ...
1
vote
1answer
42 views

Regression model result interpretation

I am trying to learn few things about generalized least square regression modeling. Here are the models that I am using., X1 ~ Y1 + Y2, X2 ~ Y1 + Y4, All predictor variables, Y1, Y2 and Y4 are ...
3
votes
1answer
36 views

Prediction error in least squares with a linear model

In the classical linear model with $$Y=X\beta +\epsilon,$$ where $Y \in \mathbb{R}^n$ is the observation, $X\in \mathbb{R}^{n\times p}$ is the known covariates, $\beta \in \mathbb{R}^p$ is the ...
1
vote
2answers
64 views

Does least squares regression imply normality of errors?

For the linear model $$y_i=\beta_0 +\sum_{k=1}^{n}\beta_k x_{ik} + \epsilon_i$$ the parameter estimates are the same for the maximum likelihood method and the least square method (minimizing ...
1
vote
2answers
53 views

What is the effect of having skewed dependent variable on scatterplot result

The histogram of my dependent variable is as following: I draw the scatter plot of my dependent variable and independent variable, and the result is as following picture? I am wondering if the ...
3
votes
2answers
49 views

what is difference between ordinary least squares and residuals

I think the ordinary least squares are the sum of the vertical distance between the observed data to the model line(regression). And residual is calculated by add up all vertical distance between each ...
5
votes
1answer
87 views

Omitted variable bias in logistic regression vs. omitted variable bias in ordinary least squares regression

I have a question about omitted variable bias in logistic and linear regression. Say I omit some variables from a linear regression model. Pretend that those omitted variables are uncorrelated with ...
2
votes
1answer
40 views

In OLS is it methodologically correct to use the variance of a variable as an explanatory variable?

Are some OLS assumptions not satisfied if I use the variance of a variable as a proxy of uncertainty in a regression? For instance, would it be methodologically correct if I use moving averages of ...
2
votes
2answers
111 views

Is there a result showing a relation between the size of the residuals and the correlation coefficient?

Consider an OLS regression between two variables. Is there any result which relates the size of the residuals (measured, perhaps, by the sum of the squares) to the Pearson correlation coefficient of ...
0
votes
0answers
22 views

Does this kind of overlap between in-sample data and forecast cause inflated $R^2$?

I am using a simple UIP model to forecast exchange rates using interest rates with a twelve month horizon. The equation I use is: $E(t+12) - E(t) = α + β(I*(t) - I(t))$. I apply OLS linear regression ...
0
votes
0answers
25 views

Is there a way to model the error term in a linear regression with R?

I'm tryin to estimate a pretty basic regression. I have a dataset containing x and y and would like to esimate the following ...
0
votes
0answers
17 views

Proxy variable approach to deal with endogeneity due to reverse causality

Is this a feasible thing to do? I've mostly seen the proxy variable approach used for dealing with omitted variable bias and measurement error. But how would one go about using it for simultaneity? ...
2
votes
2answers
89 views

Regression to solve system of a system of linear equations?

The question I am going to ask probably has a straight-forward answer, I just don't know how to frame my question in a way that would be 'normal.' I have a set of a set of linear equations that I ...
1
vote
1answer
29 views

Time dummies in panel data — absorbing effects?

I am conducting a data analysis. I have a panel with individual firms with firm-specific and macroeconomic variables. I would like to run an OLS regression adjusted for firm clustering effects and ...
2
votes
1answer
34 views

Polynomial regression rules

With OLS regression the errors must be normally distributed and be homoscedastic. Does these rules apply to polynomial regression as well?
0
votes
0answers
15 views

Non Significant MANOVA but significant OLS regression

I have data which is showing a $p$-value of $0.15$ significance value within a MANOVA (including a significant planned contrast). However, when I dummy code and put it into an OLS regression, the ...
1
vote
1answer
27 views

Variance of a variable constructed from parameter estimates and predicted values of a linear regression

My data set is an annual panel data set on individual income, the year of unemployment and a number of demographic variables. I run an OLS regression of the form $y_{it} = \sum _{j=1} ^n D_{j,it} ...
4
votes
1answer
177 views

Total least squares curve fit problem

I am trying to fit a quadratic curve across a scatter plot of two variables. Since both variables are noisy I cannot use an ordinary least square regression (OLS) and I would like to have a ...
3
votes
1answer
46 views

If not minimizing SSEs, what am I doing?

I've got a problem that can be summarized as a linear regression. Thus it takes the following form: $$ Y=X \beta +\epsilon $$ where $Y$ and $\epsilon$ are vectors of size $N\times1$, $X$ is a matrix ...
1
vote
1answer
56 views

Endogenous variable and statistical significance in OLS?

I was wondering what the following OLS scenario would imply: a variable is endogenous (i.e. correlated with the error term) yet is statistically significant. Alternatively, what if in, once again ...
1
vote
1answer
89 views

Is this an example of Pooled OLS on Panel Data?

I am looking at a study that analyzes the effect of an infrastructure index on infant mortality and child mortality rates. The database has (asset) quintile level data for 47 different countries (from ...
1
vote
0answers
47 views

choosing $β_0$ and $β_1$ to minimize the residual sum of squares

I'm reading a book called An Introduction to Statistical Learning: with Applications in R, and I have a question in regards to the material inside. I understand that we can find the residual sum of ...
1
vote
1answer
47 views

Using OLS for Model Selection and Prediction - Heteroscedasticity Issue

I am new to regression and having problem in solving Heteroscedasticity in OLS. Have done lots of homework and test before seeking your advice. Sharing the background and what I have done to solve the ...
1
vote
2answers
39 views

Why representation of AR process comes up in estimation

Let ${X_t}$, $t=...-2,-1,0,1,2...$ be a stochastic process that satisfies: $X_t=\rho X_{t-1}+\varepsilon_t$ with $|\rho|<1$ and $\varepsilon_t$ is a white noise. In that case, we also know that ...
2
votes
0answers
24 views

How to approximate error on Chi-Squared Fit when bin counts are zero

I am using a galaxy image simulator that provides a 2D histogram that has the number of photons per pixel (bin) $N$. I am currently using a least-squares residual: $\sum_{bin}(f_{data}-f_{model})^2$ ...
5
votes
3answers
153 views

How to interpret coefficients of $x$ and $x^2$ in same regression

If I have the below functional form for an OLS regression, how do I interpret the $x$ and $x^2$? I cannot interpret them separately, correct? Do I interpret them as a summation of the two ...
0
votes
1answer
72 views

Orthogonalizing predictors for least squares estimation

I know that orthogonalization in LS is to avoid inverting X'X. The idea behind it is to find variables Z that are orthogonal to each other. Although the process to find those is clear to me, I don't ...
1
vote
2answers
124 views

Hat matrix,$H=X(X^{\prime}X )^{-1}X^{\prime}$

What is the importance of hat matrix, $H=X(X^{\prime}X )^{-1}X^{\prime}$ in regression analysis? Is it only for easier calculation ?
3
votes
1answer
100 views

comparing OLS, ridge and lasso

I am trying to compare OLR, ridge and lasso in my situation. I could calculate SE for OLR and lasso but not for ridge. The following is Prostrate data from ...
2
votes
1answer
39 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
2
votes
2answers
59 views

Steps in making a Global Regression Model

I saw a journal article [1] saying that he constructed the following S-curve model : $$y=\exp\left({\beta_1 + \frac{\beta_2}{x}}\right) + \mathrm{residual}$$ The topic was about a global regression ...
1
vote
0answers
14 views

How to select an error/weight for two-dimensional binned data?

Beginning with noisy data vectors $\mathbf{x}$ and $\mathbf{y}$, I have binned the data to vectors $\mathbf{x}_b$ and $\mathbf{y}_b$ of length $N_b$ with fixed linear ($\mathbf{x}_b^i - ...
1
vote
1answer
98 views

Generalized Least Squares vs Ordinary Least Squares under a special case

This question regards the problem of Generalized Least Squares. Vectors and matrices will be denoted in bold. Premises. Let $N,K$ be given integers, with $K \gg N > 1$. The transpose of matrix ...
2
votes
1answer
66 views

N-sigma curves for a non-linear least square curve fit

I'm using python's scipy.optimize.curve_fit routine (which uses a non-linear least squares) to fit an exponential function of the form: ...
0
votes
0answers
49 views

Constrained Least Squares/Fixing Data

So I have maybe an unusual question. I have some simulation data that I use as input into a theoretical model. The problem is that the simulated data is noisy and this causes divergences when I try ...