Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
0answers
24 views

What happens if I square the variable in my log in OLS regression?

Say I have a model: ln y = B0 + B1(x1) + B2 ln(x2) + u and the B2 estimate I get is 0.5 If I change the model to be ln y = B0 + B1(x1) + B2 ln(x2^2) + u the estimate will change to 0.25, but why ...
0
votes
1answer
19 views

Robust OLS standard errors (Newey-West)

I am running a simple OLS regression with HAC adjustment (i.e. Heteroschedasticity and Autocorrelation adjustment) using the following function in hac() in matlab. ...
1
vote
1answer
26 views

Individual-specific error component in pooled OLS

I know that Pooled OLS is not efficient if there is existence of the individual-specific error component (one that doesn't vary over time) because the usual standard errors are incorrect and the tests ...
2
votes
1answer
6 views

Is the order of parameter estimates preserved from multiple simple regressions to one multivariable regression?

Assume I have $y$, $x_1$ and $x_2$. I regress $y\sim\alpha_0 + \alpha_1 x_1$, $y\sim\beta_0 + \beta_1 x_2$ and $y\sim\gamma_0 + \gamma_1 x_1 + \gamma_2 x_2$ using Ordinary Least Squares. Does ...
7
votes
1answer
97 views

finding out 2D transformation given a list of sample points

I have a bunch of 2D points that are rotated by $\theta$ and translated by $(\Delta_x, \Delta_y)$. I.e. $$ x'=x \cos(\theta)-y \sin(\theta)+\Delta_x \\ y'=x \sin(\theta)+y \cos(\theta)+\Delta_y $$ ...
0
votes
1answer
25 views

OLS parameter estimation of an expression?

During my research for a class, I came across a paper that said they estimated an equation using OLS. But the parameter they were estimating appeared to be an expression that looked like this (not the ...
1
vote
1answer
58 views
+100

Local polynomial regression: Why does the variance increase monotonically in the degree?

How can I show that the variance of local polynomial regression is increasing with the degree of the polynomial (Exercise 6.3 in Elements of Statistical Learning, second edition)? This question has ...
0
votes
0answers
12 views

Solve intercept coefficient Dummy Variable regression

Suppose we have the model $y=\beta_0+\beta_1 x_2+\beta_2x_3+e_i$ where $x_1, x_2, x_3$ are binary variables, taking on values 0 and 1, so for example, if $x_1=1, x_2=x_3=0$. Now we want to regress ...
1
vote
0answers
23 views

Linear regression with ordinal scaled variables

A friend of mine is working on a problem set and asked me an interesting question: if we have 2 variables, which are ordinal scaled how to compute a linear regression with those? $$x \in ...
0
votes
0answers
29 views

Recode ordinary scaled variables

Iam trying to make a linear regression in STATA with ordinary scaled(in dataset): dependant variable(0;10) independant variable(0;10) Does anyone know how can recode the variables into metric scaled ...
0
votes
0answers
8 views

Interpolating singular values

So I have the singular values associated with a data matrix and I would like to interpolate them and then find the maximum curvature of the interpolation in order to decide how many singular values to ...
0
votes
0answers
7 views

Multinomial Heckman Models

I'm interested in modeling left censored data with a tobit model using R. Mine is a two step, as I want to generally predict probablity, and then quantity, so I'm planning on using a heckman. The ...
0
votes
0answers
12 views

Interpreting standardized log coefficients in OLS

I used the log for my dependent variable as well as for some independent variables. Then I standardized all variables. Now I'm not sure how to interpret the coefficients. Are the log and non loged ...
0
votes
1answer
21 views

Java and R: Least Squares Coefficient Estimation - Start at time Zero?

This is the data set I have: vector <- c( -7.459981, 13.26651, 12.10128, 2.380662, 26.42393) Doing an estimation of the coefficient with a linear regression ...
0
votes
0answers
27 views

Standard error/deviation of the coefficients in OLS

In OLS, the variance of the regression coefficients are computed as $$ \mathrm{Var}(\hat{\beta}) = \sigma^2(\mathbf{X}^\mathrm{T}\mathbf{X})^{-1}. $$ Now, if I need to compute the standard ...
0
votes
0answers
12 views

Least squares (vs) and cost function

What is the sense behind to minimize the error of a hypothesis function with a cost function instead of the applying least squares method? I think you actually get the best approximation by using the ...
1
vote
0answers
16 views

log transformation decreased model fit?

I just wondered why logged income (independent variable) decreased my model fit for OLS regression. My income distribution is skewed to the right and I am trying to transform the data. I separately ...
0
votes
1answer
31 views

Least-square fit with uneven distribution of data

I'd like to perform least-squares fit to data which is unevenly distributed on the x-axis. For example, if I was to bin the data, it would be something like x = 0~5: 10 data points x = 5~10: 20 ...
1
vote
2answers
73 views

Does possible non-stationarity matter if the model is OLS?

I am working on an assignment where the current model is an OLS model that models the percent change in a variable, X, by regressing it against a bunch of economic variables such as unemployment rate, ...
0
votes
1answer
67 views

OLS assumptions

It is known that conducting post-estimation tests for OLS assumptions (Multicollinearity, heteroscedasticity, and endogeneity) is necessary. But is it statistically necessary to carry out these OLS ...
0
votes
0answers
30 views

Are there multiple ways to interpret the slope parameters in linear regression?

I am struggling to understand an interpretation of regression parameters presented in a paper comparing and contrasting OLS regression to quantile regression. The authors present an example linear ...
10
votes
2answers
252 views

What is the “partial” in partial least squares methods?

In partial least squares regression (PLSR) or partial least squares structural equation modelling (PLS-SEM), what does the term "partial" refer to?
1
vote
3answers
45 views

Why is it the case that when we try to fit an OLS model to a system with more variables than observations, that the residuals are zero?

I am trying to fit an OLS model to some data, where the number of variables $k$ is greater than the number of observations, $N$. In this case, it is obvious that we will have a infinite amount of ...
14
votes
4answers
885 views

Why do we usually choose to minimize the sum of square errors (SSE) when fitting a model?

The question is very simple: why, when we try to fit a model to our data, linear or non-linear, do we usually try to minimize the sum of the squares of errors to obtain our estimator for the model ...
2
votes
1answer
59 views

numerical difference between sum of squared residuals and likelihood

I previously asked a question that got labelled as duplicated because I did not explain it correctly. I should not have used the regression model as an example because I can see how, by using that as ...
0
votes
0answers
20 views

Least squares: Calculus to find residual minimizers?

Reading a section on simple regression in "An Introduction to Statistical Learning with Applications in R" I got a question on residual sum of squares minimization. Quoting from the book: ... ...
2
votes
1answer
77 views

Why is $E[u] = 0$ in OLS and what is the difference between the error and constant?

How is the constant term in a linear least squares regression different from the error term? Why does the error term have a normal distribution with mean "0" and sd $\sigma$? I mean I can see that ...
0
votes
2answers
103 views

Residual vs. fits

Based on the plot, would it be OK to assume the errors have mean zero (aprox. half of them are under and the other half above zero line) given the strict exogeneity assumption by OLS?
1
vote
0answers
70 views

Bayes Linear regression- logarithmic transformation of prior distribution of the variance

I have a Bayesian version of a linear regression with 3 covariates. The model is given by \begin{align*} Y\sim N(\mu,\tau)\end{align*} \begin{align*} \mu=\alpha + \sum\beta_{i}x_{i}\end{align*} where ...
1
vote
0answers
27 views

Different values predicted by OLS model and Time Series model

Let us say , I have an explanatory variable X and a dependent variable Y and I use OLS and find Y = 0.5 + 3X. Now let us assume that both X and Y are time series data, so using ARIMA modelling ...
1
vote
0answers
50 views

Equivalence of the partial least square regresssion's iterative algorithm and its optimization problem

I am reading The Elements of Statistical Learning. This is a page from the partial least square section: The exercise asks to prove the equivalence between Algorithm 3.3 and Eq. (3.64). Here's my ...
0
votes
1answer
22 views

Is testing for endogeneity necessary?

I was wondering if testing for endogeneity is necessary when doing an OLS multiple regression. Similarly, is testing for autocorrelation necessary if there is no time series data? Is testing for ...
0
votes
0answers
55 views

Least squares method for parameter estimation in AR(1) model

In order to estimate parameters $ \mu $ and $ \alpha$ least squares method can be used.This is what I did to find the least squares. $S=\sum_{t=2}^n(X_t-\mu-\alpha X_{t-1}+\alpha\mu)^2$. And ...
0
votes
1answer
48 views

How to explain change of sign on regression coefficient when another variable is added to OLS model? [duplicate]

I am trying to run an OLS regression, with log of per capita calorie as my dependent variable and age and years of education of household head, log per capita expenditure as my independent variables ...
3
votes
0answers
17 views

Will a pooled coefficient be bounded by the coefficients from the group regressions? [duplicate]

Consider the linear regression model, $y = \beta_0 + \beta_1x_1 + \beta_2x_2 + u, u \sim N(0, \sigma^2 )$ If we estimate the model twice using OLS on two mutually exclusive groups (say, group 1 and ...
0
votes
1answer
84 views

Linear model vs. linear regression

I have a question that I find really confusing regarding linear modelling and linear regression. I have expectation regarding the way some dependent variable (DV) are going to evolve with an ...
1
vote
1answer
27 views

Statistical model that finds a coordinate (x,y) that minimizes the distance from a group of coordinates

A example would be if I launched a 100 tennis balls in the air and plotted the coordinates of where each landed. I would like to be able to find the point in the center of all those coordinates. I ...
0
votes
0answers
23 views

Does forward filtering result in OLS estimate for each time point?

I'm learning about dynamic linear models and was trying to think about the relationship between GLS and forward filtering (Kalman filtering where the state is the vector of parameters). Here's my ...
1
vote
1answer
62 views

Added interaction term, standard errors inflated

I am running a simple regression of an index of cardiovascular health (Heart Rate Variability) on Age and Gender (as a dummy variable), n=430. I first ran: $$HRV \sim \beta_0 + \beta_1Age ...
0
votes
1answer
34 views

Determine weights in weighted least squares regression

Assume we have a cross-section of $N$ stocks. $Y_i$ is an sample variance estimate of stock returns for stock $i$. This sample variance is estimated using $T_i$ number of observations. All $T_i$ are ...
1
vote
1answer
25 views

Covariance of OLS estimator and residual = 0. Where is the mistake?

$Cov(b,e|X)$, where $b$ is the OLS estimator of the coefficients, $e$ is the residual vector, and $X$ is the regressor matrix. We know that $Cov(b,e|X)=E(be'|X)-E(b|X)E(e'|X)$ where ' $'$ ' is the ...
0
votes
0answers
33 views

OLS with dummy and few observations (12 mean values from 2.5k observations)

I have a dataset of 12 markets, in total ~2.5k trades happened over all markets. I now calculated 6 different measures for the each markets performance (my 12 observations per measure I want to use as ...
0
votes
0answers
19 views

Asymptotic distribution of t-ratio

I am looking at a problem where I have to calculate the asymptotic distribution of a t-ratio, after having run a OLS regression. I have re-written the expression so as to be t = z * sqrt ...
0
votes
1answer
71 views

About stepwise regression and correlation

I am trying to fit a model to some observed data with the least squares method. Now, I am at the stage where I have run a stepwise regression (traditional), with Entry level $=0.025$ and Stay level ...
8
votes
2answers
166 views

How do residuals relate to the underlying disturbances?

In the least squares method we want to estimate the unknown parameters in the model: $$Y_j = \alpha + \beta x_j + \varepsilon_j \enspace (j=1...n)$$ Once we have done that (for some observed ...
0
votes
0answers
26 views

Trouble with horner function in MATLAB [closed]

I have the following homework question: Apply linear least squares with the two models S1(A, B, C) = Ax^2 + Bx + C and S2(A, B, C, D) = Ax^3 + Bx^2 + Cx + D to the data set (0, 4), (1, −1), (2, ...
0
votes
0answers
29 views

OLS derivation question [duplicate]

How come I always see the derivation of $\hat{\beta}$ in OLS using matrix differentiation and solving for when the derivative is $0$. Couldn't one just derive it also by noting that in $Y = X\beta + ...
1
vote
1answer
41 views

Why does this have zero sample correlation?

http://web.stanford.edu/~mrosenfe/soc_meth_proj3/matrix_OLS_NYU_notes.pdf On page 4, it says that $x_k'e = 0$ implies that each regressor has zero sample correlation with the residuals. I don't see ...
0
votes
0answers
22 views

creating an indexed dummy variable as a predictor in OLS

I am performing on OLS with two predictors and a response variable. The data is a time series of 450 days approximately. There is an irregular pattern in my response variable - it sometimes ...
2
votes
0answers
56 views

How is the use of OLS in estimating ARCH(q) models justified?

So when estimating ARCH(q) model, some procedures go like estimating first AR(q) model using OLS and then using OLS again against error terms of AR(q). But ARCH suggests heteroskedasticity, and OLS ...