Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
1answer
42 views

Can someone explain the mechanics of the variance-covariance matrix in OLS?

I have read many of similar posts here already as well as other resources on the topic, but they all generally just show the steps that generate this equation: $\hat{\sigma^2}({X}'X)^{-1}$ What I ...
5
votes
2answers
74 views

OLS, Heteroscedasticity, Asymptotic Efficiency

I know that OLS is unbiased but not efficient under heteroscedasticity in a linear regression setting. In Wikipedia http://en.wikipedia.org/wiki/Minimum_mean_square_error The MMSE estimator is ...
6
votes
2answers
82 views

multiple local optimum solutions when we solve a linear regression?

I read this statement on one old exam that be (True / False). We can get multiple local optimum solutions if we solve a linear regression problem by minimizing the sum of squared errors using ...
0
votes
0answers
27 views

ols and multiple regression model wth two variables

My model is as follows: $y=b_1*X+b_2*\max(X,0)+u$ Will I have any problems with the variables $X$ and $\max(X,0)$, concerning any correlation issues? Can I just apply the classic OLS methodology?
1
vote
1answer
32 views

Are $\hat{\beta}_{\text{ls}}$ and $S^2$ independent if errors are not normally distributed?

When estimating a linear model $$ Y_i = X_i\beta + \varepsilon_i \quad \quad 1\leq i\leq n$$ We have $\hat{\beta}$ the least squares estimation of the slope and the estimation of the variance, $S^2 = ...
0
votes
1answer
35 views

Need help in understanding continuous-dummy interaction in OLS regression

I am having some conceptual difficulties in understanding and interpreting interaction terms (between a dummy and a continuous variable) in OLS regressions. I was hoping someone could help me out. I ...
5
votes
1answer
65 views

If $\operatorname{Var}\left(\epsilon_i\right) = h\left(X\right) \neq \sigma^2$, what can we know about $\operatorname{Var}\left(\hat{\beta}\right)$?

This question uses the derivations found here. The short version Consider a regression model. If the error variance is a known function of the data (rather than a constant), under what conditions ...
0
votes
0answers
51 views

Maximum likelihood method vs. least squares method

What is the main difference between maximum likelihood estimation (MLE) vs. least squares estimaton (LSE) ? Why can't we use MLE for predicting $y$ values in linear regression and vice versa? Any ...
2
votes
1answer
43 views

Motivation for gradient descent method over canonical method (for OLS/MLE) for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
0
votes
0answers
18 views

Weighting scheme in ols

I am trying to build a weighted least squares model from some data that I have (x,y,z). y is known to be linear in x, but I have data for different groups of z. So z is a factor variable. The function ...
3
votes
3answers
54 views

Does inconsistent causation mean inconsistent estimator?

I have this problem. I have Y (market share) and X (store size). I want to predict Y from X using a linear regression ... I run OLS to find the betas, their pvalue is meaningful, yada, yada, yada ...
0
votes
1answer
33 views

Preventing overfitting with Least Squares Linear Regression via QR decomposition

I am trying to solve a linear regression problem in an automated fashion, however am having a problem with extremely large weights. I have several thousand datasets, and am running linear regression ...
0
votes
3answers
40 views

Equating the two equations of Ridge Regression

I'm studying Ridge Regression now and I'm having a bit of trouble understanding how to relate the two equations that pop up when I read about it. There is the coefficient estimate: $$\hat{\beta} = ...
0
votes
1answer
33 views

When a slope doesn't match a visual trend

I am trying to determine the linear slope between two variables in a dataset. The ordinary least squares (OLS) method returns a slope which does not appear to fit the trend that one's eye sees in the ...
0
votes
0answers
15 views

What issues may I face when interpolating my dependent variable in an OLS regression?

I'm doing my undergrad dissertation on what host-country factors impact FDI inflows - FDI inflows to the UK is my dependent variable. All of the independent variables I have managed to find at a ...
2
votes
1answer
37 views

$F$-test for hypothesis $\beta_1+\beta_2=2\beta_3$ in a regression

In a regression $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$, how do I use an $F$-test to test the hypothesis $\beta_1+\beta_2=2\beta_3$? The standard $F$-test would test a ...
3
votes
0answers
24 views

Adjust linear regression penalty for under/over-estimations

Basically I have a case where under-predictions are worse than over-predictions. Is there a way to penalize the linear regression model during training according to some predefined ratio? E.g. I want ...
3
votes
1answer
42 views

What happens in linear regression if Y's are not independently sampled?

What happens in linear regression $Y\sim X$ when the $Y$'s are not independently sampled and, particularly, may be autocorrelated? I believe the estimator will still work. But what will happen to ...
2
votes
1answer
104 views

Is $Y \sim X$ equivalent to $ln(Y) \sim ln(X)$?

I read in this thread that $Y \sim X$ is equivalent to $ln(Y) \sim ln(X)$ (assuming $X>0$ and without considering standard error issues). Indeed OLS theory says that heteroskedasticity of the ...
0
votes
0answers
13 views

Interaction model with measurement error without replicate measurements

I am currently working on a simple OLS model with two independent variables and one dependent variable: \begin{equation} \ Y_i = \beta_0 + \beta_1 X_i + \beta_2 S_i + \beta_3 X_iZ_i \end{equation} ...
2
votes
1answer
48 views

Nonlinear total least squares / Deming regression in R

I've been using nls() to fit a custom model to my data, but I don't like how the model is fitting and I would like to use an approach that minimizes residuals in ...
0
votes
0answers
52 views

Pooled OLS, fixed, random or mixed effects?

I am analysing a simple balanced panel data with the following variables: ...
2
votes
0answers
24 views

Generating Random Sample to Fit LM Output

I am trying to reverse generate a dataset that led to a certain R lm() output, l I tried to generate random sample like this, and ran lm() ...
0
votes
0answers
10 views

panel data wage equations

I have in my panel data X countries, personal IDs for each individual per country and year. The panel runs for X years. I set the panel based on ID and year. IDs are grouped per country. Running ...
0
votes
0answers
23 views

Specification Error and Bias

True model: $Y = α + βX+ γZ + U$, where U is the error with OLS properties If $X= δ_0 + δ_1Y + δ_2A + δ_3B + R$ , where A and B are exogenous, and R is the error , How can we see that the OLS ...
0
votes
0answers
18 views

regression with autocorrelated errors and specific error structure

I have to fit a linear regression model that takes into account both a specific - conditional variance relationship and a regression form $y_i=a+\beta \times x+\sqrt{\gamma \times x^2}\times ...
12
votes
3answers
138 views

Why trace of $I−X(X′X)^{-1}X′$ is $n-p$ in least square regression when the parameter vector $\beta$ is of p dimensions?

In the model ${y} = X \beta + \epsilon$, we could estimate $\beta$ using the normal equation: $$\hat{\beta} = (X'X)^{-1}X'y,$$ and we could get $$\hat{y} = X \hat{\beta}.$$ The vector of residuals ...
5
votes
4answers
277 views

Recommendation for linear regression with least squares book

I watched several videos on linear regression, mainly from Khan Academy. As I have no background in statistics, I thought this was a good way to get an idea of the topic. However I'm currently writing ...
1
vote
0answers
32 views

Is the sum of all elements of the residual matrix equal zero under OLS?

I have the following OLS model $$ y_i= α+βx_i+ε_i , i = 1,...,N $$ I want to prove that $$ \sum_{k=1}^N\sum_{j=1}^N e_je_k =0$$ I did the following $$ \sum_{k=1}^N (e_1+e_2+e_3+...+e_n) e_k ...
3
votes
2answers
204 views

Why logistic regression cannot be solved by OLS

I know I mess up different things. However I want to get better understanding what is motivation behind logistic regression. $p(y_i=1|x_i)=(1+e^{-x_iw})^{-1}$ and according to OLS ...
0
votes
0answers
15 views

Degrees of freedom with duplicate data points

The degrees of freedom of the residual in an OLS model is $n - p - 1$, where $n$ is the number of samples, and $p$ is the number of independent variables. I.e., the data matrix $X$ is $n\times p$. If ...
3
votes
0answers
45 views

Can I use OLS with robust standard errors to analyze proportions?

I have a response variable that is a proportion. It is not the outcome of a series of Bernoulli trials, so there is no numerator/denominator, just the proportion. I'd like to assess the relationship ...
2
votes
1answer
53 views

Big data database + software for advanced statistical analysis?

I need to run some statistical hypothesis testing, Anova, student's, least square fit, median, data mining, clustering... on a very large quantity of distributed data. (>100TB, Maybe columnar or ...
0
votes
0answers
38 views

What happens if I square the variable in my log in OLS regression?

Say I have a model: ln y = B0 + B1(x1) + B2 ln(x2) + u and the B2 estimate I get is 0.5 If I change the model to be ln y = B0 + B1(x1) + B2 ln(x2^2) + u the estimate will change to 0.25, but why ...
0
votes
1answer
57 views

Robust OLS standard errors (Newey-West)

I am running a simple OLS regression with HAC adjustment (i.e. Heteroschedasticity and Autocorrelation adjustment) using the following function in hac() in matlab. ...
1
vote
1answer
28 views

Individual-specific error component in pooled OLS

I know that Pooled OLS is not efficient if there is existence of the individual-specific error component (one that doesn't vary over time) because the usual standard errors are incorrect and the tests ...
2
votes
1answer
14 views

Is the order of parameter estimates preserved from multiple simple regressions to one multivariable regression?

Assume I have $y$, $x_1$ and $x_2$. I regress $y\sim\alpha_0 + \alpha_1 x_1$, $y\sim\beta_0 + \beta_1 x_2$ and $y\sim\gamma_0 + \gamma_1 x_1 + \gamma_2 x_2$ using Ordinary Least Squares. Does ...
7
votes
1answer
114 views

finding out 2D transformation given a list of sample points

I have a bunch of 2D points that are rotated by $\theta$ and translated by $(\Delta_x, \Delta_y)$. I.e. $$ x'=x \cos(\theta)-y \sin(\theta)+\Delta_x \\ y'=x \sin(\theta)+y \cos(\theta)+\Delta_y $$ ...
0
votes
1answer
30 views

OLS parameter estimation of an expression?

During my research for a class, I came across a paper that said they estimated an equation using OLS. But the parameter they were estimating appeared to be an expression that looked like this (not the ...
2
votes
1answer
104 views

Local polynomial regression: Why does the variance increase monotonically in the degree?

How can I show that the variance of local polynomial regression is increasing with the degree of the polynomial (Exercise 6.3 in Elements of Statistical Learning, second edition)? This question has ...
0
votes
0answers
15 views

Solve intercept coefficient Dummy Variable regression

Suppose we have the model $y=\beta_0+\beta_1 x_2+\beta_2x_3+e_i$ where $x_1, x_2, x_3$ are binary variables, taking on values 0 and 1, so for example, if $x_1=1, x_2=x_3=0$. Now we want to regress ...
1
vote
0answers
25 views

Linear regression with ordinal scaled variables

A friend of mine is working on a problem set and asked me an interesting question: if we have 2 variables, which are ordinal scaled how to compute a linear regression with those? $$x \in ...
0
votes
0answers
30 views

Recode ordinary scaled variables

Iam trying to make a linear regression in STATA with ordinary scaled(in dataset): dependant variable(0;10) independant variable(0;10) Does anyone know how can recode the variables into metric scaled ...
0
votes
0answers
8 views

Interpolating singular values

So I have the singular values associated with a data matrix and I would like to interpolate them and then find the maximum curvature of the interpolation in order to decide how many singular values to ...
0
votes
0answers
8 views

Multinomial Heckman Models

I'm interested in modeling left censored data with a tobit model using R. Mine is a two step, as I want to generally predict probablity, and then quantity, so I'm planning on using a heckman. The ...
0
votes
0answers
17 views

Interpreting standardized log coefficients in OLS

I used the log for my dependent variable as well as for some independent variables. Then I standardized all variables. Now I'm not sure how to interpret the coefficients. Are the log and non loged ...
0
votes
1answer
30 views

Java and R: Least Squares Coefficient Estimation - Start at time Zero?

This is the data set I have: vector <- c( -7.459981, 13.26651, 12.10128, 2.380662, 26.42393) Doing an estimation of the coefficient with a linear regression ...
0
votes
0answers
32 views

Standard error/deviation of the coefficients in OLS

In OLS, the variance of the regression coefficients are computed as $$ \mathrm{Var}(\hat{\beta}) = \sigma^2(\mathbf{X}^\mathrm{T}\mathbf{X})^{-1}. $$ Now, if I need to compute the standard ...
0
votes
0answers
14 views

Least squares (vs) and cost function

What is the sense behind to minimize the error of a hypothesis function with a cost function instead of the applying least squares method? I think you actually get the best approximation by using the ...