Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
0answers
12 views

Interpreting interaction term via margins command? [on hold]

I am using Stata. I regressed test score on a wealth dummy (high/low) and a maternal education dummy (high/low) and some control variables, among them age (3,4,5 and 6 years), in a linear regression. ...
0
votes
1answer
21 views

Significance (p-value) in regression/size of parameter estimate

In a linear (OLS) regression, I find that the sizes of the coefficients vary a lot: one of them is about 0.0005 and another is ~150. The p-values are all under 0.1. Is this reliable result? These are ...
0
votes
0answers
13 views

Statistical test for whether matching is required

I am struggling to ascertain the conditions when matching is advisable over OLS. It is my understanding that matching can reduce the problems associated with outliers impacting regression results ...
1
vote
1answer
17 views

what is “Minimum Length Least Square”

I am in the process of implementing Bayesian Lasso with Normal-Gamma prior; In section 3.3 mention The prior for the scale parameter $\gamma$ conditional on $\lambda$ is given by $v_\beta = 2 \lambda ...
0
votes
0answers
21 views

Partial Least Squares regression

Assume we have a simple linear regression model expressed as $Y= X \beta + e$, where $Y$ is a vector of size $n \times 1$, $X$ is a matrix of size $ n \times p$, $\beta$ is the regression coefficients ...
1
vote
0answers
18 views

Regression analysis of a correlation coefficient

I have a time series of the 250 day historical correlation and I need to determine what causes this correlation to change as different explanatory variables change. Is there a way that I can regress ...
0
votes
0answers
9 views

General to specific approach vs information criterion

In ARDL model I want to determine proper lags for model. I have two option for this. The first is General to specific approach and deleting all insignificant variables. And the second is using ...
0
votes
0answers
21 views

least square mean

Here I read that when comparing groups with different characteristics (age, percentage of males, body mass index) means should be corrected for such imbalances. These corrected means are called ...
2
votes
0answers
85 views

Question about the answer to “Local polynomial regression: Why does the variance increase monotonically in the degree?”

I appreciated Marco's elegant answer explaining why the variance of a local polynomial regression increases monotonically in the degree. However, in the end of the proof, I find difficult to calculate ...
0
votes
0answers
17 views

Why do the residual sum of squares and the mean absolute percentage error conflict with each other?

I carried out regression with 7th degree and 8th degree polynomials. As expected, the residual sum of squares for 8th degree polynomial regression is less than that of 7th degree polynomial ...
0
votes
1answer
25 views

Combining multiple OLS Regressions

I have a single output $y$, and multiple inputs $x_1, x_2,\dots,x_n$. I am running online(streaming) regression, which would be complicated with many inputs. So, to go around it, I want to have $n$ ...
0
votes
0answers
18 views

Interaction term issue in a OLS regression

I am trying to understand the possible causes to this regression issue I am having. I have a basic time-series regression: (1) $y_t = \alpha_0 + \sum_{j=0}^{J=8}\beta_jx_{t-j} + \epsilon_t$ I am ...
2
votes
1answer
59 views

Why beta sign is different than correlation sign? [duplicate]

I am trying to interpret the sign of my 5 x-variables against y-variable. The sign of some coefficients in the regression output (command: reg) are different than the signs under correlation matrix ...
0
votes
1answer
20 views

In OLS, can I use the ratio of two regressor as an additional regressor?

Suppose I am regressing Y on x and z. Would there be any concern if I also include x/z as an additional regressor? i.e. I am regressing y on x,z and x/z Thanks!
6
votes
1answer
108 views

Interpretation of $\mathbf{y}^T(\mathbf{I}-\mathbf{H})\mathbf{y}$ in OLS

In classic OLS regression it is well-known that $(\mathbf{I}-\mathbf{H})\mathbf{y}=\mathbf{r}$, where $\mathbf{I}$ is the identity matrix, $\mathbf{H}$ is the hat matrix, $\mathbf{y}$ is the vector of ...
1
vote
0answers
28 views

Coefficient Interpretations

My regression equation is the following: dit = αdit-1 + γyit-1 + xit-1β + µt + δi + uut, where dit is my democracy score, and yit is my independent (income) variable. The rest are coviariates, time ...
0
votes
1answer
35 views

Nonlinear regression curve fitting doesn't work

I'm stuck with some seemingly easy task to fit nonlinear regression model. It worked normally until some new data came. Here is my code: ...
0
votes
1answer
31 views

True or false: Least Squares is Statistics Independent?

If one has a vector of data, $Y$, and a vector of covariate $X$, supposed to be related linearly, the least squares estimate is $\hat{\beta} = (X^{T}X)^{-1}X^{T}Y$. This is simply the orthogonal ...
1
vote
1answer
39 views

Ways to stabilize OLS betas [closed]

I am estimating the parameters of a system of OLS equations in Matlab. $y=X\beta+\epsilon \to \hat \beta=(X'X)^{-1}X'y$. My $X$ is a $5\times 5$ matrix and $y$ is a $5\times 1000$ matrix, so $\beta$ ...
0
votes
1answer
24 views

Response Surfaces and Multiple Linear Regression

Suppose I have a MLR equation $y=b_0+b_1x_1+b_2x_2 + e$. If I were to plot this equation, should it not produce a "line" ? I have looked into response surfaces and I am not sure how one would derive ...
4
votes
3answers
90 views

Proof of link between the OLS slope estimate and two sample t test statistic (categorical Xvar)

Regarding a univariate OLS regression with a single categorical predictor (coded 0,1). I am wrestling with the proof that $$t =\frac{b_1}{s(b_1)} $$ starting from the basic OLS estimator for the ...
0
votes
0answers
24 views

Linear regression: what will happen if the effect of a predictor is removed from response

Given a response $Y$ and explanatory variables $X_1, \dots, X_n$, suppose we use least squares estimate to obtain coefficients $\beta_0, \beta_1, \dots, \beta_n$ from the model $$Y = \beta_0 + \beta_1 ...
1
vote
0answers
26 views

Sequential Least Squares for Tikhonov Regularization [duplicate]

Given a Weighted Linear Least Squares problem where the cost function is given by: $$ J = { \left( x - H \Theta \right) }^{T} {C}^{-1} { \left( x - H \Theta \right) } $$ There is a Sequential ...
3
votes
2answers
40 views

The mean shifted outlier model

In the OLS setting, the mean shifted outlier model connects two behaviors: delete the $i^{th}$ observation adding a variable In other words, assume the original model, $\omega$ ,is: $$ Y = X\beta ...
4
votes
1answer
82 views

Proof of LOOCV formula

From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$ ...
0
votes
0answers
25 views

Prediction performance of OLS and Lasso

I am running a comparison of prediction performance of two model using OLS and LASSO respectively. LASSO estimates are computed from LARS algorithm, AIC and BIC were used in model selection. In ...
0
votes
1answer
19 views

Detect multicollinearity in maximum likelihood scenarios

I'm estimating a binary logit discrete choice model with BIOGEME and want to check for multicollinearity of my predictors. BIOGEME uses maximum likelihood estimation (MLE) and not ordinary least ...
0
votes
1answer
50 views

regression with ratio variables

I plan do run a regression analysis with ratio defined variables such as (FX loans/ total loans, tangible assets/total assets etc.) and I have only 13 annual observations. This regression is needed to ...
0
votes
1answer
82 views

When to not use R squared [duplicate]

I recently graduated graduate school and am looking for a proof on R squared. Specifically when to not use it. I really remember a professor impressing upon me multiple times not to report R squared ...
1
vote
1answer
18 views

Regression with double-counting

I am using regression to model production cost based on multiple regressors. Two of them are related, but may have different effects: number of products made and number of unique products made. I am ...
0
votes
0answers
17 views

I inserted n variables into statsmodels.formula.api.ols and returned > n predictions?

This makes no sense to me? I have split the data into training and testing data with sklearn. ...
0
votes
1answer
18 views

if we have 2 independent variables w correlation can we still use them in OLS

I've read from some resources that one of the assumptions in ols is that the predictor variables do not have a correlation. Is this because correlation => linear dependency? Which I know we cant have ...
10
votes
3answers
258 views

Why sigmoid function instead of anything else?

Why is the de-facto standard sigmoid function, $\frac{1}{1+e^{-x}}$, so popular in (non-deep) neural-networks and logistic regression? Why don't we use many of the other derivable functions, with ...
0
votes
1answer
35 views

How can I obtain the log-likelihood value in a OLS estimation?

So, in some papers using panel data, I noticed that in the estimate results inherent a pooled OLS regression, they report the value of the log-likelihood. I was wondering how this is possible, in ...
1
vote
1answer
17 views

Transforming models in order to use linear least squares estimations

As a pre-exam question, I found a question asking to consider the following three models $$ y = \beta_{0}(x_{1})^{\beta_{1}}(x_{2})^{\beta_{2}}\epsilon $$ $$ y = \frac{1}{\beta_{0} + \beta_{1}x + ...
1
vote
2answers
41 views

What does LS (least square) means refer to?

I have been reading clinical papers and recently come across the term "LS-means", referring to what seems to me as an estimation of some population's mean measure. Obviously, I know what "mean" refers ...
2
votes
0answers
28 views

Total least square intuition

I still have to find a good intuitive explanation of TLS. Online resources tend to focus on the vertical vs. perpendicular square error pictures (I don't need to see perpendicular lines to understand ...
0
votes
0answers
11 views

Statistic to summarise multiple timeseries observation with missing data

I have a data frame containing timeseries data on 200 patients ...
0
votes
1answer
37 views

Use Linear Regression to Estimate Conditional Probability for Bayes Net?

When reading and watching video regarding building and using Bayes Nets, the examples typically use binary outcomes for the nodes. 'Probability of it raining', 'Probability of x disease', ect... ...
0
votes
1answer
47 views

Regression imputation of missing data based on OLS effects

Let's say we have a two-way with interaction experiment with missing data. Being the dataset: ...
2
votes
1answer
47 views

Regression without intercept: deriving $\hat{\beta}_1$ in least squares (no matrices)

In An Introduction to Statistical Learning (James et al.), in section 3.7 exercise 5, it states that the formula for $\hat{\beta}_1$ assuming linear regression without an intercept is $$\hat{\beta}_1 ...
0
votes
0answers
20 views

Can the inclusion of exogenous variables in an ARMAX control for non-stationarity?

I have a non-stationary time series. If I run an OLS regression, the residuals appear non-stationary but serially correlated. Can I then run an ARMAX model on this time series, since the inclusion of ...
1
vote
1answer
95 views

Using linear regression for count data - will this introduce bias?

Say I am fitting a model to Poisson count data, but I am only interested in estimating the mean of the count variable. I understand a ordinary linear regression is a good approximation when the ...
2
votes
0answers
16 views

Comparing goodness of least-squares fits through origin

I was wondering how to measure the goodness of fit of a linear least squares regression constrained through the origin. I have been using r-squared for comparing unconstrained fits, but I understand ...
0
votes
1answer
54 views

Multivariate OLS - Partialling Out

I have bee wondering why in a multivariate OLS-Regression it is not possible for R² to decrease when increasing the number of explanatory variables. The Point is that for example in the model ...
2
votes
1answer
39 views

How to represent goodness of fit for multiple least squares fits

I have many sequences (~50) of time series data that I have fit to a non-linear model using a least squares fit. If I had a single sequence of time series data, and I fit a model to it using a least ...
1
vote
0answers
15 views

Using OLS with missing panel data

I am trying estimate the following model using OLS $$ Y_{t} = \beta_{0} + \beta_{1}X_{1t} + \beta_{2}X_{2t} + \epsilon_{t} $$ where $Y_{t}$ is the difference between two observed variables, $Y_{t} ...
0
votes
0answers
94 views

Finding the optimal threshold parameter

Assume we are penalizing the least squares by the hard thresholding penalty: $argmin_\theta 2^{-1}(z_2-\theta)^2 + p_\lambda(|\theta|)$ where $p_\lambda(|\theta|)$ is the hard thresholding penalty ...
2
votes
0answers
31 views

How to choose between different options in partial least square regression?

There seem to be several methods of performing partial least square regression. For example in pls pacakge in R, following are available: ...
0
votes
0answers
24 views

Multifactor Covariance Matrix

hanks for taking a look. I am struggling to understand a rather simple concept. I ran a simple linear regression of the form $$A= \alpha+ \beta X + E$$ $$C = \alpha +\beta X + E$$ Then i ...