Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
1answer
44 views

When to not use R squared

I recently graduated graduate school and am looking for a proof on R squared. Specifically when to not use it. I really remember a professor impressing upon me multiple times not to report R squared ...
1
vote
1answer
16 views

Regression with double-counting

I am using regression to model production cost based on multiple regressors. Two of them are related, but may have different effects: number of products made and number of unique products made. I am ...
0
votes
0answers
9 views

I inserted n variables into statsmodels.formula.api.ols and returned > n predictions?

This makes no sense to me? I have split the data into training and testing data with sklearn. ...
0
votes
1answer
15 views

if we have 2 independent variables w correlation can we still use them in OLS

I've read from some resources that one of the assumptions in ols is that the predictor variables do not have a correlation. Is this because correlation => linear dependency? Which I know we cant have ...
9
votes
3answers
156 views

Why sigmoid function instead of anything else?

Why is the de-facto standard sigmoid function, $\frac{1}{1+e^{-x}}$, so popular in (non-deep) neural-networks and logistic regression? Why don't we use many of the other derivable functions, with ...
0
votes
1answer
29 views

How can I obtain the log-likelihood value in a OLS estimation?

So, in some papers using panel data, I noticed that in the estimate results inherent a pooled OLS regression, they report the value of the log-likelihood. I was wondering how this is possible, in ...
1
vote
1answer
14 views

Transforming models in order to use linear least squares estimations

As a pre-exam question, I found a question asking to consider the following three models $$ y = \beta_{0}(x_{1})^{\beta_{1}}(x_{2})^{\beta_{2}}\epsilon $$ $$ y = \frac{1}{\beta_{0} + \beta_{1}x + ...
1
vote
2answers
27 views

What does LS (least square) means refer to?

I have been reading clinical papers and recently come across the term "LS-means", referring to what seems to me as an estimation of some population's mean measure. Obviously, I know what "mean" refers ...
2
votes
0answers
27 views

Total least square intuition

I still have to find a good intuitive explanation of TLS. Online resources tend to focus on the vertical vs. perpendicular square error pictures (I don't need to see perpendicular lines to understand ...
0
votes
0answers
9 views

Statistic to summarise multiple timeseries observation with missing data

I have a data frame containing timeseries data on 200 patients ...
0
votes
1answer
25 views

Use Linear Regression to Estimate Conditional Probability for Bayes Net?

When reading and watching video regarding building and using Bayes Nets, the examples typically use binary outcomes for the nodes. 'Probability of it raining', 'Probability of x disease', ect... ...
0
votes
1answer
40 views

Regression imputation of missing data based on OLS effects

Let's say we have a two-way with interaction experiment with missing data. Being the dataset: ...
2
votes
1answer
36 views

Regression without intercept: deriving $\hat{\beta}_1$ in least squares (no matrices)

In An Introduction to Statistical Learning (James et al.), in section 3.7 exercise 5, it states that the formula for $\hat{\beta}_1$ assuming linear regression without an intercept is $$\hat{\beta}_1 ...
0
votes
0answers
12 views

Can the inclusion of exogenous variables in an ARMAX control for non-stationarity?

I have a non-stationary time series. If I run an OLS regression, the residuals appear non-stationary but serially correlated. Can I then run an ARMAX model on this time series, since the inclusion of ...
1
vote
1answer
77 views

Using linear regression for count data - will this introduce bias?

Say I am fitting a model to Poisson count data, but I am only interested in estimating the mean of the count variable. I understand a ordinary linear regression is a good approximation when the ...
2
votes
0answers
13 views

Comparing goodness of least-squares fits through origin

I was wondering how to measure the goodness of fit of a linear least squares regression constrained through the origin. I have been using r-squared for comparing unconstrained fits, but I understand ...
0
votes
1answer
51 views

Multivariate OLS - Partialling Out

I have bee wondering why in a multivariate OLS-Regression it is not possible for R² to decrease when increasing the number of explanatory variables. The Point is that for example in the model ...
2
votes
1answer
38 views

How to represent goodness of fit for multiple least squares fits

I have many sequences (~50) of time series data that I have fit to a non-linear model using a least squares fit. If I had a single sequence of time series data, and I fit a model to it using a least ...
1
vote
0answers
13 views

Using OLS with missing panel data

I am trying estimate the following model using OLS $$ Y_{t} = \beta_{0} + \beta_{1}X_{1t} + \beta_{2}X_{2t} + \epsilon_{t} $$ where $Y_{t}$ is the difference between two observed variables, $Y_{t} ...
0
votes
0answers
82 views

Finding the optimal threshold parameter

Assume we are penalizing the least squares by the hard thresholding penalty: $argmin_\theta 2^{-1}(z_2-\theta)^2 + p_\lambda(|\theta|)$ where $p_\lambda(|\theta|)$ is the hard thresholding penalty ...
2
votes
0answers
24 views

How to choose between different options in partial least square regression?

There seem to be several methods of performing partial least square regression. For example in pls pacakge in R, following are available: ...
0
votes
0answers
22 views

Multifactor Covariance Matrix

hanks for taking a look. I am struggling to understand a rather simple concept. I ran a simple linear regression of the form $$A= \alpha+ \beta X + E$$ $$C = \alpha +\beta X + E$$ Then i ...
1
vote
1answer
50 views

Identical observations in linear regression

I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
0
votes
1answer
24 views

Least square estimation with quadratic fit. Any simple solution?

We consider the least square problem in the case where we got only one independant variable $x_i$ and only one dependant variable $y_i$. The number of observations is $n$. In the case of the linear ...
0
votes
0answers
35 views

Practical beginners resource for building a dynamic OLS model

I need to model the current account balance of a country. The regressors are the real effective exchange rate, the domestic GDP and the GDP of the world. I am using data for 30 years (in logs). It is ...
2
votes
1answer
33 views

Is OLS in Engle-Granger a valid method to use when finding the cointegrating vector?

In this post mpiktas showed that the sample correlation measure for two random walks (possible correlated) is a random variable and does not estimate the theoretical correlation. When trying to find a ...
0
votes
0answers
104 views

Interpreting regression coefficient - what units are fractions?

I am regressing a growth measure (in fraction form, between 0 and 1) on another fraction that lies between 0 and 1 (let's call the variable ``share"). The regression is performed using OLS. What is ...
1
vote
0answers
19 views

Preinititalising Neural Networks using Least Squares solution

I am attempting to train a neural network, to approximate an unknown function. The function domain and range is real valued vectors with 300 elements: all of which are between -1 and 1 (exclusive). ...
3
votes
1answer
42 views

OLS estimation for Nonlinear model

Consider the following model which may be nonlinear: $Y_{t} = f (X_{t}, \beta_{0}) + \mu_{t}, \hspace{0.2cm} t=1, ..., T$ If we assume that: $\mu_{t}$ i.i.d with mean = $0$ and ...
2
votes
0answers
43 views

Autocorrelation in DOLS: will HAC standard errors work?

I am currently estimating a cointegrating regression (DOLS), where my residuals have autocorrelation. Sometimes it is just in one or two lags, but sometimes it is more. My question is: Can I apply HAC ...
0
votes
0answers
23 views

What is the reverse of the between estimator called?

Between estimator is defined as running a regression of the form: $Y_i = C + B_0X_i + e_i$ On panel data, where Y is the cross sectional mean of Y, X is the cross sectional mean of X. As a result of ...
0
votes
0answers
23 views

All models (LPM, Logit, HGLM, etc.) yield same results — which to report?

I'm interested in people's opinions here on a modeling choice given this design. We have two years of data from students in a school district. An intervention was introduced in year two. We have a ...
0
votes
0answers
26 views

How to model “aggregate” dependent variable in case of variable transformation?

Background: I have a panel data set consisting of dependent variable $y_{i,t}$ and several independent variables $x_{j}$, where $i$ indicates observation (ID), $j$ serves as dependent variable index ...
0
votes
0answers
24 views

Tikhonov regularizes least square dual function

I'd like to find the dual problem for least squares with Tikhonov regularization. For now, I have the primal problem expressed as minimize $||Ax-b||_2^2 + \gamma||x||_2^2$. I'm introducing a dummy ...
2
votes
1answer
40 views

Intrumental variable for covariates

I am only interested in the causal and unbiased effect of x on y and I have used additional covariates in my model to control for other effects. I have a pretty decent instrument for my potentially ...
0
votes
0answers
52 views

What model to use? Heckman-Two-Stage? Tobit? OLS?

I am currently researching innovation with firm level data. I have a nice dataset, which allows to analyze the variable inno which is ...
0
votes
1answer
41 views

Can heteroskedastic residuals be justified by variance in dependent variable?

This is a very basic question and I hope it is not a duplicate. Im using a pooled regression model with a log-transformed dependent variable (electricity consumption meter values). The variance of ...
1
vote
0answers
35 views

Instrument variable that indirectly related to the error term. Valid?

I try to grasp the concept of instruments in 2SLS regressions. I have a variable that is correlated with an endogenous regressor (informative). I also believe that it is uncorrelated with the error ...
1
vote
1answer
60 views

How to determine if GLS improves on OLS?

I have a multiple regression model, which I can estimate either with OLS or GLS. The weights for the GLS are estimated exogenously (the dataset for the weights is different from the dataset for the ...
1
vote
0answers
20 views

Does it make sense to test for nonlinear moderation if my independent variable is insignificant?

Currently doing an OLS regression in Stata. I have 2 independent variables(also included square terms), dummy variables and 1 moderator variable. However, to the best of my knowledge, I have to ...
0
votes
0answers
28 views

General to specific: t-stat, Akaike, Schwarz, and Adjusted R-squared

Specifying a linear model from general to specific i find that removing regressors corresponding to insignificant coefficients actually makes the adjusted r-squared, the Akaike and the Schwarz stats ...
0
votes
0answers
26 views

Prediction interval with Weighted Least Squares Linear Regression

I have been looking for this kind of stuff on the internet for a while and I cannot find any answer. In the classical linear regression (without weights), one can compute the standard deviation and ...
0
votes
1answer
27 views

Testing OLS Model Prediction Accuracy?

How can I test the accuracy of my ordinary least squares model? Is it a simple comparison between the predicted values of my test set and their actual values (with perhaps a maximum threshold of ...
1
vote
1answer
44 views

Normality of data in OLS

I am trying to perform an OLS on time series for a project for college. The professor told me that I need my regressors to be normal in order to justify the use of a linear regression. His argument ...
0
votes
0answers
53 views

Why does the normality of the coefficient's probability distribution follow from the normality of the errors in OLS?

So suppose after a simpple OLS regression we want to know what the chance(P-value) is that the Beta coefficient is 0 . First we assume that many random processes caused the errors ($\epsilon\!_i$), ...
6
votes
1answer
101 views

Interpretation of Saturated Model vs. Model with Interaction and One Main Effect

Say that I have two regressions: 1) $Y_i = \alpha_0 + \alpha_1 X_i + \alpha_2 X_i*Z_i + \epsilon_i$ 2) $Y_i = \beta_0 + \beta_1 X_i + \beta_2 Z_i + \beta_3 X_i*Z_i + \epsilon_i$ $X_i$ and $Z_i$ are ...
4
votes
1answer
73 views

Stein's estimator vs James-Stein estimator

I read a lot of sources concerning stein's estimator and James-Stein estimator. Unfortunately, a lot of sources do not write the correct formulas of each estimator. And so I am now confused!! Kindly, ...
1
vote
1answer
28 views

F-test of joint significance vs multiple t-test for regression parameters? [duplicate]

In the context of linear regression, I don't understand why you need to perform an F-test for the H0 that all parameters are zero, instead of just looking at all the t-tests for each parameter. I ...
1
vote
2answers
66 views

Insignificance by confounding variables

I am confused about a result in my OLS regression. I am regressing health on both crime level and ubanization and a couple of commonly encountered covariates in the literature such as, for example, ...
4
votes
1answer
120 views

Making sense of the first difference regression model

There must be a fundamental error in my approach. Let's start by stating we have a simple regression with two variables $X_t$ and $Y_t$: $Y_t = BX_t + e_t$ Where $B$ is the coefficient and $e_t$ is ...