Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

0
votes
0answers
13 views

Constrained Linear Regression with factor variables in R

I am trying to figure out how to run a simple linear model with two factor variables as regressors without the intercept. I need that the sum of the coefficients estimated for ONE regressor is ...
1
vote
2answers
19 views

How to interpret multiple regression coefficients [duplicate]

I'm running multiple linear regression with 6 variables. For one of the variables D, the correlation coefficient between D and the response Y is - 0.34. But in the regression output, the coefficient ...
1
vote
1answer
18 views

Using datapoint multiple times in error [on hold]

For a simple regression problem, say I have a function $f = x^2 + ax$ and am using mean squared error as a loss function. In each calculation of mean squared error, each datapoint gets used twice (...
0
votes
0answers
19 views

How does this proof assume the existence of an inverse?

I am reading the proof of theorem 2 here https://www.tandfonline.com/doi/full/10.1080/03610926.2016.1183786 Part of this proof says that If $AX \perp BX$ then $AX \perp f_B(BX) = X^T BB^- BX = X^T B ...
4
votes
1answer
51 views

Why lasso for feature selection?

Suppose I have a high-dimensional dataset and want to perform feature selection. One way is to train a model capable of identifying the most important features in this dataset and use this to throw ...
16
votes
3answers
1k views

How is it possible to obtain a good linear regression model when there is no substantial correlation between the output and the predictors?

I have trained a linear regression model, using a set of variables/features. And the model has a good performance. However, I have realized that there is no variable with a good correlation with the ...
1
vote
0answers
18 views

Can R-square be a criterion of simple linear models?

For example, I constructed three simple linear models, say Y ~ A Y ~ B Y ~ C A, B and C are highly correlated Now I have their R-squares and P-values, can I say that one with highest R-square and ...
2
votes
1answer
32 views

Linear Regression: Why do the coefficients change on the original IVs when you interact them, and add that new interacted-variable to the model?

Basically I want to know how the 'constant' value differs in each of the following models: Model 1: DV=income; IV1=gender (0=male, 1=female); IV2=location (0=east, 1=west) Here, I understand the ...
-1
votes
0answers
11 views

How can I interpret my log linear regression (OLS) when my regressor is a non-zero ratio? [duplicate]

My model is a log linear when my dependent variable is log GDP, and my predictor is not log transformed. One of my regressor is ratio between consumption and income. I understand that to interpret log ...
3
votes
1answer
39 views

Are Linear Regression associations correct with a binary dependent variable?

Probably the simplest model when dealing with a binary classification problem is logistic regression. This relies on fitting a linear regression model to the data and then applying a sigmoid function ...
1
vote
1answer
37 views

What is the relationship of long and short regression when we have an intercept?

Consider the linear model estimated by OLS: $$ y = X\hat{\beta} + \hat{u} = X_1 \hat{\beta}_1 + X_2 \hat{\beta}_2 + \hat{u} $$ We say that the above equation is the long regression, Consider also ...
0
votes
0answers
104 views

P-value adjustment for hierarchical / multilevel data

I'm dealing with a large data set which is hierarchical in nature, that is 1000 schools, within every school 1000 students. The dependent variable is a measure of performance. Ideally I would like to ...
0
votes
0answers
12 views

multi class logistic regression : $K-1$ regressors or $K$ regressors (softmax)?

I read that, in multiclass logistic regression, we have a pivot class $K$ and $K-1$ set of $\vec{w}$ weights, then, for the pivot class: \begin{eqnarray} P( C_K | \vec{x} ) &= 1- \sum_\limits{t=...
1
vote
1answer
37 views

How come p-value for ivreg and manual lm differs so much?

Can anyone tell me why the p-values for two stage least squares for manual lm vs ivreg way ...
0
votes
0answers
20 views

Estimation in case of data dependent noise

I am trying to estimate $a$ and $b$ in the below linear model $$y = ax + b + \epsilon$$ where $x \in R^n$ and $y \in R^n$ are given, and $\epsilon$ depends on the parameters and the $x$. Also, it is ...
0
votes
0answers
15 views

Right Inverse of an under-determined linear system

I am looking at a slides from a neural network class and there is a brief discussion on the inverse of an over-determined and under-determined system. I understood the left inverse of an over-...
1
vote
0answers
16 views

ANCOVA - Non-linearity of covariate

I have a dataset where I calculated genetic distances between populations of some animals (a continuous variable). I want to know if these differences are explained by physical or/and ecological ...
0
votes
1answer
12 views

Effect of log transformation on AICc

I have been re-running some multiple linear regression models (using R package nlme) that I had initially log-transformed in R due to violation of the assumption of ...
0
votes
0answers
78 views

AR(p) forecast is almost always above actuals in pseudo out-of-sample forecasting

I have a trouble with my out-of-sample forecasts. The task is to evaluate the out-of-sample performance in forecasting the CPI(price index). In order to do this, I have estimated simple AR(1) with ...
5
votes
1answer
68 views

Solving Linear Regression with Fused Lasso Regularization by MLE

I am currently reading a paper stating the following regression problem $$\text{min} \sum_{i=1}^N ||\beta\cdot x_i-y_i||\\ \text{subject to} \sum_{j=2}^M ||\beta_{j}-\beta_{j-1}|| \leq S $$ for ...
0
votes
0answers
14 views

confusion about multiclass linear classifier

I notice that there is a bit of confusion in multiclass linear classifier notation in at least 2 points: from Bishop's book and for example these slides they call the One-versus-the-rest approach (...
0
votes
1answer
32 views

Is Gradient Boosting Regression Tree able to learn linear models

Assume $Y$ is a linear function of a vector of variables $X$ (plus a noise term). The train data consists of ($X,Y$) such that $X \in [0,1]$. Assume one use gbdt to learn this linear model. And if ...
0
votes
1answer
40 views

Interpreting Regression Coefficient

I am using panel data and I have a very simple regression equation of the following form: $y_{st} = \alpha_s + \beta_t + \gamma F_{st} + \epsilon$ Where, $y_{st}$ is my dependent variable (...
0
votes
0answers
29 views

Extracting latent variable from multivariate linear model, based on residuals

I need some help with some basic regression method. Let's say that we have a tri-variate linear model with continuous variables (as dependent and as independent). $$y=\beta_0+\beta_1 x_1+\beta_2 x^*...
1
vote
1answer
43 views

Regression on subset of observations. Is this valid?

I have a dataset that compiles voting results for transportation referendums. Each observation is a city that has held a referendum. I am interested in the community factors contributing to support so ...
1
vote
0answers
11 views

Intuition: What is the difference between linear factor models and regular linear regression?

So, I have a very vexing theoretical question that I hope some experienced econometricians can help me with. Being in finance, I have recently been exposed to linear factor models, which are models ...
0
votes
1answer
26 views

Log-transformations and concave functions

Consider a linear equation $O = SW$ with $O \in \mathbb{R}^{g \times n}, S \in \mathbb{R}^{g \times k}, W \in \mathbb{R}^{k \times n}$, with $g \gg n, g \gg k, k < n$. $W$ is a frequency matrix and ...
0
votes
0answers
8 views

Checking for change in significance of regression coefficient after adding covariate

I'm wondering whether there is a way to check whether a coefficient in a linear mixed effects model changed significantly after introducing a new variable. I have two models: ...
1
vote
0answers
35 views

Comparing effect of two IVs

I use a linear regression model to understand the effects of two regressors $X$ and $Y$ on a dependent variable $Z$. Let us assume that a unit change in $X$ and $Y$ are comparably interpretable. I now ...
11
votes
4answers
1k views

Using Decibels in Statistics

I'm working on a project that involves reading RFID Tags and comparing the signal strength the reader sees when you change the antenna configuration (number of antenna, position, etc...). As part of ...
4
votes
1answer
35 views

linear vs non-linear kernel SVM

The dataSet contains 213 examples of 7 classes . Each example are 25000 features. I want to learn model with SVM (test scenario used are 10-fold cross validation). I am a beginner in machine learning, ...
0
votes
0answers
53 views

Linear regression with feature representation confusion - relationship of design matrix column space to the feature space?

I am trying to visualise the geometry of linear regression with feature representation. I have a regression problem with $n$ data pairs $\mathcal{D}:=\{(\mathbf{x},y)_{i}\}_{i=1}^{n}$, independent ...
3
votes
2answers
63 views

Testing complex hypotheses involving glm/linear model coefficients

I have one independent variable (X) and three dependent variables (W, Y and Z). I am fitting a generalized linear model to each: $Y = g(\alpha X + \epsilon)$ $Z = g(\beta X + \epsilon)$ $W = g(\...
5
votes
2answers
339 views

What is the intuition behind getting a slope distribution in linear regression?

If I understand it correctly, linear regression finds one best fitting line for the given data. It can do it either by using calculus and solving for intercept and slope equations or it can solve it ...
2
votes
1answer
40 views

Can a random slope in a linear mixed model mask the effect of my intervention?

I want to assess the impact of my intervention in a repeated-measures design. I have subject as a random intercept in order to account for the dependence of measurements within subjects: ...
0
votes
0answers
15 views

How to obtain expressions for coefficients from OLS formula?

Consider the standard linear regression model: $y_i = \alpha + \beta D_i + e_i$ where the coefficients are defined by linear projections and $D_i$ is a dummy variable. In the population, the ...
1
vote
1answer
37 views

Why to remove skewness from the data?

I am a beginner in statistics and I read an article which said "Linear algorithms love normally distributed data". I wanted to know why do we have to transform the variable having skewness.
0
votes
1answer
38 views

Can you help me to understand this deduction for proving Naive Bayes is a Linear Classifier?

In this tutorial on Naive Bayes Classififer in section 1.1, the author proved naive bayes is a linear classifier. Consider binary classification where $y=0$ or $1$. Our classification rule with ...
0
votes
1answer
35 views

forecast beyond the data set in r

I am forecasting GDP growth for my dissertation. The data is from 1984-2017 (quaterly values). I am using the following code to fit and forecast a linear regression model. ...
0
votes
0answers
8 views

Should geographic location always be included as a random model effect?

Under what sort of experimental conditions and/or objectives might someone be justified in modeling geographical location as a fixed effect (assuming that most times location is included as a random ...
0
votes
0answers
15 views

Sensitivity study: measuring the effects of free parameters on performance measures of simulation

I am running a series of computer simulations in which I am outputting several performance measures. I want to know how much of the variability of these measures is due to the free parameters of the ...
1
vote
1answer
35 views

Linear regression with low variations in the explanatory variables

Below is a scatter plot of some data with the best fit OLS line. From $R^2$ we can see it is a quite a poor fit. But is there a way to predict before running the regression that the results won't be ...
1
vote
0answers
50 views

Do general linear models actually fit a linear function to the data?

If I understand correctly, linear regression involves fitting a linear function to a dataset and computing the error in that function. There's also a notion of "general linear model" which seems to be ...
1
vote
0answers
41 views

In regression, when partitioning SS among predictors, what determines which predictors get the SS that can be attributed to more than one predictor?

In regression analysis, predictors sometimes correlate (and in my field, psychology, they always do; often because they are simply measurements of the same aspects of human psychology). If predictors ...
2
votes
0answers
23 views

Rss and sample variance indipendence in simple linear regression

Suppose that $ (X_1 ,Y_1...X_n,Y_n) $ is an i.i.d. random sample from a simple homoschedastic linear model $Y=\alpha +\beta X+e $ , with $e|X \sim N(0,\sigma_e^2)$. I want to understand if $ \frac{...
0
votes
0answers
23 views

Studentization ad pivotal quantity for the slope in simple linear regression

Take the usual linear model: $$ Y=\alpha+\beta X + e$$ In good hypothesis (IID sample, normal error assumption $e \sim N(0,\sigma_e^2) $ and homoschedasticity) the distribution of the sample slope $\...
0
votes
1answer
55 views

Coding categorical variables for linear regression and random forest, factors/characters

I am a newbie in Data Science so that do not judge me for this questions. Making a regression model (linear model, lm_model) with numeric and categorical variables, I realized that Estimate ...
6
votes
1answer
69 views

Fitting a curve OVER OR UNDER a set of points

I want to fit a curve $f(x) = mx+b$ on my data points $x_1, \ldots, x_N$ using linear regression with a single predictor. However, the cost function is not even, rather, it has different weights on ...
0
votes
0answers
20 views

What's the difference between “cluster” option in linear regression and mixed models?

Is there a difference in estimates from linear regression with "cluster" option and linear mixed model? Take for example this scenario : "...dataset contains data on 400 schools that come from 37 ...
1
vote
1answer
35 views

Multiple linear regression - interpretation diagnostic plots

I am learning so bear with me. Aim: I am trying to figure out if my data fit the criteria for multiple linear regression. Context: My model has two numeric and four categoric variables. ...