Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

0
votes
0answers
9 views

How can I interpret relative and absolute income of both partners in one regression?

Suppose you want to examine the effect of income on the amount of housework for women. Does it make sense to include both relative income (compared to partners income) and absolute income of BOTH ...
2
votes
2answers
42 views

In linear regression, are the noise terms independent of the coefficient estimators?

In the Wikipedia article on the bias-variance tradeoff, the independence of the estimator $\hat f(x)$ and the noise term $\epsilon$ is used in a crucial way in the proof of the decomposition of the ...
0
votes
0answers
25 views

Comparing linear sets of data

I'm trying to validate the use of different sample tubes in a pharma environment. We have an established method with a linear calibration curve and QCs at three concentrations within the curve range. ...
0
votes
0answers
29 views

Regularization of linear regression problem [duplicate]

Consider a vector $a \in R^n$. I want to know how I can find analytically the solution of the following optimization problem: $x^* = argmin_{x \in R^n} f(x)$, where $f(x) = ||x-a||_{2}^2 + \lambda ||x|...
0
votes
0answers
9 views

Least squares and identificability condition

Let a discrete-time system (which is minimal) with input $u \in \mathbb{R}^m$ and state $x \in \mathbb{R}^n$ be $ x_{k+1} = [x^T_k \quad u^T_k]\begin{bmatrix} A^T \\ B^T \end{bmatrix} + v^T_k $ ...
0
votes
1answer
16 views

Cluster-robust standard errors in panel data analysis

In a simple panel data analysis with data on 64 firms over 8 years, I use cluster-robust standard errors (at the firm level) to evaluate significance of coefficients. I observe important differences ...
1
vote
1answer
30 views

How do you deal with “nested” variables in a regression model? in R

A conceptual solution for this scenario has been posted in: How do you deal with "nested" variables in a regression model? Problem is I am having trouble using this solution in R - glm() ...
0
votes
0answers
10 views

When binary predictor = 0, all other predictors = NA - what model structure do I need?

I have a genetics dataset which I want to build a model for. The dependent variable y is case or controls status (binary). The first independent variable x1 is whether or not they have a variant in ...
0
votes
0answers
18 views

Additional Property of Singular Value Decomposition

I am new to SVD so forgive me if the question is trivial. Following is my question. If I have two sets of linear equations, Y1 = T1.X Y2 = T2.X where T1 and T2 are mxn rectangular matrices. Now let'...
0
votes
1answer
34 views

math behind polynomial regression

I am creating a polynomial regression model with Python sci kit learn package, and I was wondering how I can use the predict features in machine learning ...
0
votes
1answer
51 views

Curved regression lines

I had already asked a similar question here, but I'm experiencing the same problem for a different data-set and for a different family of mixed models. My response variable is a binary outcome of ...
1
vote
1answer
51 views

Is this dataset lineary seperable? How can I find it out using (linear) algebra?

I have this dataset: I want to know if it is linearly separable (fully separable). I want to use this rule, but I'm not sure if it's correct: Make $X'$ - matrix with d+1 column of all 1's. Then ...
0
votes
0answers
15 views

Controlling for confounding variables

I have a dataset where some variables need to be controlled for body size and seasonal variation. There is a paper which describes controlling for skeletal size by using the residuals from a linear ...
0
votes
1answer
11 views

Significance of the sum of the main effect and interaction term

Consider a simple linear regression with an interaction term: $Y=b_0 + b_1X +b_2Z+b_3XZ$ where $X$ is continuous and $Z$ is a dummy. I want to find out whether $X$ has a significant impact on $Y$ ...
1
vote
1answer
13 views

Problem with Softmax decision boundary

While reading this paper: sphere face on page 2, it explains that original softmax boundary is given by: $$(W_1 −W_2)x+b_1 −b_2 = 0$$ While trying to obtain the boundary on a toy generated 2D dataset ...
0
votes
0answers
12 views

R: Any benefits of converting factor column to scaled double? [duplicate]

When building a (generalized) linear model, is there any benefit of converting a factor column (i.e. True/False) to a scaled double vector? With booleans, say: ...
0
votes
0answers
21 views

Optimization Technique Needed

I am trying to figure out an optimization technique to below: For context, everything to the left of "New Upcoming Games" is historical data. In my actual dataset I have about 200-300 rows that will ...
1
vote
0answers
16 views

Uniqueness of partial covariance/corrlation if OLS is not unique

Let $X,Y,Z=(Z_1,...,Z_n)$ be random variables. Define the partial covariance between $X$ and $Y$ given $Z$ as: $$\rho_{X,Y \cdot Z} := cov(\hat{X}-X, \hat{Y}-Y)$$ where $\hat{X}$ and $ \hat{Y}$ are ...
1
vote
0answers
33 views

How are confidence intervals calculated for lm in R using predict?

Here, a simple linear model, given x = 98, yields a predicted value of 24.47 with 95% confidence interval [23.97, 24.96]. ...
0
votes
0answers
14 views

Multivariate linear regression - optimizing one coefficient at a time

I have a few questions about solving the multivariate linear regression problem: What is the most popular numerical method used to get the coefficients from multivariate linear regression? I assume ...
3
votes
3answers
44 views

Linear model for positive response variables

Very concise question: if I model a phenomenon which takes only positive values (for example, revenues or production) using the classical OLS, what are the consequences in terms of bias, efficiency ...
0
votes
0answers
23 views

Writing the matrix form of a linear regression model?

I don't know how to write a simple linear regression model in a matrix form.. in our book we are given a table having values of $ x,y,x2,y2,xy.$ . I created a very small example and I attached it as ...
0
votes
0answers
10 views

Linear Regression on Boston Housing Price? [duplicate]

As far my knowledge, Linear Regression assumes that data or columns are normally distributed and doesn’t have multicollinearity amongs the features, But when I apply Shapiro test, it shows that none ...
1
vote
1answer
105 views

Linear Regression Approach For Trends

I'm working on a project with genetics but I think my problem is applicable to general statistics. I want to test frequency (Minor Allele Frequency) of a SNP/variant across 5 age categories to see ...
2
votes
0answers
33 views

Instrumental variable analysis: ivreg (R) vs. naive estimation

Short version of my question: Is it true that the naive, 2-step instrumental variable approach overestimates the standard errors (I expected an underestimation)? Long version: I am working with an ...
0
votes
1answer
24 views

How to fix and understand linearity

The model I have run is a simple multiple linear regression. The model looks like a great fit, but R is telling me otherwise. My question is 3 fold. 1) How do we estimate linearity (not visually) 2) ...
0
votes
0answers
12 views

Introducing random slopes in models with nested random effects

I'm trying to see how latency to emerge (response variable) is varies with time (trials). Individuals (ID) are nested within colonies. The nesting is such that individuals 1-20 belong to colony 1, 21-...
0
votes
0answers
27 views

Calculate $R^2$, $R^2_{adj}$, and F-statistic from $\text{R}$ model summary

I am given the full model, $M_{\tt f}$, with the regression line $$ {\tt response} = \beta_0 + \beta_1{\tt A} + \beta_2{\tt B} + \beta_2{\tt C} + \beta_4{\tt D} + \beta_5{\tt E} + \beta_6{\tt F} + ...
2
votes
0answers
20 views

unconditional prior distribution of g-prior (Bayesian Linear Regression)

Consider a Gaussian regression model $\boldsymbol{Y} =\boldsymbol{X}\boldsymbol{\beta} + \boldsymbol{\epsilon},\quad \boldsymbol{\epsilon}\sim N( \boldsymbol{0},\sigma^2I)$ I put a Hyper-g Priors (...
0
votes
1answer
22 views

Logarithmic or square root transformation for econometric modeling

I am doing econometric research on firm financial ratios. Using linear panel data modeling, I am going to transform some predictors in order to reduce variance. At this regard, I am uncertain about ...
2
votes
1answer
40 views

Introducing random slopes in nested model improves model fit but residuals variances become unequal

I have measured boldness scores (continuous variable) across time (trials) for individuals (ID) within colonies (colony). The data is coded such that individuals 1-30 belong to one colony, 31-60 to ...
0
votes
0answers
13 views

Sigma interpretation in Bayesian Linear Model?

I have two question concerning my output of my bayesian linear regression. 1) I have all beta posterior and obviously, having used a prior for Sigma, i have a posterior for Sigma too, but what can i ...
0
votes
0answers
25 views

Solving correlation between explanatory variables using instrumental variables

I am currently stuck on a task where I am interested in estimating the production function for agricultural output as follows: \begin{equation} y_{i} = x_{i}\beta + \alpha_i + \epsilon_{i} \end{...
1
vote
1answer
58 views

Assumptions of linear mixed model not met

I have a repeated measures dataset with which I'm testing if individuals are consistent in their boldness scores (continuous variable) over time (trials). Towards this, I generated linear mixed ...
1
vote
1answer
29 views

GLM standardisation with quadratic terms

I had understood that using linear transformations, such as centering and scaling, of predictor variables in GLMs does not affect the t/z-values, and thus nor the p-values (except for the intercept). ...
0
votes
0answers
15 views

Transforming panel data OLS into cross-sectional data model

I am currently stuck on a task where I am interested in estimating the production function for agricultural output using panel data as follows: \begin{equation} y_{it} = x_{it}\beta + \alpha_i + \...
0
votes
0answers
29 views

Calculation of standard deviations for regression coefficient standardization when intercept is not present

I took the code of function lm.beta::lm.beta in R and wrote it in this way to be more understandable (and commented the lines I'm not interested in): ...
1
vote
0answers
21 views

Discrete-time survival model as linear probability model

I am trying to fit a discrete-time survival model. This is easily done by fitting a logit (or probit or complementary log-log) model of failure on time and on the covariates of interest. Standard ...
0
votes
0answers
14 views

BIC under linear mixed model

I know usually, we do not use Bayesian Information criterion(BIC) for model selection if we have a linear mixed model (problems involve like the sample size in the linear mixed model is not well ...
0
votes
1answer
35 views

The Standard Error in R language lm (linear regression) is Standard Deviation or Standard Error of the Mean?

When we are estimating the coefficient in R, "Std Err" will be produced by "lm". Is it Std or SEM? How is it calculated? Run lm function with any R data set will give an example.
9
votes
2answers
250 views

In linear regression, why should we include quadratic terms when we are only interested in interaction terms?

Suppose I am interested in a linear regression model, for $$Y_i = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_1x_2$$, because I would like to see if an interaction between the two covariates have an ...
0
votes
1answer
62 views

R: how is the Pr(>|z|) in the results of glm.fit calculated and why?

I've been searching but I can't find anywhere an explanation of how the Pr(>|z|) column is calculated in the results of R's glm.fit function. I would really appreciate: a) an explanation so I can ...
0
votes
1answer
37 views

Show that target variable is gaussian in simple linear regression

Given the simple linear regression model $$ y_i = \beta_0 + \beta_1 x_i + \epsilon_i$$ where $\beta_0$ and $\beta_1$ are fixed paramters, $x_i$ are nonrandom variables and the errors $\epsilon_i$ ...
0
votes
2answers
35 views

Regression: Interactions terms of factors with several levels (interpretation)

I'm running the following linear model in R: lm(formula = Valence ~ StatusOfMandarin * Condition, data = d_afraid, na.action = na.omit) My data is as follow ...
2
votes
0answers
68 views

What is the bias in PCA regression?

Assuming we have $n$ principal components and use $k<n$ for a linear regression. What is the bias of the l.s.e estimator $\hat \beta$ for the slope parameter using just these k components of the ...
1
vote
1answer
128 views

Regression: Insignificant Intercept [duplicate]

I ran a regression and the intercept is statistically insignificant (the p-value is greater than 0.05). I tried to look in some textbooks as to how to handle this scenario but I am still unsure. One ...
0
votes
0answers
17 views

Reference for implementation of LDA in sklearn

Is there a reference explaining the implementation of sklearn.discriminant_analysis.LinearDiscriminantAnalysis._solve_svd? More specifically, I am curious about ...
0
votes
0answers
74 views

Multiple regression - Coefficients testing

I have a multiple regression model $Y=12.45+0.072X_1-0.15X_2+0.03X_3+0.17X_4$ with $R^2=0.972$ A second regression model is given as follows $Y=13.77+0.072X_1$ with $R^2=0.855$ The number of the ...
0
votes
0answers
23 views

Proof of robustness of ``Linear Discriminant Analysis"

Assume one has been given $N$ data points in $\mathbb{R}^{d_1}$ each of which comes with a label from some set $\{1,\ldots,q\}$. Now I guess the claim is that doing (Linear Discriminant Analysis) LDA ...
1
vote
0answers
11 views

uncertainty in dependent variable at single covariate

I have a dataset consisting of responses of a dependent variable measured at the same independent variables over multiple trials. It looks something like trial $i$: $(x^{(i)}_1, y^{(i)}_1) = (1.0,\...