Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

learn more… | top users | synonyms

1
vote
1answer
12 views

Model matrix for linear model with multiple contrasts

I don't fully understand the concept of a model.matrix. Assume I have the following experimental design: ...
0
votes
0answers
31 views

Linear regression interpretation

Lets say I run a port. There's this ship coming with watermelons and melons. They have multiple containers, which I cannot open, with mixed watermelons and melons. From the source port, the ...
0
votes
0answers
17 views

Strange equation for standard error of estimate calculation

I've come across the following algorithm to calculate the standard error of estimate (residual standard error): RSE=SQRT( (sum(Y^2)-b0*sum(Y)-b1*sum(XY))/(count-df) ) I have searched high and low ...
0
votes
1answer
48 views

How we can compare two coefficients of one linear regression?

I have this regression model, $$\hat{Y}=\hat{a}X_1+\hat{b}X_2+\hat{c}$$ Both $X_1$ and $X_2$ are significant at 0.01 level. $X_1$ and $X_2$ have a same unit. Now I want to find a test that tells me ...
4
votes
2answers
56 views

Can I use a linear model estimated with lasso for intepretation?

If all the assumptions are correct, a linear model can be used for interpretation. It is possible to understand if a variable has a significant effect on the response and if so, it is also possible to ...
2
votes
0answers
26 views

Interpreting linearity in regression when there are outliers

I am trying to determine whether this regression meets all of the assumptions one needs to adhere to when carrying out a multiple linear regression. In looking at the residual plots below, it seems to ...
1
vote
0answers
15 views

to keep or not to keep… block effect when not significant?

The question is in the title. I could not find an answer to this common question after searching on google nor on the StackExchange website, excepted this but the provided answer is not definite ...
0
votes
1answer
29 views

Imputation and linear regression analysis paradox

Missing values, especially in small datasets, can introduce biases into your model. There are several data imputation methods (MICE, Amelia II), which use EM algorithms to "fill in" the missing ...
0
votes
0answers
26 views

How to account for order/ repeated measures in a linear mixed model?

My data involves: 7 subjects who all received three different treatments (A, B and C), separated by one week Each subject received treatment A first, although treatments B and C were completed in a ...
0
votes
0answers
18 views

(R) Calculate the “adjusted r^2” or its equivalent in a linear model that controls for a categorical variable

So I have a dataset that includes 3 columns: a categorical (discreet) variable (which is an input) and two outputs continuous variables. Both outputs are affected by that input. I want to see the ...
0
votes
0answers
10 views

Few predictor values, many individuals

I am using linear models on a dataset structured like the table below. Columns are 0 and 1 only. I want to be able to predict the value in the cell ij for example, using a predictor variable. ...
1
vote
0answers
21 views

Variance as a function of the mean. Why does this affect linear regression? [duplicate]

I'm currently studying university level statistics and I'm struggling to wrap my head around the concept of variance as a function of the mean. How does this affect linear regression and why does it ...
0
votes
0answers
24 views

Linear regression: what will happen if the effect of a predictor is removed from response

Given a response $Y$ and explanatory variables $X_1, \dots, X_n$, suppose we use least squares estimate to obtain coefficients $\beta_0, \beta_1, \dots, \beta_n$ from the model $$Y = \beta_0 + \beta_1 ...
0
votes
1answer
48 views

R: linear mixed effects plus MCMC estimation

In a paper I wrote a few years ago, I wrote the following: All results were analyzed using linear mixed models effects, with Subjects and Items as random effects. I present p-values estimated from ...
1
vote
0answers
11 views

Effective Linear Regression for datasets with missing values in explanatory (independent) variables

I have an econometric dataset of countries consisting of features such as GDP, GDP per capita, internet penetration rate, life expectancy, poverty etc. There are a ...
2
votes
0answers
12 views

Systematic/measurement error on a linear regression

Suppose I have a set of data ${(x_i,y_i)}$ in which the uncertainty in the measurements ${(\Delta x_i,\Delta y_i)}$ (which come from the propagation of systematic errors from the measurement ...
1
vote
1answer
42 views

Why does “orthogonality” matter when using poly(x,degrees=1)?

I think I understand why orthogonality matters when doing regression with polynomial fits (so that the linear and quadratic, cubic, etc... can be evaluated independently). However, I don't understand ...
5
votes
4answers
225 views

What is the upper bound on $R^2$ ? (not 1)

It is a well known fact that if you add additional independent variables in a linear regression, the $R^2$ of the new model is at least as large as the previous model. So you obtain a lower bound for ...
0
votes
1answer
69 views

Could one say that one dataset is distributed more normally than another?

I am trying to fit a simple linear model: experiment ~ calculated_1. From the basic model, I get residuals_1. And I know that ...
0
votes
2answers
18 views

Why OLS F Statistic close to one when there is no relationship?

I might be missing something obvious here. In linear regression, F statistic is defined as (explained variance / p) / mean squared error, where p is number of independent variables. When there is no ...
0
votes
0answers
23 views

Transforming data to standard normal

I have a residual expression matrix upon which I want to conduct an eQTL analysis using a linear model: I know that a normal distribution is required to use a linear model; my question is can I use ...
1
vote
0answers
13 views

Coefficient in linear regression changes drastically if additional variables are added. Why? [duplicate]

n <- 100 x2 <- 1 : n x1 <- .01 * x2 + runif(n, -.1, .1) y = -x1 + x2 + rnorm(n, sd = .01) summary(lm(y ~ x1))$coef Coefficients (all significant): ...
1
vote
0answers
38 views

When do improper linear models get robustly beautiful?

Improper linear models are described from time to time in the literature. In general, such models can be described as $$ y = a + b \sum_i w_i x_i + \varepsilon $$ what makes them different from ...
0
votes
0answers
6 views

Test for Regression Slope different from zero in R [duplicate]

I know how to do the Test for Regression Slope in R when the H0: b1=0 H1: otherwise but how do I do the Test for Regression Slope when i want to check H0: b1=1 H1: otherwise is it possible? ...
5
votes
1answer
152 views

Linear OLS v Mixed-Effects Model with Correlated Regressors

Reading this post by @gung brought me to try to reproduce his superb illustrations, and led ultimately to question something I had read or heard, but that I'd like to understand more intuitively: Why ...
0
votes
1answer
38 views

How to combine multiple time series or linear models?

What would be the best suited method to analyze the following: ...
0
votes
0answers
8 views

Calculating “consistency” of variables in a dataset

I know am sorry in advance for the layman question :) I will try to demonstrate by an example: I have a dataset of items in a clothes store with the (dummy) variables: ...
1
vote
2answers
83 views

Do we need gradient descent to find the coefficients of a linear regression model

I was trying to learn machine learning using the coursera material Andrew Ng uses gradient descent algorithm to find the coefficients of the linear regression model that will minimize the error ...
0
votes
1answer
25 views

Regression imputation of missing data

Suppose a two-way experiment with interaction. Is it correct to estimate the missing values by OLS, input those values in the data (fill the blanks) and now perform a polynomial (or any kind of) ...
6
votes
1answer
384 views

Understanding QR Decomposition

I've got a worked example (in R), that I'm trying to understand further. I'm using Limma to create a linear model and I'm trying to understand what's happening step by step in the fold change ...
5
votes
1answer
189 views

Bayesian regression full conditional distribution

I have a problem with the derivation of the full conditional distribution of the regression coefficients in a simple Bayesian regression. The source of the following equations is: Lynch (2007). ...
2
votes
2answers
56 views

Solve $X^TX b = a$ for $b$ using $XX^T$ for a short and wide matrix $X$

I have a matrix $X$ of dimensions $n \times p$ and a fixed $p$-dimensional vector $a$, with $p \gg n$. How can I efficiently solve a problem of the following form? $$X^TXb = a$$ Perhaps using the $n ...
0
votes
0answers
24 views

Regression model for Cumulative data in R

I am having a daily data for 3-4 months and another variable which is the cumulative sum. It starts with some value on the first day and it keeps on adding and at the end of 3 months, it would be sum ...
1
vote
1answer
25 views

How to check if the correlation between two continous variables is influenced by a categorical factor?

I have a data frame (df) where I see correlation between two continuous variables (c1 and c2). I need to know whether the observed correlation between the two variables differs between groups, which ...
2
votes
1answer
38 views

Forming a prior based on the solution to a linear system

On p. 115 of the 4th edition of Machine Learning a Probabilistic Perspective, we have the following: Let $\epsilon\sim N(0,\frac{1}{\lambda}\text{I})$ and let $L$ be a matrix of dimension ...
1
vote
0answers
93 views

Gini Coefficient - Variable Importance Measure

There is a whitepaper for selecting important variables in a linear regression model. The URL of the whitepaper is http://support.sas.com/resources/papers/proceedings15/3242-2015.pdf . It explains ...
4
votes
2answers
58 views

How to generate confidence bands for $\hat{Y}$

Suppose I run a linear regression model. I am interested in generating prediction intervals. The predicted values are easy to compute, but how can I compute the standard deviations for each of the ...
0
votes
0answers
35 views

Linear regression, $R^2$

While computing $R^2$ for the test data set, what mean value should be used - the mean from the training and test or just the test data set?
2
votes
1answer
29 views

Do I need to adjust the degrees of freedom returned by pool.compare() in MICE?

I am analyzing a multiply imputed dataset produced from the MICE package in R. To assess the overall significance of my linear model, I am using pool.compare() to compare my "full" model to an ...
0
votes
0answers
31 views

How to interpret the direction of a covariate's effect in a 2-way ANOVA?

I use SPSS's GLM (general linear model) to analyze an data set based on a 2x2 between-subjects experimental design. The design has two factors: a). congruence (yes or no) and d). primed image ...
0
votes
0answers
24 views

A linear model with prior information

Suppose I have this experimental data: I have measurements of drug response from patients (let's say its blood pressure). Specifically, I have measurements after being treated with drug A (30 ...
0
votes
0answers
29 views

Multiple Comparison Correction for Linear Regressions with Dependent Variables

I could use some help clarifying the use of multiple comparison corrections for a series of linear regressions when some of the variables being tested are calculated from each other. Toy problem to ...
2
votes
1answer
76 views

Regression performed using Principal Component Analysis

I have a dataset consisting of 10 correlated variables. I need to explain a response variable using these 10 variables, so I am using PCA to reduce dimensionality. Say, I use the first 3 components ...
0
votes
0answers
17 views

Linear Model learnt on several frequency

Sorry for the unclear title but I did not know how to call this POST, I'll try to be clearer. I have data that I need to forecast with external variables. This data is a time series available on a ...
2
votes
1answer
36 views

linear path models vs. pls path models (structural equation models)

Assume we have the following linear path model: Structural (inner) model: $Y_{1} = \beta_{1}Y_{2}+\theta_{1}\delta$ Measurement (outer) model: $X_{1} = \lambda_{1}*Y_{1}+\epsilon_{1}\delta$ ...
0
votes
1answer
44 views

Deleting Outliers in a regression model

Working on a linear regression problem in R, I created a first model flights_lm = lm(freq~dist+capa+nbrt+depf+lcco+prbi) where freq is frequency, dist is ...
0
votes
1answer
44 views

What is criterion for Breusch-Pagan test?

Could someone explain to me what is criterion for interpretation of Breusch-Pagan test? I have applied ncvTest test from the package car in R on a simple linear regression with one predictor variable ...
2
votes
1answer
24 views

Regression with Probabilitistic Explanatory Variable

Let X be a categorical variable. Instead of knowing for certain whether a particular observation is equal to a given level of X, I have a probability distribution over the possible values of X. So, ...
0
votes
0answers
34 views

What measure to use in finding the best linear model

I have a bunch of linear models (say 20 of them), and a bunch of datasets (e.g. 400). I wrote a code in R so that each dataset is exposed to each model, and the goal is to select the best model that ...
1
vote
1answer
27 views

Coefficient of determination increases with the number of regressors

Suppose we deal with the linear regression model $Y=X\beta+\epsilon$, where $X$ is determined matrix, $\beta$ - the vector of coefficents, $\epsilon$ - the vector of errors. I often meet the ...