Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

learn more… | top users | synonyms

0
votes
0answers
21 views

Transforming data to standard normal

I have a residual expression matrix upon which I want to conduct an eQTL analysis using a linear model: I know that a normal distribution is required to use a linear model; my question is can I use ...
1
vote
0answers
13 views

Coefficient in linear regression changes drastically if additional variables are added. Why? [duplicate]

n <- 100 x2 <- 1 : n x1 <- .01 * x2 + runif(n, -.1, .1) y = -x1 + x2 + rnorm(n, sd = .01) summary(lm(y ~ x1))$coef Coefficients (all significant): ...
-1
votes
0answers
7 views

In the case of an ELM (or linear regressor), how is the numerically stable way to forget/unlearn an instance?

Would it be enough to just increment* training with the instance we want to be forgotten, but with a negative output? If we perform the process many times, will the network deteriorate? *: using ...
1
vote
0answers
31 views

When do improper linear models get robustly beautiful?

Improper linear models are described from time to time in the literature. In general, such models can be described as $$ y = a + b \sum_i w_i x_i + \varepsilon $$ what makes them different from ...
0
votes
0answers
6 views

Test for Regression Slope different from zero in R [duplicate]

I know how to do the Test for Regression Slope in R when the H0: b1=0 H1: otherwise but how do I do the Test for Regression Slope when i want to check H0: b1=1 H1: otherwise is it possible? ...
5
votes
1answer
143 views

Linear OLS v Mixed-Effects Model with Correlated Regressors

Reading this post by @gung brought me to try to reproduce his superb illustrations, and led ultimately to question something I had read or heard, but that I'd like to understand more intuitively: Why ...
0
votes
1answer
36 views

How to combine multiple time series or linear models?

What would be the best suited method to analyze the following: ...
0
votes
0answers
5 views

Loops for covariates: making SPSS choose covariates according to predictor in linear regression [migrated]

In my data set, I have several independent variables (here named "predictor1", "predictor2" etc.) and several dependent variables ("outcomeA", "outcomeB" etc). Furthermore, I have several covariates ...
0
votes
0answers
5 views

Calculating “consistency” of variables in a dataset

I know am sorry in advance for the layman question :) I will try to demonstrate by an example: I have a dataset of items in a clothes store with the (dummy) variables: ...
1
vote
2answers
56 views

Do we need gradient descent to find the coefficients of a linear regression model

I was trying to learn machine learning using the coursera material Andrew Ng uses gradient descent algorithm to find the coefficients of the linear regression model that will minimize the error ...
0
votes
1answer
24 views

Regression imputation of missing data

Suppose a two-way experiment with interaction. Is it correct to estimate the missing values by OLS, input those values in the data (fill the blanks) and now perform a polynomial (or any kind of) ...
6
votes
1answer
365 views

Understanding QR Decomposition

I've got a worked example (in R), that I'm trying to understand further. I'm using Limma to create a linear model and I'm trying to understand what's happening step by step in the fold change ...
5
votes
1answer
179 views

Bayesian regression full conditional distribution

I have a problem with the derivation of the full conditional distribution of the regression coefficients in a simple Bayesian regression. The source of the following equations is: Lynch (2007). ...
2
votes
2answers
56 views

Solve $X^TX b = a$ for $b$ using $XX^T$ for a short and wide matrix $X$

I have a matrix $X$ of dimensions $n \times p$ and a fixed $p$-dimensional vector $a$, with $p \gg n$. How can I efficiently solve a problem of the following form? $$X^TXb = a$$ Perhaps using the $n ...
0
votes
0answers
22 views

Regression model for Cumulative data in R

I am having a daily data for 3-4 months and another variable which is the cumulative sum. It starts with some value on the first day and it keeps on adding and at the end of 3 months, it would be sum ...
1
vote
1answer
24 views

How to check if the correlation between two continous variables is influenced by a categorical factor?

I have a data frame (df) where I see correlation between two continuous variables (c1 and c2). I need to know whether the observed correlation between the two variables differs between groups, which ...
2
votes
1answer
38 views

Forming a prior based on the solution to a linear system

On p. 115 of the 4th edition of Machine Learning a Probabilistic Perspective, we have the following: Let $\epsilon\sim N(0,\frac{1}{\lambda}\text{I})$ and let $L$ be a matrix of dimension ...
1
vote
0answers
61 views

Gini Coefficient - Variable Importance Measure

There is a whitepaper for selecting important variables in a linear regression model. The URL of the whitepaper is http://support.sas.com/resources/papers/proceedings15/3242-2015.pdf . It explains ...
4
votes
2answers
52 views

How to generate confidence bands for $\hat{Y}$

Suppose I run a linear regression model. I am interested in generating prediction intervals. The predicted values are easy to compute, but how can I compute the standard deviations for each of the ...
0
votes
0answers
34 views

Linear regression, $R^2$

While computing $R^2$ for the test data set, what mean value should be used - the mean from the training and test or just the test data set?
2
votes
1answer
24 views

Do I need to adjust the degrees of freedom returned by pool.compare() in MICE?

I am analyzing a multiply imputed dataset produced from the MICE package in R. To assess the overall significance of my linear model, I am using pool.compare() to compare my "full" model to an ...
0
votes
0answers
21 views

How to interpret the direction of a covariate's effect in a 2-way ANOVA?

I use SPSS's GLM (general linear model) to analyze an data set based on a 2x2 between-subjects experimental design. The design has two factors: a). congruence (yes or no) and d). primed image ...
0
votes
0answers
23 views

A linear model with prior information

Suppose I have this experimental data: I have measurements of drug response from patients (let's say its blood pressure). Specifically, I have measurements after being treated with drug A (30 ...
0
votes
0answers
21 views

Multiple Comparison Correction for Linear Regressions with Dependent Variables

I could use some help clarifying the use of multiple comparison corrections for a series of linear regressions when some of the variables being tested are calculated from each other. Toy problem to ...
2
votes
1answer
72 views

Regression performed using Principal Component Analysis

I have a dataset consisting of 10 correlated variables. I need to explain a response variable using these 10 variables, so I am using PCA to reduce dimensionality. Say, I use the first 3 components ...
0
votes
0answers
16 views

Linear Model learnt on several frequency

Sorry for the unclear title but I did not know how to call this POST, I'll try to be clearer. I have data that I need to forecast with external variables. This data is a time series available on a ...
2
votes
1answer
28 views

linear path models vs. pls path models (structural equation models)

Assume we have the following linear path model: Structural (inner) model: $Y_{1} = \beta_{1}Y_{2}+\theta_{1}\delta$ Measurement (outer) model: $X_{1} = \lambda_{1}*Y_{1}+\epsilon_{1}\delta$ ...
0
votes
1answer
42 views

Deleting Outliers in a regression model

Working on a linear regression problem in R, I created a first model flights_lm = lm(freq~dist+capa+nbrt+depf+lcco+prbi) where freq is frequency, dist is ...
0
votes
1answer
39 views

What is criterion for Breusch-Pagan test?

Could someone explain to me what is criterion for interpretation of Breusch-Pagan test? I have applied ncvTest test from the package car in R on a simple linear regression with one predictor variable ...
2
votes
1answer
24 views

Regression with Probabilitistic Explanatory Variable

Let X be a categorical variable. Instead of knowing for certain whether a particular observation is equal to a given level of X, I have a probability distribution over the possible values of X. So, ...
0
votes
0answers
32 views

What measure to use in finding the best linear model

I have a bunch of linear models (say 20 of them), and a bunch of datasets (e.g. 400). I wrote a code in R so that each dataset is exposed to each model, and the goal is to select the best model that ...
1
vote
1answer
26 views

Coefficient of determination increases with the number of regressors

Suppose we deal with the linear regression model $Y=X\beta+\epsilon$, where $X$ is determined matrix, $\beta$ - the vector of coefficents, $\epsilon$ - the vector of errors. I often meet the ...
0
votes
1answer
28 views

Testing OLS Model Prediction Accuracy?

How can I test the accuracy of my ordinary least squares model? Is it a simple comparison between the predicted values of my test set and their actual values (with perhaps a maximum threshold of ...
0
votes
0answers
27 views

Should I delete the intercept [duplicate]

I'm working on a small project where I need to create a multivariate linear regression model to predict the frequency of some airline companies. I'm a bit confused as I don't know if I have to remove ...
3
votes
2answers
188 views

Least Squares Regression Step-By-Step Linear Algebra Computation

As a prequel to a question about linear-mixed models in R, and to share as a reference for beginner/intermediate statistics aficionados, I decided to post as an independent "Q&A-style" the steps ...
-1
votes
2answers
80 views

Regression analysis low R2 value - Result interpretation

When I run linear regression on my test data I get the following report: You can find the test data in here. The graph of actual vs predicted looks like: I would like to know if this is fairly a ...
0
votes
1answer
24 views

How to finalize the factors to be considered for simple linear regression

I have a list of 100 variables from which i want to filter down to the most important variables to be considered for applying simple linear regression? Is there a method for this? Or do i have to ...
0
votes
0answers
30 views

Understanding lm() function in R with weights

Consider the following simplified dataset (sales and percent of color-type sales by region): ...
3
votes
3answers
111 views

linear regression - modelling explanatory variables which depend on each other

I'm trying to estimate the value of an apartment, by doing a regression through similar apartments. The regression model looks now like this ...
3
votes
1answer
48 views

How can we have non-random patterns in the plot of simple linear regression residuals vs the predictor variable?

A) When considering a simple linear regression model, it is important to check the linearity assumption. Graphing the residuals vs the predictor variable can often give a good idea of whether or not ...
0
votes
0answers
50 views

Correct interpretation of linear regression coefficient?

Let's say I'm running a survey and I ask "Are you confident in the economy, yes or no?" Then I ask the respondent "are we spending too much or not too much on scientific funding? I let the answer ...
1
vote
1answer
50 views

Show $J(I-H) = 0$

This is in multiple linear regression. Given $m, n \in \mathbb{N}$ and matrices $X \in \mathbb{R}^{m \times (n+1)} (m > n + 1), H = X(X'X)^{-1}X' \in \mathbb{R}^{m\times m}$, the hat matrix, $, I = ...
0
votes
0answers
10 views

Weighted linear model with few different values

I am trying to come out with a model but I am encountering many problems as it is very unbalanced. The model is a weighted linear model where one of the points has a much higher weight than the rest. ...
0
votes
0answers
12 views

How to find the distribution of Betas of a Single/Multiple Linear Regression model?

How would I go about finding the distribution of betas for a multiple linear regression? That is, a matrix with the same dimensions as the data whose column means are the regression coefficients ...
3
votes
1answer
109 views

Linear regression with violated assumptions

I am trying to find out the determinants of cognitive function. The outcome variable is the mini–mental state examination which is a 30 point questionnaire response that has score values from 0 to ...
2
votes
2answers
136 views

How to verify a linear model?

Given a dataset and liner model, how can I verify its sufficient quality? ...
0
votes
0answers
22 views

Selecting proper Linear Model with interaction considered

Data consists of Temp(3lv Factor), Water(num), Fert(num), and Growth(num). Considered model is LM with DV: Growth, IV: Temp with Water and Fert as covariate. Fitted 2-Factor ANCOVA (Full Model) and ...
3
votes
2answers
131 views

Best Subset Selection Questions

I am reading Introduction to Statistical Learning and I have a question about the Best Subset Selection Algorithm. The algorithm goes like this: Suppose there are $p$ predictors, the aim is to ...
0
votes
0answers
43 views

How to exploit relationships between independent variables?

Data: Each instance (representing a document) is a bag-of-entities (like BOW, except they're Wikipedia entities instead of words), so each feature is a binary or tfidf-like score based upon the ...
1
vote
0answers
26 views

Data Smart Book Exercise [closed]

In the book, Data Smart, by John Foremann, at the end of chapter 4, the reader is asked to optimize a model where only 95 % of the sampled scenarios are met by the model constraints. The relevant ...