Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

learn more… | top users | synonyms

1
vote
0answers
14 views

Critical point for 'hat values' in a normal linear model

I have a normal linear model $y=X \cdot \beta$, where $\beta$ is a $13$-dimensional vector. I want to see if there are any points (I have $287$ data points in total) I can throw out. I want to use hat ...
0
votes
1answer
14 views

Is regression line (for a simple “Y given X” regression, no interaction etc) always unique?

Is it possible for more than one "linear regression" line to fit a given set of points? (i.e.... "least squares" is minimised equally in both cases) I'm assuming a simple one-variable regression in ...
0
votes
0answers
6 views

A linear model for testing difference in pairs value between two groups

I have the following experimental design: Values of expression of 3 genes taken from 3 different patients and 3 different controls. R code for generating these data: ...
0
votes
0answers
21 views

Scale of weights in Gradient descent for Linear Regression

Let's take a very simple example where one wants to get a linear model of weight in kg of people given their height in cm as input. We then have two weights: the bias $w_0$ and $w_1$, and we want a ...
-4
votes
0answers
31 views

Best linear fit [duplicate]

Which one is the best fit according to these information? fit1 or fit2. I am NOT working with this in an academic context so explanations are not important to me.
0
votes
0answers
6 views

Breusch Pagan test vs graph

I have data where there is one dependent variable (X) and one dependent variable (Y). When I fit a linear model to this and look at residuals vs X, I see that there is heteroscedasticity. Whereas, ...
3
votes
1answer
59 views

compare two linear models. Linear regression

I have made two linear regressions to estimate y and I get this results: One: ...
3
votes
0answers
32 views

Is there a model with additive effects for always positive dependant variable?

When modeling a dependant variable always positive and continuous, models as log-transformed linear model or GLM with log link are generally used. The log-transformed linear model is : ...
3
votes
2answers
56 views

Does cross-validation on simple or multiple linear regression make sense?

Does it make sense to apply train-test split or k-fold cross-validation to a simple linear regression model or multiple linear regression model? I'm really confused about this because I saw this ...
0
votes
0answers
16 views

Linear model in R. Basics; how to read and understand the table [duplicate]

I am asked to fit a linear model for some data that my teacher has given. The first 5 lines of the R code is given by my teacher. X1 has 4 kind of outcome (1, 2, 3 and 4) and x2 has 6. I believe I ...
0
votes
1answer
31 views

Evaluating fitlm (linear model) in matlab on a separate test set

I've built a linear model with Matlab using fitlm to predict the next value in a series of doubles. Now I would like to test this model on a different dataset so I get accuracy, p-value etc. I've ...
0
votes
1answer
33 views

Correlation between fitted and observed values in linear model

This is abour linear model. How do I show/calculate the Correlation between the fitted values and observed values?
0
votes
0answers
38 views

Effect of input dimension on generalization

I have been asked to derive a generalization bound for sparse linear classifiers in an assignment and to explain how the bound is affected by the number of features. So I was wondering if someone ...
0
votes
0answers
30 views

Do I need a special kind of linear regression for aggregated data?

I have two separate databases on individuals. But these individuals are not both present in the two databases. So I decided to aggregate them into area-level data (such as State-level). One of the ...
0
votes
2answers
42 views

Output of linear model in R [closed]

I am learning R for statistical analysis. Earlier, also I've fitted some linear models and they worked well. But this time I am getting a weird problem. For data set in file "Auto.csv", I am trying to ...
0
votes
0answers
10 views

Choice of the objective function in linear model

I'm reading the book "Data Analysis and Data Mining" by Adelchi Azzalini and Bruno Scarpa. At the end of chapter 4, the authors consider a regression problem in which the response variable is always ...
0
votes
0answers
20 views

Linear models with Matrix notation - eigenvalues, eigenvectors and geometric meaning

Find the eigenvalues and the eigenvectors: 1) $ C=\alpha I_n - (1-\alpha)J $, where $J=ii'$, such that $i=(1,1,...,1)' \in \mathbb{R}^n$, and $\alpha \in (0,1)$. 2) $B=I_n - \frac{1}{n}J$. So, ...
0
votes
0answers
36 views

which model is better when conducting linear regression

I am doing a regression on two variables x and y. And I have two approaches: assume the mean of y is in the form: $g(x;\alpha,\beta)=\alpha \rm{exp}(\beta x)$. Then use least squares solution to ...
1
vote
2answers
170 views

What is the distribution of $e=Y-\mathbb{E}(Y)$ where $Y=\exp(u), \ \ \ u\sim\mathbb{N}\left(\mu,\sigma^2\right)$

As $Y$ is log-normal we've $Y\sim \mathbb{LN}\big(\exp(\mu+\sigma^2/2),\exp(2\mu+\sigma)(\exp(\mu^2)-1)\big)$. Now I define $e = Y - \mathbb{E}(Y) = Y - \exp(\mu+\sigma^2/2)$. As $e$ is the ...
0
votes
0answers
21 views

Inverse probability weighting and regression adjustment

I am trying to estimate causal effects, the model is: $Y_i = \beta_0 + \beta_{1,i}T + \gamma x + u$ And where the treatment indicatior is: $T = I(\lambda x + d > 0)$ Where $x$ is a dummy. The ...
1
vote
1answer
38 views

What is the “systematic component of the model”, in bivariate linear regression?

I need to discuss the systematic component of the model in bivariate linear regression, but what is it in the first place? I have never come across this terminology in our class textbooks.
1
vote
0answers
23 views

Is there an appropriate way to control for individual subject identity in a linear model when data points arise from pairs of subjects?

Is there an appropriate way to control for individual subject identity as a Random effect in a linear model if each data point arises from a pair of subjects (for example, how long it takes two ...
1
vote
0answers
14 views

A special case of zero simultaneity bias?

Consider a system of structural equations, $$ Y=\alpha _{0}+\alpha _{1}X_{1}+\alpha _{2}X_{2}+\varepsilon:=X\alpha+\varepsilon\\ X_1=\pi _{0,1}+\pi _{1,1}Y+\eta _{1}\\ X_2=\pi _{0,2}+\pi _{1,2}Y+\eta ...
3
votes
1answer
26 views

How to create standardised regression estimates from unstandardised estimates

if we have two vectors $Y$ and $X$. Lets first standardise $Y$ by its mean so we get ($y$): $$ y = Y/\overline{Y}$$ We can get the slope of the least squares regression of $y$ and $X$ as: $$ \beta ...
0
votes
0answers
14 views

Dealing with linear regression with several categories

So, I want to do a linear regression for independent variable x and dependent variable y. I have 23 data for x and y and it is categorized into 4 categories. The value is log 2 transformed. The ...
6
votes
1answer
128 views

Bibliography for linear models

My main question: what bibliography would you recommend for linear models theory? I'm thinking of acquiring Plane answers to complex questions: the theory of linear models, by Ronald Christensen. Has ...
3
votes
2answers
40 views

Converting several t-statistics to a single F-statistic?

I have dummy coded a categorical regression, and ran OLS to get parameter estimates, along the lines of: $$ y= \left( \begin{array}{ccc} 1 & 0 &0\\ 1 & 0 & 0 \\ 1 & 1 & 0 ...
3
votes
0answers
25 views

Linear model with biased estimator

Consider a linear regression model. Suppose that the estimator $\hat{\beta}$ for the vector of the parameters of the model $\beta$ is, for some reasons, biased. As a consequence: ...
3
votes
0answers
25 views

Major League Soccer: Taking number of teams per season into account when modeling

Im doing research on the effect of salary inequality on team performance in the Major League Soccer, seasons 2007-2014. So for every season, I have data of all the salaries of all the teams, so I ...
0
votes
0answers
34 views

Time Series and Testing Auto Correlation

Consider the following asset pricing model: $$RET_t=0.621+1.414(M_t)+0.732(HML_t)+1.9349(SMB_t)+0.250(RET_{t-1}) $$ $$(0.077) \hspace{5mm} (4.141) \hspace{5mm} (3.242) \hspace{5mm} (3.294) ...
3
votes
1answer
25 views

Standard error for the sum of regression coefficients when the covariance is negative

I have a question about appropriately calculation the standard error for the sum of two coefficients in a linear regression model. My question is similar to this and this, but I can't seem to solve ...
4
votes
1answer
40 views

The distribution of $\bf{x}$ given an underdetermined system $A{\bf x}={\bf b}\sim N(0,\sigma^2 I)$

Suppose I have an linear system $A{\bf x}={\bf b}$ such that $\bf b$ is a vector of IID normal random variables and $A$ has dimension $n\times p$ with $p > n$. What can be said about the ...
0
votes
0answers
17 views

How to see a Gaussian Discriminant Analysis (GDA) as a linear model for multiclass case?

In GDA we can assume that posterior probability for each of $K$ possible classes is Gaussian with same variance $\Sigma$, and different means $\mu_k$, ie. $$p(\mathbf x|C_k):\mathcal N(\mathbf ...
3
votes
1answer
116 views

Multicollinearity in simple linear regression (not multiple)?

I am doing a simple linear regression analysis with 1 independent variable. I am checking data against assumptions. As I am checking against Tolerance and VIF level, I get the their values equal to 1 ...
10
votes
3answers
222 views

Why is it important to make a distinction between “linear” versus “non-linear” regression?

What is the importance of the distinction between linear and non-linear models? The question Nonlinear vs. generalized linear model: How do you refer to logistic, Poisson, etc. regression? and its ...
3
votes
1answer
87 views
0
votes
1answer
35 views

Linear Model - Interaction Term Interpretation

I've been working with simple linear models, to try and see the mean differences between groups, along with batch effects. ~0 + SampleType + Batch As far as I'm aware, this uses "Batch" as a ...
0
votes
0answers
14 views

How to do a post hoc test on a linear model with interaction?

I am new and I don't have much experience with R. That's why all the previous posts about this topic seem to be too complicated for me to apply for my case. I have ...
0
votes
0answers
13 views

What metric can be used to find the agreement of two variables - like correlation? [duplicate]

I would like to measure how well the output of my ODE model agrees with actual data. Using plotregression in MATLAB I obtained the following graph which shows the ...
0
votes
0answers
4 views

Error term with simple linear model

In simple linear regression model like this.  y =  a + bx + ε If I do not know whether ε(error) is independent and identically distributed how can I deal with the hypothesis test on b?
2
votes
1answer
33 views

“General” normal equations?

I am quite familiar with the usual OLS estimate for $\boldsymbol\beta$, given by $$\hat{\boldsymbol\beta} = (X^{T}X)^{-1}X^{T}\mathbf{Y}$$ for the linear model $\mathbf{Y} = X\boldsymbol\beta + ...
0
votes
0answers
13 views

How to estimate growth functions of class of linear classifiers?

I want to estimate Rademacher complexity of this family of classifiers: $$h(x) = {\rm sign}((w, x) + b)$$ I think I should use this inequality: $$R_l < \sqrt{{2 \ln B(l)}\over{l}}$$ where $B(l)$ ...
1
vote
0answers
15 views

Selecting Model Based On Data Set

I am doing a project for class in which we choose a data set and use SAS to analyze potential models to predict the response variable. I ran an preliminary proc reg in SAS to determine which variables ...
2
votes
0answers
35 views

Comparing Classical and Robust (Huber-White/sandwich/heteroscedasticity consistent) Standard Errors in Linear Multiple Regression

I'm running a linear multiple regression model of the type $y_i = \beta_0 + \beta_1 X_{i1} + \beta_2 X_{i2} + \beta_3 X_{i3} + u_i$. I came across King and Roberts' 2015 paper called "How Robust ...
1
vote
1answer
27 views

Linear regression on set of points with two lines

I have a set of points, in 2D space, where there are two tight (with minimal scatter) lines, with different slope and offset. There are also randomly scattered points that do not fall onto either ...
0
votes
0answers
28 views

Singular Value Decomposition and Least Squares

From Elements of Statistical Learning (pp. 64, 66), they explain how the $N \times p$ data matrix $X$ can be written as $$X = UDV^{T}$$ Here $U$ and $V$ are $N \times p$ and $p \times p$ ...
2
votes
1answer
46 views

How to deal and interpret local minima in a [time series] cross-validation error plot?

I am working with some time series data and I developed a linear regression model to make future predictions. The model has the following form: $$ y\left( t \right) =\sum _{ i=1 }^{ M }{ { \alpha ...
0
votes
1answer
32 views

Deriving $\text{Var}[\boldsymbol{\hat{\beta}}]$

I have already read How to derive variance-covariance matrix of coefficients in linear regression. Assume we're working with the usual simple regression model, $\mathbf{Y} \in \mathbb{R}^N$, $X \in ...
0
votes
0answers
10 views

About locally linear regression (as a nonparameter regression)

I use package loess in R to build a local linear regression model, but loess can only predict the data that is in the range of training data, when the new data is outside of the range of training ...
0
votes
0answers
18 views

Inverse linear model doesn't seem exact inverse [duplicate]

I'm dealing with a quantity that diminishes over time from 100% to 0%. I'm trying to plot the values, a lm abline, and large indicative points where the graph intersects ...