Questions tagged [linear]

For statistical topics which involve the assumption of linearity, for example, linear regression or linear mixed models, or for the discussion of linear algebra as applied to statistics.

Filter by
Sorted by
Tagged with
2
votes
2answers
35 views

How to show $\rho(\beta_0, \beta_1) \leq 0$ when $x_i \geq 0$?

Consider a simple linear regression modelled by $Y_i = \beta_0 + \beta_1x_i + \epsilon_i$ where each $\epsilon_i$ is independent and follows a normal distribution with mean $0$ and variance $\sigma^2$....
0
votes
1answer
37 views

Determining if linear regression is appropriate

I am designing an experiment, and I want to know if using a hypothesis test on the linear regression model to analyze my data is appropriate. The experiment will have five independent variable values ...
2
votes
2answers
52 views

Linear model with both additive and multiplicative effects

In linear regression, the independent variables have an additive effect on the response (level-level regression): $y=\beta_0+\beta_1x+\epsilon$ In a log-level regression, the independent variables ...
0
votes
1answer
23 views

Multiple linear regression approach: difference in set-up and dataset split

Firstly, I am new to modelling linear regression models. I want to build a linear regression model to predict building energy use based on building parameters. I'm having a dataset of 100000 buildings ...
4
votes
1answer
135 views

Linear and Nonlinear Components of a Time Series

I am starting to develop a hybrid ARIMA-ANN model for forecasting. Most of the journals I read mention mostly a linear component for ARIMA and a nonlinear for ANN. How can you know which components ...
0
votes
2answers
46 views

Why is $Y=\beta_0 x^{\beta_1} e$ a linear model?

Why is $Y=\beta_0 x^{\beta_1} e$ a linear model? When we apply the transform, it becomes $lnY = ln\beta_0+\beta_1 lnx +lne$, and why is it still linear when the $\beta_0$ part is under ln?
1
vote
1answer
27 views

Variance of OLS estimates proof

I recently came across this question on stats stack exchange where a user asked for the proof that the T statistic has a t distribution. The question assumes a linear model of the form; $$Y=X\beta+\...
0
votes
0answers
15 views

Deriving vectorized form of linear regression

We first have the weights of a D dimensional vector $w$ and a D dimensional predictor vector $x$, which are all indexed by $j$. There are $N$ observations, all D dimensional. $t$ is our targets, i.e, ...
0
votes
1answer
30 views

Can you use the residuals of a first regression model as a new term to improve the model?

In the house price prediction example of Practical Statistics for Data Scientists (by Bruce, Bruce, and Gedeck; p. 168), the median residual for each zip code is taken and binned. The authors note ...
0
votes
1answer
19 views

Shape of derivative with respect to vector

I'm confused about the shape of the derivative with respect to vector. In the book MATHEMATICS FOR MACHINE LEARNING page 150: For formula 5.56a, x1 to xn are horizontal. For another pdf "...
7
votes
1answer
941 views

Why does Covariance measure only Linear dependence?

1) What is meant by linear dependence? 2) How can I convince myself that covariance measures linear dependence? 3) How I can convince myself that non-linear dependence is not measured by covariance?...
4
votes
1answer
84 views

Is there a more intuitive term for a “Linear model”?

A model is linear if it is linear in the parameters. Therefore, a linear model can describe a curve, if a function (such as raising to a power, but not necessarily) is performed on one of the ...
2
votes
1answer
50 views

Can this be written as a linear model?

$$y_i=\frac{(x_{0i}\beta_0+x_{1i})+\log(\beta_1^2 x_{2i})}{x_{3i}}+e_i\quad,\,i=1,\ldots,n,\,x_{pj}>0$$ I am wondering if this can be written as a linear model, I didn't think so because of the ...
0
votes
0answers
16 views

simple linear regression forecast in matlab

How could I do a linear regression forecasting in Matlab, please? I am not asking for the code itself, but for some guidelines on how can I structure the problem and what to use. I have three ...
0
votes
1answer
26 views

Meaning of slopes of $1$ and $0$ in a linear regression equation

I have data for the height of certain trees in 1996 (dependent variable) and their height in 1990 (explanatory variable). The question: Is the value of $1$ included in the confidence interval for the ...
15
votes
5answers
2k views

Why does linear regression use a cost function based on the vertical distance between the hypothesis and the input data point?

Let’s say we have the input (predictor) and output (response) data points A, B, C, D, E and we want to fit a line through the points. This is a simple problem to illustrate the question, but can be ...
1
vote
1answer
16 views

Goodness of fit test for LASSO

How would you do a goodness of fit test for Lasso regression? Im guessing that the $R^2$ value, as for linear regression, wont work anymore. Why is that?
0
votes
1answer
15 views

multiple curve fitting on scatter plot [duplicate]

I have the following problem: Let's assume we have a dataset with X and Y values. If we plot them on a scatter plot it would look somehow like the following graph: If we have a look at this, we see ...
14
votes
4answers
2k views

What is the best programmatic way for determining whether two variables are linearly or non-linearly or not even related

What is the best programmatic way for determining whether two predictor variables are linearly or non-linearly or not even related, maybe using any of the packages scipy/statsmodels or anything else ...
0
votes
0answers
4 views

ESL; Linear Regression and Nearest Neighbours doubt

I've been trying to self work my way through ESL but seem to have run up against the following issue right off the bat. I understand that the authors are trying to introduce the idea that the way that ...
0
votes
1answer
22 views

Preprocessing a data set for linear regression

I'm currently a student in a machine learning course studying for an upcoming exam. Here's a question I've been given for practice: You have a very large dataset of employees and you'd like to ...
1
vote
2answers
224 views

Linear Mixed Effects Models: how to model dependent categorical variable?

I am trying to fit a linear mixed effects model with several fixed effects and a random intercept that varies per subject. My problem is that I know that one of the fixed variables, let's call it 'A'...
6
votes
0answers
594 views

Which theorem in Cover's 1965 paper is actually referred to as Cover's Theorem?

Cover's Theorem is stated on Wikipedia (and similarly elsewhere) as A complex pattern-classification problem, cast in a high-dimensional space nonlinearly, is more likely to be linearly separable ...
0
votes
1answer
45 views

How can I calculate the mean and variance of a linearly transformed random variable?

Say I have a random variable $x$, with mean $\mu_x=35$ and standard deviation $\sigma_x=10$. I want to linearly transform $x$ to $y$ according to the formula $y=a+bx$ so that $\mu_y=100$ and $\sigma_y=...
4
votes
2answers
4k views

Understand Link Function in Generalized Linear Model

I am still trying to learn (may be the terminology issue) what does "link function" mean. For example, in logistic regression, we assume response variable is coming form binomial distribution. The $\...
0
votes
1answer
20 views

Relationship of x regress y and y regress x on the slope

Consider a linear regression model y on x and x on y. We have $Y = a'X + a$ where $a' = \frac{cov(X,Y)}{Var(X)}$. Equivalently, we have $X = b'Y+b$ where $b' = \frac{cov(X,Y)}{Var(Y)}$. I am ...
1
vote
1answer
26 views

Equivalent Variance-Stabilizing Function

I've been reading the book "Generalized Linear Models with Examples in R" by Dunn and Smyth. In chapter 5, they claim: Variance-stabilizing transformations h(y) used with linear regression ...
1
vote
1answer
94 views

Why significant predictors are different for two highly correlated dependent variables?

I am using linear mixed-effects (LME) models to investigate the longitudinal effects of maternal factors on infant adiposity indices. Infant adiposity was measured at 3-time points (birth, 3 months ...
0
votes
0answers
13 views

standard error of sums of estimates of a linear model

Say I have a linear model with X1 a continuous variable and X2 a categorical variable: y = b0 + b1X1 + b2X2 Is there a way to find the standard error of b1 + b2 from the independent standard errors of ...
1
vote
2answers
51 views

Regression and the CEF

I recently read in this page (https://www.timlrx.com/2018/02/26/notes-on-regression-approximation-of-the-conditional-expectation-function/#fn1) that: "Regression offers a way of approximating ...
3
votes
2answers
107 views

Multiple Regression: Two Binary ind. vars. - Can an interaction term be significant, when the main effects are not?

I think this is a simple question, but I'm having difficulty coming up with a test or example for this. Consider lm(A ~ B*C, data=D) where A is continuous; B and C ...
0
votes
0answers
11 views
1
vote
1answer
27 views

What are the degrees of freedom for “linear regression t-test” with one binary variable?

Quick, simple question. I read that DF = n-2 when doing "a linear regression t-test" with a continuous independent variable. (Testing that the slope is 0). https://stattrek.com/regression/...
1
vote
2answers
53 views

Is this a linear model? [duplicate]

mpg = mileage per gallon and hp = horsepower Why is this model a linear model despite having a square of horsepower in it?
1
vote
2answers
45 views

Linearity assumption for Pearson correlation

It's known that values must have a linear relationship to count Pearson correlation between them. I'm wondering if there are any ...
0
votes
0answers
8 views

How to generate binary data that is additive through levels of a factor?

Suppose we define $Y\in \mathbb{R}$ as a function of some random variable $Z$ so that $$ Y(Z)=\alpha Z + \beta X $$ for $Z\sim Bern(p)$ and $X \sim N(0,1)$. Then $$ Y(1) \sim N\left(\alpha, \beta^2\...
1
vote
1answer
20 views

Explaining the effect of adding a dummy variable on a restricted model

So I'm learning basic econometrics. I'm looking at a linear regression model on Stata where I have to qualitatively compare a restricted model, where $y_i = b_1 + b_2 x_i$ and $y_i = b_1 + b_2x_i + ...
1
vote
1answer
35 views

Imputing missing values for linear regression model, using linear regression

I scraped a real estate website and would like to impute missing data on total area (about 40% missing) using linear regression. I achieve the best results using price, number of rooms, bedrooms, ...
2
votes
0answers
22 views

Test for statistically significant difference in means across different conditions

I've never had a formal statistics course, so I hope these questions aren't too basic or using incorrect terminology. I have samples of bivariate data across different scales of 3 kinds, which I ...
1
vote
1answer
29 views

Linear transformation *not* to use to scale mean and SD

I'm working through the Exercises in Regression and Other Stories. Exercise 3.6 on Linear transformations asks you to provide a formula to rescale a variable with mean = 35 and SD = 10 to have mean = ...
4
votes
1answer
54 views

Which linear model?

I encountered the following problem: I have a biomarker measured at baseline and the renal function measured at baseline and Follow-up. 2 continuous variables My question is: does the biomarker ...
2
votes
2answers
47 views

What do exponential of coefficients (like odds ratio in logistic regression) from linear regression indicate?

The long title says it all. For example, I have performed linear regression (OLS) with commonly used iris dataset using following formula: PL ~ SW + Species ...
1
vote
1answer
242 views

Recommendation for fitting mainly linear time-series data to a curve

Most of the time-series data I'll be looking at is linear and uniform - a straightish light parallel to the x-axis. The exceptions I need to find are those that deviate recently, the last one, or two ...
2
votes
1answer
69 views

Do I accept or reject the null hypothesis?

M1 : Y ∼ β0 + β1x1 + β2x2 M2 : Y ∼ β0 + β1x1 anova(M1,M2) shows a p-value of 0.0001, so we prefer M1 at significance level 0.05. Would that be correct? I thought that if .0001<.05, I should reject ...
3
votes
0answers
44 views

What is the significance of “orthogonal” vectors in statistics?

So I am reading What does orthogonal mean in the context of statistics?, and there are contradictory answers. The most upvoted answer says that "Therefore, orthogonality does not imply ...
0
votes
1answer
2k views

Effect of Sample Size on Model Mean Squared Error

I am studying Linear Regression by performing a simulation study with different sample sizes. As I increase the sample size the mean squared error seems to reduce but with a very large sample size (...
0
votes
1answer
31 views

Relationship between logistic regression and linear regression

I've encountered a problem where I need to analyze the relationship between a movie's length, a movie's price and it's sale on a video streaming platform. Now I have two choices to quantify sale as my ...
1
vote
1answer
289 views

Residual Analysis assumptions for non-linear regression

I understand Regression analysis relies on the following assumptions about the residuals: Normally Distributed (normal plot of residuals) Be independent of each other (random and data must be time ...
3
votes
1answer
40 views

Linear mixed effect model in statsmodel package

I try to use linear mixed effect model in Python statsmodels package. However, I have no idea how to conduct and interpret the result. Group 1 (20 people) : base line & follow up Group 2 (20 ...
1
vote
1answer
51 views

How covariance matrix of the error term in linear regression can be NON-singular?

I don't understand linear regression. Assume the classic linear model: $$Y = X \beta + \epsilon,\\ \epsilon \sim \mathbb{N}(0, \sigma^2 I_n), $$ where $Y$ is a vector of length $n$, $X$ is a matrix of ...

1
2 3 4 5
20