All Questions

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

Using a subset of parameters in joint confidence region of a linear model

For a standard linear model of the form $y = X\beta + \epsilon$, where $\beta$ is a vector of parameters. we can calculate an individual confidence interval for each parameter (of 1-$\alpha$ quartile)....
1
vote
2answers
55 views

Bootstrapping with repeated measurements

I am trying to estimate a linear relation between body temperature and body mass, and I have a sample of measurements from subjects, with most subjects having one measurement, but several subjects ...
2
votes
2answers
50 views

Understanding simplification of constants in derivation of variance of regression coefficient

In looking over TooTone's answer in Derive Variance of regression coefficient in simple linear regression, there's a step in line 3 below where $(\beta_0 + \beta_1x_i + u_i )$ is simplified to $u_i$ ...
3
votes
1answer
98 views

Interpretation of standardized (z-score rescaled) linear model coefficients

I have analyzed some data on vegetation change as a function of change in soil parameters. I compared a dataset from 2001 with a dataset from 2018 (fully balanced). To investigate the change in ...
1
vote
1answer
43 views

How to account for incrementation in a log-linear model

I want to perform a mixed regression analysis with random intercept and uncorrelated random slope after multiple imputation. The dependent variable is continuous, namely a duration as number of days ...
1
vote
0answers
56 views

How can I interpret relative and absolute income of both partners in one regression?

Suppose you want to examine the effect of income on the amount of housework for women. Does it make sense to include both relative income (compared to partners income) and absolute income of BOTH ...
0
votes
1answer
138 views

R: how is the Pr(>|z|) in the results of glm.fit calculated and why?

I've been searching but I can't find anywhere an explanation of how the Pr(>|z|) column is calculated in the results of R's glm.fit function. I would really appreciate: a) an explanation so I can ...
0
votes
1answer
81 views

Linear regression very significant βs with multiple variables, not significant alone

Could anyone provide intuition on why for y ~ β1x1 + β2x2 + β3x3, β1 β2 and β3 can be significant in a multiple variable model (p range 7x10-3 to 8x10-4), but the βs are not significant in separate ...
0
votes
2answers
30 views

Statistical measure for linear regression with two distinct clusters of points

In the following plot, I have a linear regression of 30 points, representing 10 treatments with three replicates each. As you can see, the r-squared value is quite strong (0.83) and the p-value is ...
4
votes
1answer
43 views

Construct a $95\%$ confidence interval for $5\beta_4$

Construct a $95\%$ confidence interval for $5\beta_4$. If this question were about $\beta_4$ without the $5$, I would absolutely know what to do. But I have to idea how the $5$ comes into play. I can'...
0
votes
0answers
82 views

How to recover the original coefficients in principal components regression?

Let $y$ be a response variable of size $n\times1$, and $X$ be a covariates matrix of dimension $n\times p$, being $p>n$. Since $p>n$, I cannot directly solve the linear model $\tilde{y}=X\tilde{\...
1
vote
2answers
60 views

How to interpret multiple regression coefficients [duplicate]

I'm running multiple linear regression with 6 variables. For one of the variables D, the correlation coefficient between D and the response Y is - 0.34. But in the regression output, the coefficient ...
2
votes
1answer
37 views

Linear Regression: Why do the coefficients change on the original IVs when you interact them, and add that new interacted-variable to the model?

Basically I want to know how the 'constant' value differs in each of the following models: Model 1: DV=income; IV1=gender (0=male, 1=female); IV2=location (0=east, 1=west) Here, I understand the ...
6
votes
2answers
660 views

What is the intuition behind getting a slope distribution in linear regression?

If I understand it correctly, linear regression finds one best fitting line for the given data. It can do it either by using calculus and solving for intercept and slope equations or it can solve it ...
1
vote
0answers
65 views

In regression, when partitioning SS among predictors, what determines which predictors get the SS that can be attributed to more than one predictor?

In regression analysis, predictors sometimes correlate (and in my field, psychology, they always do; often because they are simply measurements of the same aspects of human psychology). If predictors ...
5
votes
2answers
457 views

Regularized parameter overfitting the data (example)

Possible duplicate of (Why) do overfitted models tend to have large coefficients? How does regularization reduce overfitting? In the Coursera's machine learning course by Andrew Ng, I came across ...
1
vote
0answers
90 views

Can you remove an independent variable that is questionable?

One of the key independent variables is in a form of a probability. The probability is outside the range of [0, 1]. There's approximately 10 of them out of 100. For example, it has 10 values with a ...
-1
votes
1answer
59 views

How to achieve r value 1.0 with Linear slope angle 10° in R [closed]

I have been reading about the r value and its purpose, however the below image has altered my understanding(that r value represents Direction and Strength and also ...
0
votes
1answer
4k views

Relation between regression coefficient and correlation coefficient

For simple linear regression, the regression coefficient's sign and the correlation coefficient's (between independent and dependent variable) sign should be matching or not?
3
votes
2answers
72 views

Linear regression with $n<p$: Solution of $Ax=b$ that minimizes the $2$-norm of $x$

Consider a full-rank $n\times p$ matrix $A$ and $b\in\mathbb{R}^p$. If $n<p$, I want to minimize the norm $||x||^2=x_1^2+\dots+x_p^2$ over $x\in\mathbb{R}^p$, subject to the condition $Ax=b$. So, ...
1
vote
2answers
145 views

Interpretation of dummy variables

Lets say that i have linear regression $Y= a +XB$, and in my $X$ matrix i have a dummy for the gender, lets call it $d_{g}$ which is 1 for male and 0 for female. The coefficient $b$ of $d_{g}$ shows ...
0
votes
1answer
188 views

F-statistics and coefficient p-value of model with only one variable

F-test tests the null hypothesis that all coefficient of variables in the model equal to zero. P-value in a hypothesis test shows the probability of having observed results if null hypothesis is ...
1
vote
0answers
752 views

How can I interpret coefficients of quadratic and linear term?

It may be a basic statistic question for someone, but I'm struggling with this. I'm trying to interpret a regression analysis. Here is examples. ...
1
vote
1answer
208 views

How to show the residual sum of squares of one model is at least the residual sum of squares of another?

This question has really been bugging me and I'm failing to understand it conceptually. I don't see why it is the case and would really appreciate any help.
2
votes
1answer
2k views

How can I get the variance $\sigma^2$ for Linear Regression under homoscadastic with no serial correlation?

The image is a copied and pasted youtube lecture on Linear Regression. I can sort of understand what the lecturer says during the lecture, but I wonder how I actually calculate the $\sigma^2$ in the ...
0
votes
0answers
161 views

Normal approximation for Negative Binomial regression

In negative binomial regression, the distribution is specified in terms of its mean, $\frac{pr}{1-p}$, which is then related to explanatory variables as in linear regression or other Generalized ...
6
votes
2answers
4k views

Linear regression with log transformed data - large error [duplicate]

I have a set of data which is has a very large positive skew, and has been transformed using a logarithm. I wish to predict one variable from another using the lm ...
1
vote
1answer
268 views

Interactions in linear probability models

Suppose, I estimate a simple linear probability model: $P(Y=1)=\beta_0+ \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_1 \times X_2 + u$, where $Y$, $X_1$, and $X_2$ are dummy variables. All standard OLS ...
0
votes
1answer
34 views

Can Relative Risk mislead us when choosing predictors for a logistic model?

My friend taught me to use Relative Risk as a guide to check if my coefficients make sense. For example, I have a propensity to default model, where the variable fl_default is equal to 1 if the ...
1
vote
2answers
59 views

Interpreting the sign of returned coefficients in linear model?

I'm having a question regarding, how should the sign of returned coefficients (e.g. by R's lm()) in linear model be interpreted. Particularly I was doing a model on some kids' test scores and there's ...
2
votes
1answer
147 views

Understanding the meaning of the parameters in the linear regression model

When I first time learn multiple linear regression, I remember the interpretation of the regression coefficient is that: the marginal contribution of a specific predictor. Now I am rethinking this ...
0
votes
0answers
1k views

How to interpret and explain negative coefficients when they do not make sense

I have seen appearance of negative coefficients where they do not make sense (the data is related to costs where negative coefficient should not appear). If regression models are fitted to individual ...
1
vote
1answer
162 views

OLS when both independent and dependent variable are multiplied by another variable

I was given the following problem involving OLS: Suppose we have $(y_i,x_i,z_i)_{i=1}^n$ iid sequence, such that $x_i$ is a vector with K entries and $y_i$ and $z_i$ are scalars. Suppose $z_i$ is ...
3
votes
1answer
4k views

Standard error for the sum of regression coefficients when the covariance is negative

I have a question about appropriately calculation the standard error for the sum of two coefficients in a linear regression model. My question is similar to this and this, but I can't seem to solve ...
0
votes
1answer
370 views

Regression Through Origin (RTO) with 2 variables?

I am seeking a parametric expression of a RTO (regression through the origin) for a 2-variable system, that is, $Y = b_1 X_1 + b_2 X_2$. The OLS (ordinary least square) expression is commonly known, ...
1
vote
0answers
117 views

Distribution of coefficients in linear regression

I have just started studying the textbook "The Elements of Statistical Learning" and I am currently reading about Linear Regression. I have a question about the distribution of the coefficients in the ...
3
votes
0answers
497 views

What explains the correlation between the slope and intercept?

If $R^2$ explains the variation explained by a model, what explains the correlation between the coefficients given for a slope parameter and an intercept? I have been thinking of it in two ways: If ...
1
vote
0answers
32 views

Coefficient in linear regression changes drastically if additional variables are added. Why? [duplicate]

n <- 100 x2 <- 1 : n x1 <- .01 * x2 + runif(n, -.1, .1) y = -x1 + x2 + rnorm(n, sd = .01) summary(lm(y ~ x1))$coef Coefficients (all significant): (...
1
vote
1answer
2k views

Interpretation of Coefficients in linear regression using 'fitlm'

I require help with regards to the interpretation of linear regression results (I'm using the Matlab 'fitlm' function). My data has 8 features, and when each feature is plotted against the response ...
1
vote
0answers
710 views

Are sufficient statistics for regression equivalent in the frequentist and Bayesian cases? [duplicate]

If I have a Poisson regression such that $\lambda = \alpha + \beta t$, $\alpha + \beta t \geq 0$ $\forall t, \alpha, \beta$ and $Y_t \sim \textrm{Poisson}(\lambda_t)$ for which I have 10 observations ...
4
votes
1answer
6k views

Testing a regression coefficient against 1 rather than 0

Brief caveat- I haven't dusted off my stats knowledge since some university courses a few years ago, and I'm struggling with cobwebs. I have a model where a linear 1 to 1 relationship has been ...
6
votes
2answers
1k views

Prior for the coefficients of a linear regression model

I have a linear regression model $\bf Y=\bf{X}\bf{\beta}+\epsilon$. I want to assign a prior on $\bf\beta$ in order to derive the posterior predictive model $p(y_{predictive}|\bf{y},\bf{X},\beta)$. ...
1
vote
1answer
81 views

Confidence interval for a multiple of regression coefficient

I am trying to model relationship between length of stay of patients in hospital(Y) vs Age in years(X). The data set I've got doesn't specify the unit of length of stay. So now estimated value of my ...
0
votes
0answers
62 views

How can I create a linear regression model with some negative coefficients in R? [duplicate]

What I'm trying to do is to construct a linear model in a form like $$ Y = \beta_0X_0-\beta_1X_1+\beta_2X_2 + \beta_3 $$ where $\beta_0$, $\beta_1$ and $\beta_2$ are coefficient of predictors $X_0$, ...
4
votes
1answer
816 views

How can I optimize coefficients of an arbitrary model?

This might be terribly easy but I'm probably lacking the keywords to search for. Assume the following (dummy) data: ...
4
votes
1answer
9k views

Interpreting intercept for the log model in linear regression in R for small predictor

I have a dataset. Assume that y is the dependent variable and x is the independent variable. My goals for this analysis is mainly on the following hypothesis: Expecting x=0 to imply y=0 Expecting ...
1
vote
1answer
1k views

How to capture & present lm model output from R

After running iterations of lm() in R, I am now stuck with which components of the model's output to present and how to present them. I know that the $R^{2}$ value, ...
2
votes
1answer
295 views

Coefficient of Determination: For the perimeter and area of a square: Why different?

When calculating the coefficient of determination for a square, why is it that if you use the data set for the side length of as X= (1,2,3,4) and the perimeter as Y=(4,8,12,16) the Coefficient of ...
2
votes
0answers
197 views

Interpreting regression coefficients of log(y+1) transformed responses

I have measurements $y_1$,...,$y_i$,...,$y_n$ taken from a set of replicates in a factorial designed experiment. In order to use a linear regression I define my response $z_i = log(y_i + 1)$. The ...
0
votes
1answer
1k views

R: Explanation of a multiple linear regression summary [duplicate]

I am quite new with R and while i am able to perform the basics i am not yet able to understand the output results. For example: summary(lmodel) generates the ...