The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

Filter by
Sorted by
Tagged with
0
votes
0answers
21 views

Could this linear regression really has a probability of only 1 over a billion?

I am computing confidence intervals over linear regression, and I find the results to be rather counter-intuitive. I stack the two samples I want to regress together into a matrix. I compute the $2\...
1
vote
0answers
111 views

Show that $\text{Var}[\hat{\beta}_{1}] = \frac{\sigma^{2}}{\sum_{i=1}^{n}(x_{i1}-\overline{x}_{i})^{2}(1-r^{2})}$

For $i = 1,2,\ldots, n$, consider \begin{align*} Y_{i} = \beta_{0} + \beta_{1}(x_{i1} - \overline{x}_{1}) + \beta_{2}(x_{i2} - \overline{x}_{2}) + \epsilon_{i} \end{align*} where $\overline{x}_{j} = \...
0
votes
0answers
81 views

Given $Y_{i} = \beta_{0} + \beta_{1}x_{i} + \epsilon_{i}$, prove $\beta_{0}$ and $\beta_{1}$ are uncorrelated iff $\overline{x} = 0$

Let $Y_{i} = \beta_{0} + \beta_{1}x_{i} + \epsilon_{i}$ $(i = 1,2,\ldots,n)$, where $\textbf{E}[\epsilon] = 0$ and $\textbf{Var}[\epsilon] = \sigma^{2}\textbf{I}_{n}$. Find the least square estimates ...
0
votes
2answers
138 views

Feature selection using PCA for linear regression

I am using PCA to the training data set to do feature selection before applying linear regression to build a classifier model. In this scenario, would it be useful to use ridge regression to ensure ...
2
votes
1answer
79 views

Linear regression problem: how do I prove that $\sum_{i=1}^{n}(Y_{i} - \hat{Y}_{i}) = 0$? [closed]

If $\textbf{X}\in\textbf{R}^{n\times p}$ has full rank and $\textbf{Y}\in\textbf{R}^{n\times 1}$, prove that \begin{align*} \sum_{i=1}^{n}(Y_{i} - \hat{Y}_{i}) = 0 \end{align*} where $\hat{\textbf{Y}}...
1
vote
1answer
46 views

Write the null hypothesis for nested model test in algebraic form

The reduced model is: lm(y~Age+Sex, data = df); The full model is: lm(y~Age+Sex+Age*Sex, data = df). (...
1
vote
1answer
53 views

Whether there is significant difference between the slopes of different gender groups

The last question I have posted here: Whether there is significant difference between two gender groups There are 16 people in the dataset(using subjectID to ...
1
vote
0answers
80 views

Signal-to-noise ratio under heteroscedasticity

I want to be able to compare simulated datasets with and without heteroscedasticity, in the context of a linear regression: $y = X \beta + \epsilon$ Since I want to make sure that when I introduce ...
2
votes
0answers
25 views

Using transformation including a permutation matrix to fit a linear mixed model [closed]

reference: http://www.stat.wisc.edu/~bates/UseR2008/WorkshopD.pdf from page 97. I want to fit a linear mixed model. $Y|B=b\sim N(X\beta+Zb,\sigma^2 I)$ and $B\sim N(0,\Sigma(\theta))$. Here I choose ...
1
vote
1answer
44 views

How to account for incrementation in a log-linear model

I want to perform a mixed regression analysis with random intercept and uncorrelated random slope after multiple imputation. The dependent variable is continuous, namely a duration as number of days ...
1
vote
1answer
36 views

Can a linear and logit model have the same shape?

While I was working on an exercise based this book, I discovered something interesting. When I fit a logit and simple linear probability model on the data (see code below), the predictions are almost ...
1
vote
0answers
647 views

What statistics can I use?

I have done a research looking at different frequencies of abrasions (ablation, etc.) over time (in hrs) and my data mainly consists of zeros. As I am weak in statistics, I am unsure which statistics, ...
3
votes
2answers
188 views

Why does this expression simplify as such?

I'm reading through my professor's lecture notes on the multiple linear regression model and at one point he writes the following: $$E[(b-\beta)e']=E[(X'X)^{-1}\epsilon\epsilon'M_{[X]}]. $$ In the ...
0
votes
2answers
95 views

How to find all models that meet the pre-specified restrictions

Let's say I have a large number of predictors (e.g. 2000) and I'm facing the problem of choosing the linear regression model under following assumptions: There are few predictors that have to be ...
2
votes
0answers
25 views

Linear regression: right-hand tail of $\sigma$ marginal posterior

Suppose we're doing plain vanilla linear regression for $y$. The likelihood: $y_i \sim {\cal N}(\mu, \sigma^2)$, $i=1:N$. Priors: $\mu \sim {\cal N}(0,1)$, $\sigma \sim {\rm HalfNormal}(1)$. As ...
2
votes
2answers
105 views

Bias in parameter estimates for Cox proportional hazard model when covariates are collinear

For linear regression, if $y$ actually depends on two positively correlated covariates $x_1$ and $x_2$ (we can call it the true model), and if we only include one covariate, say $x_1$, in the ...
0
votes
1answer
48 views

How to interpret low R^2 value when we have the whole population

I am predicting the performance in a subject given the percentage of a gender that is in a group, for example, a group might be 70% female and 30% male. There is a significant relationship (p < 0....
2
votes
1answer
154 views

Random vs Fixed variables in Linear Regression Model

Reading "Econometrics" by Fumio Hayashi, from Princenton University Press, ISBN 0-691-01018-5, in page 13 by "Fixed Regressors" subtitle, it is stated: "We have presented the classical linear ...
0
votes
0answers
9 views

Median of demeaned variables and fixed effects

I have observations of the profit $y_{it}$ for several firms, noted $i$, over time $t$. For each firm, I compute the mean of $y$ over the years and call it: $\bar{y_i}$. Then I substract this firm-...
0
votes
0answers
63 views

Test for structural break with panel data

I want to study a variable $Y_{it}$ that represents the profit of several firms over time. In particular I want to evaluate whether there is a structural break after a specific year. the idea is to ...
0
votes
0answers
24 views

Within estimator and time fixed effects

If I data on a variable Y for 3 firms (A,B,C) over 3 years (1,2,3), then the linear firm fixed-effects model with year dummies (i.e. time fixed effects) without covariate would be: $Y_{it}=\alpha + \...
-1
votes
1answer
64 views

Is it possible for regression model to predict patterns separately from data has multiple patterns?

I want to predict sold number of each drink(hot and cold) without clustering. I have data which contains sold number of hot and cold drinks. I trained it with linear model in scikit, and I thought I ...
1
vote
1answer
31 views

What minimization problem has this solution

Consider the following basic minimization problem \begin{equation} {\displaystyle \min _{\beta\in R^{n}}{\frac {1}{n}}\|Y-X\beta\|_{R^{n}}^{2}},\end{equation} with solution \begin{equation} {\beta^*=(...
5
votes
2answers
844 views

What does it mean when I add a new variable to my linear model and the R^2 stays the same?

I'm inclined to think that the new variable is not correlated to the response. But could the new variable be correlated to another variable in the model?
0
votes
0answers
8 views

Generic expression used to compute output feature value map

Is there a generic expression used to compute output feature value map given an mxm input feature map and an nxn filter? ...
2
votes
1answer
2k views

glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric

I've been receiving the warning message in the title and have reviewed posts such as e.g. this one. I would like to understand how this feature has perfect separation with the target variable, since ...
1
vote
0answers
15 views

For linear least square regression, what is the relationship between the optimal solution and the empirical solution with finite samples

Consider a linear system $y_i=Wx_i+v_i$, where $x_i\in R^{d\times1} \sim N(0,\Sigma_x)$, $v_i\in R^{p\times1} \sim N(0,\Sigma_v)$, $y_i\in R^{p\times1}$ and $W\in R^{p\times d}$. Now we consider a ...
0
votes
0answers
168 views

p value in backward elimination regression

I need some help with the backward elimination output from Minitab below. Can p values A, B, C, D be equal to 0.745? Or the p value should be smaller than 0.745?
0
votes
1answer
26 views

Fitting a linear model using lm and a variable as factor [closed]

I am new to R and I'm confused by lin_mod <- lm(temp~as.factor(activ), data=beav2) specifically as.factor(activ). Why do ...
2
votes
1answer
29 views

Interaction term and sample selection

I have a dependent variable Y which is continuous. I want to study the impact of X on Y using OLS in a linear model, but I suspect the impact of X is more important for observations with a high value ...
0
votes
0answers
11 views

Independent variables with an important share of zeros

In a linear panel data model, is it an issue to have explanatory variables with an important share of zeros (e.g. 40% of observations are zeros)? Can the coefficients of an OLS regression be biased?
0
votes
0answers
16 views

Can visualization help to identify a dataset is linearly separable with polynomial features?

This data set cannot be linearly separable. If the polynomial and interaction features $X_1^2, X_2^2, X_1 \times X_2$ are used, can the data set linearly separable? I wanted to know there is any way ...
1
vote
1answer
36 views

Why do we use quadratic form for random vectors? [closed]

I am studying linear regression. I have studied this in the past, but this is my first time exposing myself to the matrix form of multiple linear regression. My matrix algebra/linear algebra skills ...
0
votes
0answers
18 views

How to interpret regression coefficients when each predictor variable contains different categories

Overview: I have conducted two types of statistical analysis using both linear regression and multiple regression. Overall, there were two observation periods, and the idea is to gauge if the rate ...
0
votes
0answers
38 views

Visualising Generalised linear models

I read about linear regression where we assume, the response is linear and the noise $\epsilon$, follows $N(0, \sigma^2)$ (Gaussian noise model), this leads us to conclude $E[Y|X] = b^*x$ and that the ...
0
votes
2answers
38 views

Linear Regression - holding predictor fixed at its mean

I am trying to create a linear model to predict House Price ($y$). The predictors in the dataset are Area (continuous) & Location (factor: West, Midwest, South, Northeast). I am asked to assess ...
1
vote
0answers
225 views

Learning algorithm vs Model in Machine Learning [duplicate]

In ML, I learned that we have a model and a learning algorithm. The learning algorithm is used to train the model with training data, does that sound correct? If the model is trained using linear ...
0
votes
1answer
27 views

How to compare two increasing trends to determine if rate of increase is statistically different?

My problem is that I have two groups and I am tracking their procedure cost over 6 years. I know that treatment CAS is significantly more than CEA, however I am trying to find out if the rate of ...
0
votes
2answers
31 views

In Linear regression is it possible to have same sign coefficients for dummies coming from the same variable?

So I have a categorical variable color which can take the values white, black, red. I created dummy variables for each of those ...
2
votes
0answers
111 views

TV Attribution: Fit linear model with additive and multiplicative terms

I am currently experimenting with a TV attribution approach proposed by Google: Liu, Y., Schwarzkopf, Y., & Koehler, J. (2017). TV Impact on Online Searches. They propose comparing website ...
2
votes
0answers
164 views

compare Bayesian linear regression vs standard linear regression

1st question, I recently learnt bayesian linear regression, but I'm confused that in what situation we should use bayesian linear regression, and when to use standard linear regression? What is the ...
1
vote
0answers
18 views

What do we use variance of the error term for in regression analysis?

So I get that, for simple linear regression where Y = B_0 + B_1(x) + E, Var(Y|x) = Var(E). Variance of the mean response involves it, as does variance of future responses, but is this ever actually ...
1
vote
1answer
52 views

Use linear projection without constant to obtain the linear projection with constant

We know that the linear projection of $y$ on $x_0$ $x_1$, $x_2$, . . . $x_K$ always exists and is unique: $$L(y|x)= \gamma_0 x_0 + \gamma_1 x_1 + ... + \gamma_k x_k = x\gamma$$ where $x = (x_0, x_1, ...
0
votes
1answer
49 views

How can the prior distribution of bayes regression be estimated by empirical bayes?

Neither in Efron's book Large-scale Inference:Empirical Bayes Methods for Estimation, Testing and Prediction nor by Internet search, did I find a prior distribution estimation method of Bayes ...
0
votes
0answers
17 views

Explained variance of incremental feature?

Suppose I have two features, and I know the explained variance of feature A for feature B. I build a linear model on feature A only, and I have a the explained variance of my target using this model. ...
3
votes
0answers
55 views

How to run linear regression with constraints in R? [duplicate]

If I have the following data n<-1000 x1<-rnorm(n,1,1) x2<-rnorm(n,2,2) x3<-rnorm(n,3,3) e<-rnorm(n) y<-3+0.5*x1+0.2*x2+0.3*x3+e I want to fit a ...
0
votes
0answers
26 views

How to infer the bounds on the R-squared value given the relationship between individual features?

Let say you have three variables X1, X2, and Y, all normally distributed, zero mean, unit variance. When you build a simple linear regression using: ...
0
votes
1answer
59 views

How to statistically analyze the relationship of right skewed data

I struggle to analyze these continuous data: The last four plots show the diagnostic plots on my model (model <- lm(data 1 ~ data 2). My aim is to investigate the relationship between data 1 and ...
4
votes
2answers
120 views

How can I calculate the critical t-values of a linear regression model?

I have implemented a linear regression in R (lm) and would like to show the significance and direction of the coefficient by means of the t-value. But now I'm not sure how to compute the critical t-...
6
votes
2answers
108 views

How are Bayes factors actually Bayesian?

I have been doing some linear model analyses involving Bayes factors lately and I have two probably very basic questions: 1) As far as I understand, a Bayes factor is simply a likelihood ratio, i.e. ...