Questions tagged [regression]
Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.
27,272
questions
0
votes
0
answers
7
views
is it possible for odds ratios' or risk ratios' confidence interval to contain 1 and still be statistically significant?
my understanding is that if a 95% CI for odds ratios and risk ratios contains 1 then it is not statistically significant. However I have seen in studies such as https://pubmed.ncbi.nlm.nih.gov/...
- 103
2
votes
1
answer
24
views
When does R-squared in multiple linear regression equals the sum of the R-squared from two simple linear regression?
I know that in the simple linear regression, the $R^2$ is just the sample correlation between the response and covariate. My question is that suppose I fit a linear regression by using two covariates, ...
- 21
3
votes
1
answer
38
views
What estimation methods other than ordinary least squares guarantee the orthogonality of predictions and residuals?
From my question here, it is evident that estimation approaches to linear regression other than ordinary least squares can result in the predictions and residuals lacking orthogonality, despite the ...
- 35.7k
0
votes
0
answers
10
views
If residuals are serially correlated does that mean they are normally distributed
In linear regression if the residuals are serially correlated does that mean then that they are normally distributed
- 1
1
vote
1
answer
15
views
Correction for multiple comparisons using sum contrasts with linear regression
I am computing the following model using the lme4 package in R:
...
- 39
0
votes
1
answer
23
views
How to calculate variance of AR(1) process
I have a stationary AR(1) process: $Z_t = \alpha_{1}Z_{t-1} + \nu_{t}$, where $\nu_t$ is white noise and $|\alpha_1| < 1$.
I have to show that the variance of $\Delta Z_t$ is $$V[\Delta Z_t] = 2\...
0
votes
0
answers
4
views
Why same t-score for contrast pairs involve continuous x continuous interaction (emtrends)?
I want to calculate the difference in simple slope estimates for two-way interactions that involve two continuous variables. But I found the t-score is all the same for the contrast-pair results when ...
- 25
-2
votes
0
answers
16
views
regression plots for each group separately [closed]
I want to plot regression lines for each group separately. Each group's regression plot as 1 plot. I have around 100 groups. So it will be 100 plots. Can this be done automatically without selecting ...
- 1
0
votes
0
answers
12
views
How I can choose best regression model on my data? [closed]
[I want to find the best fit curve or model for my data and my data follows the following pattern. I have tried the curve fitting tool on Matlab but no model fits precisely on the data. Could you ...
- 1
0
votes
0
answers
25
views
Beyond R2: Is there a better way to assess my regression?
I wondered if someone could offer some advice on the following problem that I have been puzzling over. Sorry if this is a basic question.
Problem: I have two sets of data and I need to build a linear ...
- 1
1
vote
1
answer
30
views
Does multicollinearity among control variables matter?
I am conducting a regression analysis between X and Y, where X is the main independent variable. However, I want to control for several variables that are related to Y. For example if my dependent ...
- 11
0
votes
1
answer
25
views
Why does the estimation of the variance of the outputs in linear regression include the fitted values?
I was revisiting my notes about the classical linear regression model, $Y = X \beta$.
If we want to estimate the variance of the least squares estimator we usually suppose that the outputs $y_{i}$ are ...
1
vote
0
answers
43
views
Derivation of the variance of the ${\hat{\beta}}$
First of all, thank you so much for all the input and for pointing out my mistakes. I made some corrections and try to add more details in my problem.
Edit 1:
I was trying to derive the variance of ${...
- 111
2
votes
1
answer
15
views
Simple Linear Regression Question: How does correlation between X and Y affect MSE?
Suppose we know that the correlation between $X_1$ and $Y$ is $\rho_{X_1Y}$ and the correlation between $X_2$ and $Y$ is $\rho_{X_2Y}$. Furthermore, suppose $0 < \rho_{X_2Y} < \rho_{X_1Y}$. Now, ...
- 105
1
vote
1
answer
16
views
Calculation of leave-one-out cross-validation statistic in linear regression when design matrix is square
Suppose we are performing a least-squares multiple linear regression of the form
$$
\mathbf{Y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}\,,
$$
where $\mathbf{Y}=(y_1,y_2,...,y_N)$ are the ...
- 13
1
vote
3
answers
246
views
Understanding multiple linear regression residuals
I'm a bit confused whether residual vectors in OLS are orthogonal to every vector in X.
The problem I have is: Which of the following are true statements for our multiple linear regression with ...
- 11
0
votes
0
answers
22
views
Exercises for Linear Regression Models
I am looking for exercises (more theoretical than applied) about linear regression models. Maybe you can advice me where I can find such exercises. Here is what my linear models course deals with if ...
- 111
1
vote
1
answer
28
views
Model validity for Ordinal logistic model
I am doing a study using OLR. The model tries to assess the satisfaction of ground level stakeholders (scale of 1(extremely dissatisfied) to 5(highly satisfied) in an urban area. The independent ...
- 11
0
votes
0
answers
29
views
log of average or average of logs for regression?
I have to perform regression for product prices over a period of time and I have to do so for multiple categories. For eg. Category_1 includes prices of item_1, item_2 and item_3 over a period of time,...
- 21
0
votes
0
answers
12
views
Detection of Multivariate Outliers (in a multiple linear regression problem) [duplicate]
In a multiple regression problem, suppose we have responses $Y_1, Y_2, \cdots , Y_n$ corresponding to data $\mathbf{X}_1, \mathbf{X}_2, \cdots, \mathbf{X}_n$ where each $\mathbf{X}_i$ is a $d$-...
- 561
1
vote
1
answer
15
views
on the rank condition of regression model with cross validation
Consider a standard linear regression model in a matrix form given by
\begin{equation}
y=X\beta+u
\end{equation}
where $y$ is $n \times 1$, $X$ is $n \times k$ with $n>k$. Assume that $X$ has full ...
- 153
1
vote
1
answer
22
views
Zeros in Dependent Variable : Bad- Zeros in Independent Variables: Not Bad?
I am an MBA Student taking courses in statistics.
We have been learning about regression models for count data. Recently, our professor has been talking about situations in which there are a lot of ...
0
votes
0
answers
17
views
Do I need to include the information (Pvalue) of confounders in the final table?
I was wondering if I need to provide a regression table including the estimation of all main varibales + confounders? Can I just present the final result (coefficeint and pvalues) of the main ...
- 13
0
votes
0
answers
14
views
Performing deconfounding using multilinear regression for select number of variables
I'm trying to perform a correction on my data so I can use deconfounded residuals in later analyses.
My data object is a subject by observation matrix (each subject has i observations - if it helps ...
- 23
2
votes
0
answers
27
views
calibration according to Kuleshov
i was reading this paper https://arxiv.org/pdf/1807.00263.pdf in which Kuleshov gives a definition of a well calibrated neural network in a regression task
I would like to ask one thing:
why does he ...
- 195
-1
votes
0
answers
21
views
Interpretation of coefficient in log-level regression [duplicate]
The result of my panel study with log-transformed dependent variable l_passengers results in a coefficient of 1.15076 of my independent variable lccshare. I'm not quite sure how to interpret this as ...
- 1
0
votes
0
answers
16
views
Classification email newsletter imbalanced
I'm given a case to determine the best time to send an email newsletter based on whether the email is opened.
The problem is that over 70% of the emails are sent on Tuesday and the dataset is ...
0
votes
0
answers
19
views
How to perform multilevel regression on Y?
We know multiple linear regression has the equation
$$
Y_1 = \beta_0 + \beta_1x_{11} + ... + \beta_k x_{1k} + \epsilon_1 \\
Y_2 = \beta_0 + \beta_1x_{21} + ... + \beta_k x_{2k} + \epsilon_2 \\
\dots \...
- 161
0
votes
1
answer
39
views
One of the main effects not significant, but interaction term significant
A is significant
B is not significant
A x B is significant
Do I say that
(1) There exists a ...
- 145
0
votes
1
answer
21
views
For ecological data, when is a gaussian distribution appropriate?
This may seem like a very basic question, but is something I have become more confused about the more I read.
Say I have a dataset with morphological measurements of various plant traits (e.g. leaf ...
- 15
0
votes
1
answer
38
views
How to determine if the log likelihood of logistic regression is too large or not?
I am running a logistic regression on STATA with binary response variable, and 2 predictor variable that are discrete, as such one is in % (but takes only 2 values strictly i.e., 5% or 10%) and ...
- 51
0
votes
0
answers
19
views
Prove that the sample covariance between observation and OLS fittings are nonnegative
I am trying to show that $$\frac{1}{n} \sum_{i = 1}^n (y_i - \overline{y})(\hat{y}_i - \overline{\hat{y}}) \geq 0$$
where $y_i$s are the observations, $\hat{y}_i$s are corresponding LS fitting values, ...
- 111
0
votes
0
answers
12
views
Build three models in one study
I build three models. First model has two mediators, one is not significant, then I removed it from the second model. The second one has one moderator and one mediator. The third one has another ...
- 1
0
votes
0
answers
18
views
What is the best model selection method for high-dimensional linear regression?
Model selection (best subset selection) in linear regression is quite important in many applications. Among the methods belonging to different frameworks such as information criterion, hypothesis ...
- 31
4
votes
2
answers
428
views
Simulated data for logistic regression
I used the code below to create the random variable x1 and binary variable y, and fit the regression with y and x1. My questions are:
Why regression coefficient estimates are not close to 2 and 10 (...
- 113
0
votes
0
answers
14
views
What is better OMP or LASSO?
Variable selection in linear regression models is quite important. In this regard, the orthogonal matching pursuit (OMP) is a classical greedy approach to variable selection. On the other hand, LASSO ...
- 31
0
votes
0
answers
10
views
Calculate uncertainty of surface normal based on points uncertainty
How to calculate angular variation of fitted plane from points that has positional uncertainty with normal probability distribution?
For example, I have 4 points
P1 = 102.0000 84.0000 139.5443
P2 = ...
0
votes
0
answers
7
views
Knockoff filters simple explanation and importance!
Knockoff filters are new in the field of variable selection. Can someone provide (or refer to) a slightly simple understanding of the topic? Also, what is the fuzz about this new method compared to ...
- 31
0
votes
0
answers
14
views
Controlling for Variables in Binary Logistic Regression
How do you control for variables such as gender within a binary logistic regression? I have a yes/no dependent variable with several variables that are also either yes/no. I'd like to control for ...
- 11
2
votes
1
answer
31
views
Why linear mixed effect regression over multiple linear regressions for individual random effect?
I have a dataset on green-up days pan-Arctic and relate this to weather variables.
I therefore want to use a linear mixed effect model where region (e.g. Alaska, ...
- 352
3
votes
2
answers
59
views
Expression of $\hat{\beta}_1$ in term s of $\beta_1$ in simple linear regression
I'm reading a pdf document about simple linear regression, and it gave an expression of the least square estimator $\hat{\beta}_1 = \beta_1 + \frac{\sum^n_{i=0} \epsilon_i (x_i - \bar{x})}{\sum^n_{i=0}...
- 33
-4
votes
0
answers
19
views
I want to plot regression lines for each group separately in different plots. I have around 100 groups [closed]
I want to plot regression lines for each group separately. Each group's regression plot as 1 plot. I have around 100 groups. So it will be 100 plots. Can this be done automatically without selecting ...
- 1
1
vote
0
answers
21
views
Multicollinearity Market mix modeling
I want to know what can be the best approach to handle multicollinearity. I am building a regression model with just 4 independent but all important variables and am not able to control the VIF.
...
- 21
0
votes
0
answers
13
views
Is it incorrect to calculate residuals directly from a phylogenetically-controlled linear regression?
I would like to calculate the residuals from a regression of log body mass and log brain mass, controlling for phylogeny. I originally used phylolm in R to run this regression, under a Brownian Motion ...
- 193
0
votes
0
answers
5
views
How to train a ML pipeline with a classification model followed by a regression model for a given class
I am trying to create a pipeline that has 2 distinct models: a multiclass-classifier and a regressor.
The main goal is to predict for 3 different classes using the classifier and if the output belongs ...
3
votes
1
answer
37
views
How can I compute correct standard errors after implementing the FWL Theorem?
I am trying to implement the FWL theorem for some sample data in Stata. This theorem tells us that given a multivariate regression of the form $y = \beta_{1}x_{1} + \beta_{2}x_{2} + \varepsilon$, the ...
- 43
0
votes
0
answers
11
views
Does one need consistent estimate of "$S$" in ordinary least squares for unbiased, consistent fixed-effects level-one estimate and correct inference? [closed]
What is a complete list of the usual assumptions for linear regression? says that one obtains a Consistent Estimate of $S$ when the regressors have finite fourth moments.
$$
\hat{S}=\frac{1}{n} \sum_{...
- 782
0
votes
0
answers
11
views
Reasoning about modelling uncertainty w.r.t input
I am trying to build up my reasoning about uncertainty modelling and ways of modelling it.
What I am trying to essentially get at is how changes in input variables results in different posteriors(...
1
vote
0
answers
22
views
What parameters of a neural network should be changed when increasing the number of training samples?
I was testing a DL model (GNN with two GCN layers and one linear layer) on a small dataset for a regression purpose, the resulted MAE and the scatter plot showed some really good results.
However, ...
- 11
3
votes
1
answer
532
views
For linear regression, why do people usually standardize the X variables and log transform Y variables to make them normally distributed?
In many Kaggle competitions where linear regression has been applied, I see people plot the y distribution and then take the log of (or other transformation of) the dependent variable to make y normal ...
- 111