Questions tagged [multiple-regression]
Regression that includes two or more non-constant independent variables.
1,960
questions with no upvoted or accepted answers
8
votes
0
answers
623
views
Calculate goodness-of-fit (with deviance) to compare averaged models?
I need to compare the goodness of fit of several averaged logistic regression models by calculating the deviance explained. I'm using the MuMIn package in R to ...
7
votes
0
answers
139
views
Identifying non-linearities in relationship between variables
Logistic regression is often used to identify the effect of $x$ on a binary variable $y$ after adjusting for potential confounders $x_1,...,x_n$. In the medical literature, I will sometimes encounter ...
- 440
7
votes
0
answers
2k
views
What are the differences between HC estimators and their small sample properties?
I am currently using R to run regression with the following code:
...
- 458
6
votes
0
answers
1k
views
Overfitting a neural network to a single batch as a sanity check - how small a loss value is small enough and long to run for?
I'm currently developing a neural network for a regression task. Following on from the advice given in places like here, here, and here I'm attempting to overfit my model to a single batch of 5 ...
- 91
6
votes
0
answers
68
views
Before using CV-selected Regression model for Inference, shouldn't model performance be evaluated on unused test set?
I just came across a biokinesiology paper that used some Machine Learning methods, but I think there is a flaw in their methodology.
The authors had data on stroke patients and used Lasso regression ...
- 547
6
votes
0
answers
113
views
using `lmer` to fit the linear mixed effects models
Edit: I know some people vote this question is off-topic since it is more like a Cross Validated question. However, I am not here to ask about the coding thing (but I might word in the wrong way). I ...
- 609
6
votes
1
answer
90
views
Importance of regressors in time series data
Could anyone recommend bibliography or name some useful methods to analyze which (exogenous) variables are most important in determining the value of a time series?
For context, I have a random time ...
- 698
5
votes
1
answer
317
views
Why do we need to model lower-order effects in models with interactions?
I recently saw a paper with a four-way interaction. That already is difficult to interpret (maybe if you have 1 or more categorical variables but definitely near impossible to interpret if all ...
- 838
5
votes
2
answers
4k
views
Linear model with both additive and multiplicative effects
In linear regression, the independent variables have an additive effect on the response (level-level regression):
$y=\beta_0+\beta_1x+\epsilon$
In a log-level regression, the independent variables ...
- 2,097
5
votes
1
answer
428
views
How to train Prediction model for longitudinal data, with large number of time points?
Given a longitudinal data, that has date (in month-year format) as one of the independent variables and other independent variables being Gross metric tonnes, Tensile strength(UTS), weight per unit ...
5
votes
0
answers
327
views
Equivalence of ANCOVA and repeated measures model
Consider an RCT with individuals i in 2 arms (group, with 0 = control and 1 = treatment) in which one metric outcome (score) is collected at baseline (pre) and after some treatment (post).
In an ...
- 481
5
votes
0
answers
339
views
Why shouldn't I standardize my predictors when putting them into a regression model?
There are multiple reasons for applying standardisation/mean centre for predictors before putting them into a regression model. However, in the literature, some people do not do so or even argue ...
- 1,155
5
votes
0
answers
2k
views
Covariance matrix of multivariate multiple regression coefficients
I would like to perform a regression analysis on a dataset comprising one independent variable (X) and two dependent variables (Y1 and Y2) which may be affected by correlated errors. R's stats::lm ...
- 51
5
votes
0
answers
4k
views
Is it possible to do a time series analysis with more than one explanatory variable?
I am working on a project, and I am absolutely new to forecasting and not so strong in statistics. I have an employee data for the last 7 years, along with the other variables like economic growth, ...
- 161
5
votes
0
answers
99
views
Testing differences between slopes by examining proportions rather than interactions
In a recent article published in Nature Genetics, Francioli et al. argue that the relationship between paternal age and mutation rate depends on what part of the genome you're looking at (intergenic ...
- 1,209
5
votes
0
answers
1k
views
Why would I need both a validation set & a test set if I'm not selecting a model?
I have a dataset with two features and one outcome. I was asked to separate the data into three parts such that 70% of the data is a training set, 20% is for validation and 10% for testing. The model ...
5
votes
0
answers
3k
views
VAR model for price forecasting in multiple time-series context. How to get "real figures" as forecasts?
Sorry for the rather long introduction, but since I was (legitimately) critizised for not explaining my cause and questions enough, I will do so now.
I would like to conduct a (price)-forecast based ...
- 391
5
votes
0
answers
519
views
Partial regression plots vs scatter plots for checking linearity
In a multiple linear regression analysis, what is the most suitable plot for checking linearity? I have seen a number of examples that use scatterplots as a preliminary test to use a linear model. But,...
- 95
5
votes
0
answers
242
views
Variance partitioning - why be cautious?
I'm about to use variance partitioning to interpret my results of a given model and across models and have come across various criticisms of it most notably by Pedhazur (1982, 1997). Also, the ...
- 51
5
votes
1
answer
3k
views
Comparing observed and predicted values across several measurements
As a neuropsychology graduate student with some experience in statistics (I'm usually the guy other psychologists come to with statistics problems after trying it themselves but before seeing a ...
- 227
5
votes
0
answers
3k
views
Why is correlation between y and $\hat{y}$ in a model with and without intercept equal?
I consider a very simple example with two models like this;
...
- 1,073
5
votes
0
answers
1k
views
Hierarchical regression with dummy variables
I need to perform hierarchical regression with dummy variables. I also need to check moderation by introducing in the model interactions of these dummy variables and the moderator. My questions are:
...
- 101
4
votes
2
answers
52
views
What is the ideal approach to determine relationship between candidate predictors and a dependent variable in a data driven way?
I have asked several related questions (1, 2, 3), but now I would like to ask the most basic questions and hope to get a very solid answer.
I have 40 treatment variables, and I am interested to find ...
- 2,481
4
votes
0
answers
256
views
Can we compare the effects of continuous covariate and categorical covariate on response variable in generalized linear regression?
I want to construct a linear model among several variables. The model is $y = \beta_0 + \beta_1 x + \beta_2 z + \varepsilon$, in which $x$ is a continuous variable, and $z$ is a dummy variable, i.e. $...
- 511
4
votes
0
answers
100
views
Multivariable residual analysis and goodness of fit
I am reading the paper Statistical Modeling: The Two Cultures (2001) by Leo Breiman.
In section 5.2 he claims:
Residual analysis is similarly unreliable. In a discussion after a presentation of ...
- 491
4
votes
0
answers
1k
views
How to prove an OLS estimator is inconsistent
I have two equations
$Y_i = \beta_0 + \beta_1X_i + \epsilon_i$
$X_i = Y_i + Z_i$
and additional information that $cov(\epsilon_i, Z_i) = 0$
And I need to prove that using the OLS in the first ...
- 381
4
votes
0
answers
164
views
Multiple regression for left-censored independent and dependent variables
I am interested in developing a predictive multiple regression model which predicts a concentration of one compound based on the measured concentrations of several other compounds. Both the dependent ...
- 61
4
votes
0
answers
61
views
A simulation study of linear mixed model
I am reading this paper, a note on BIC in mixed effects models, and I was trying to repeat their simulation study. And I will paste part of the experiment settings here to clarify my question.
Now, I ...
- 609
4
votes
1
answer
109
views
Lasso on squared parameter
Assume a linear regression problem where I want to force sparsity of some parameters. However, due to some physics, I know that one of my parameters is always positive. For instance, I have that
$$ ...
- 2,164
4
votes
0
answers
294
views
How can I find and categorise the effect size of a single coefficient in a multiple regression?
Question
How do I find the effect size for the different hierarchical multiple-level regressions used by papers in my review? And how do I categorise their effect size?
Detail
I’m publishing a ...
- 41
4
votes
0
answers
600
views
Benchmark priors for Bayesian ridge regression
Consider a Bayesian linear regression model
$$\mathbf{Y=X\beta} + \boldsymbol{\varepsilon}$$
where $\mathbf{Y} \in \mathbb{R}^n$ and $\mathbf{X} \in \mathbb{R}^{n,p}$ are given, $\boldsymbol{\...
- 311
4
votes
0
answers
88
views
Ill-behaved, nonnormal residuals of multiple regression: should I be concerned?
I have data on waist circumference (cm) (waist), gender, age and physical activity (vigorous MET minutes per week) (PA).
I was trying to run linear regression in R on the model waist ~ gender + age + ...
- 49
4
votes
0
answers
180
views
Continuous distributions as independent variable in regression
Problem: My research issue concerns logistic regression where each observation is an area, not a simple point. As such, each independent variable ($x_i$ of $\boldsymbol{X}$) is a distribution of ...
- 303
4
votes
0
answers
372
views
Connecting Poisson and multinomial models
Let's say we have multinomial counts $y_{jp}$ (corresponding to observations $j$ over categories $p=1,...P$) that are arranged in a table of $n$ rows and $P$ columns. Then say we have the log-linear ...
- 193
4
votes
1
answer
3k
views
How to create a regression model object from intercept and coefficients values only (without the database) in R
I want to recreate a regression model based on what was given in a scientific paper. They gave intercept and coefficient terms.
I know how to create regression models in R, but is this possible to ...
- 250
4
votes
2
answers
335
views
Calculate effect size (e.g., Pearson r) from the output of a multiple regression
I'm looking at a multiple regression analysis in which I'm only interested in 1 of the several variables included in the model. However, the bivariate associations are not reported. I'm wondering if I ...
- 505
4
votes
0
answers
691
views
What's the difference between a random intercept and a dummy variable?
I usually work in SAS or R, so when I code a GLM with a random intercept, it's usually pretty easy. However, I've been running into a few problems (too complicated to get into here) where it might be ...
- 551
4
votes
0
answers
1k
views
Coefficients of dummy variables in multiple regression
I am struggling with interpreting coefficients from a multiple regression analysis with multiple categorical (dummy) variables. I am running a linear mixed model with biodiversity (...
4
votes
0
answers
2k
views
Analysis of Multiple Time Series Data with Exogenous Shocks
Real Life ProblemThis one is a tough one and some crowd sourcing seems like a good way to get some feedback. I am trying to determine the effect of Non-Farm Payroll surprises on a subsector of the ...
- 141
4
votes
0
answers
625
views
Longitudinal data analysis where meaning and metric of response variable varies over time
Determining what factors predict change over time is a topic of investigation in many fields and there are a variety of readily implemented methods for analysing repeated measures in the same metric.
...
- 1,209
4
votes
0
answers
557
views
What is the best way to simulate data for a linear regression model?
I am concerned with simulating data for a linear regression model. I need to control the means, variances, and correlations (covariances) between the predictors and the criterion variable. In addition,...
- 6,412
4
votes
0
answers
542
views
Standardize continuous predictor variables on [0, 1] scale?
I'm working on a health care regression model predicting # of inpatient visits. My analysis dataset includes a number of hybrid continuous/categorical predictor variables which can hold values on a 0 ...
- 5,572
4
votes
0
answers
812
views
Assumption violations with heteroscedastic data and OLS regression
I'm trying to model the typical performance of an experimental approach I've developed. I have a total of 3000 observations for 72 different case studies. My observations consist of a reading for <...
4
votes
0
answers
200
views
Estimating covariate effects on Likert-type response variable
I am working on a problem where we elicit medical residents perceptions regarding the importance of specific core medical competencies (professionalism, collaboration, medical expertise, etc.) on a 5 ...
- 973
4
votes
0
answers
10k
views
Fitting a zero-inflated negative binomial regression with R
In this thread, I laid out a problem involving fitting a model that attempts to use minor league baseball statistics to predict success at the major league level (explained in full in the thread). ...
- 869
4
votes
0
answers
3k
views
How to do a multiple regression with ARIMA using R?
I am analyzing some tree physiology data (transpiration) in relation to a number of environmental variables (many of which are predictors such as temperature, PAR and vapour pressure deficit).
I ...
- 41
4
votes
0
answers
215
views
Relationship between LASSO T and LARS number of steps k
We can see on the figure (cf Least Angle Regression p30, Efron, Hastie, Johnstone, Tibshirani - link: Least Angle Regression) that there is a direct relationship between:
LASSO T absolute norm of $\...
- 41
3
votes
1
answer
86
views
Difference between multivariate regression and running multiple linear regression models for every dependent variable
This post is to understand differences between multivariate linear regression models (i.e multiple independent variables predicting multiple dependent variables) and running multiple linear ...
- 131
3
votes
2
answers
110
views
High Adj. $R^2$ (by economic data standards) but insignificant p-values
I'd like to start by saying I'm not a statistician - I have stats education at the Masters level, but no specialization or advanced work experience.
I'm currently trying to regress financial return ...
- 31
3
votes
0
answers
112
views
Why Hotelling's $T^2$ is taught in the book but it is not used anywhere so far?
I have seen various multivariate statistics book covering Hotelling $T^2$ statistics. However, when reading medical journals and papers, it does not show up anywhere.
Q1:Is this test of any practical ...
- 1,365