Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

4
votes
1answer
143 views

What distribution to use to model changes in ratios?

I have some data corresponding to changes in a binomial variable - i.e, in month 1 there were n1 trials and k1 successes, and in month 2 there were n2 trials and k2 successes. Say I have M of these ...
10
votes
2answers
21k views

Explain model adjustment, in plain English

Reading about methods and results of statistical analysis, especially in epidemiology, I very often hear about adjustment or controlling of the models. How would you explain, to a non-statistician, ...
5
votes
4answers
2k views

Start time requirements or assumptions for survival analysis

We have prospective data from an observational registry and wish to consider the affects of a gene on time to cardiovascular events. The data includes standard data like age, gender, ... and also the ...
2
votes
2answers
5k views

Adding coefficients to obtain interaction effects - can I add standard errors?

I posted this question earlier and am rewriting it in hopes of getting some guidance. I am using a weighted regression (after propensity score matching) to obtain estimates of the effects of a ...
13
votes
3answers
7k views

Adding coefficients to obtain interaction effects - what to do with SEs?

I have a multivariate regression, which includes interactions. For example, to get the estimate of the treatment effect for the poorest quintile I need to add the coefficients from the treatment ...
2
votes
2answers
243 views

Should we regress x or use logistic regression on x>5000

Suppose we need to take an action on a population with income (x) more than $5,000. Income is not observed directly. Should we use logistic regression to estimate x, or should we use logistic ...
8
votes
4answers
3k views

Cox regression and time scale

Does X (hazard) variable in Cox proportional hazard regression analysis always have to be time? If not, could you provide an example, please? Can age of cancer patient be a hazard variable? If so, ...
19
votes
6answers
12k views

Dealing with correlated regressors

In a multiple linear regression with highly correlated regressors, what is the best strategy to use? Is it a legitimate approach to add the product of all the correlated regressors?
65
votes
9answers
39k views

Why is it possible to get significant F statistic (p<.001) but non-significant regressor t-tests?

In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests? In my model, there are 10 ...
8
votes
3answers
799 views

Doing regressions on samples from a very large file: are the means and SEs of the sample coefficients consistent estimators?

I have a fairly larege file 100M rows and 30 columns or so on which I would like to run multiple regressions. I have specialized code to run the regressions on the entire file, but what I would like ...
11
votes
3answers
13k views

Why use a lagged DV as an instrumental variable?

I have inherited some data analysis code that, not being an econometrician, I am struggling to understand. One model runs an instrumental variables regression with the following Stata command ...
3
votes
1answer
374 views

Using Covariance Estimator to Perform Linear Regression?

Suppose you had a method for estimating the population covariance of a vector-valued random variable given observations of that random variable, say $f(Z) \rightarrow C$, where the rows of $Z$ are ...
49
votes
5answers
35k views

Is adjusting p-values in a multiple regression for multiple comparisons a good idea?

Lets assume you are a social science researcher/econometrician trying to find relevant predictors of demand for a service. You have 2 outcome/dependent variables describing the demand (using the ...
1
vote
1answer
2k views

Cross-section and fixed effects models

Imagine 2 models: 1 for production, 1 for revenue, have cross-section and time dummies. Is it permissable to use CS in one and time in the other? How do I justify this? F-test for fixed effects is ...
7
votes
4answers
12k views

Linear model with constraints

I'm quite new to R and I have a following problem: I have a simple 2-factor linear model: ...
4
votes
1answer
322 views

Quantifying the degree of consistency of two fitted curves

I previously asked how to estimate the latent potential of a runner who ran the 100 metres each day for 200 days. Latent skill was defined as "the latent time it would take the individual to run if ...
6
votes
3answers
2k views

Regression analysis and parameter estimates with populations

I've seen a little bit here about the difference between statistical inference for random samples, and what happens when we actually have population data. Most arguments seem to suggest you "never ...
12
votes
3answers
2k views

OLS is BLUE. But what if I don't care about unbiasedness and linearity?

The Gauss-Markov theorem tells us that the OLS estimator is the best linear unbiased estimator for the linear regression model. But suppose I don't care about linearity and unbiasedness. Then is ...
4
votes
3answers
269 views

What is a good internet based source of information on Hierarchical Modeling?

I am talking about the regression method that measures the impact of several layers of independent variables upon a dependent variable.
14
votes
3answers
15k views

How to detect when a regression model is over-fit?

When you are the one doing the work, being aware of what you are doing you develop a sense of when you have over-fit the model. For one thing, you can track the trend or deterioration in the Adjusted ...
5
votes
4answers
4k views

Significance of the slope of a straight line fit

I know the significance of the slope of a LMS linear regression can be calculated using the r2 coefficient of determination and looking up the appropriate value in an F table. However, I was thinking ...
3
votes
2answers
13k views

How to conduct conditional Cox regression for matched case-control study?

I am attempting to find a program that will let me conduct Cox regression on my matched case-control dataset. Please assist. p.s. I have STATA, SPSS, and MedCalc
8
votes
2answers
1k views

Whether to use robust linear regression or bootstrapping when there is heteroscedasticity?

I have a dataset where I need to do linear regression. Unfortunately there is a problem with heteroscedasticity. I´ve rerun the analysis using robust regression with the HC3 estimator for the variance ...
6
votes
1answer
12k views

Difference between Norm of Residuals and what is a “good” Norm of Residual

I am doing some basic fitting of data and exploring different fits. I understand that the residual is the difference between the sample and the estimated function value. The norm of the residuals is a ...
6
votes
2answers
3k views

Pitman's test of equality of variance and testing for regression to the mean: am I doing the right thing?

I have 2560 paired observations from an experiment in which participants provided two ratings for a set of objects, at two different points in time. Half of the objects in the set had the value of an ...
7
votes
3answers
494 views

How to choose df for comparisons between summary statistics (e.g. slope values)?

In order to correlate or compare means of two dependent variables. In my case, I need to correlate individual (e.g. subjects=30) slope values from different conditions (e.g. conditions=4), and each ...
14
votes
4answers
2k views

LOESS that allows discontinuities

Is there a modelling technique like LOESS that allows for zero, one, or more discontinuities, where the timing of the discontinuities are not known apriori? If a technique exists, is there an existing ...
13
votes
3answers
2k views

Can CART models be made robust?

A colleague in my office said to me today "Tree models aren't good because they get caught by extreme observations". A search here resulted in this thread that basically supports the claim. Which ...
52
votes
4answers
95k views

Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math

Are multiple and multivariate regression really different? What is a variate anyways?
55
votes
6answers
17k views

Alternatives to logistic regression in R

I would like as many algorithms that perform the same task as logistic regression. That is algorithms/models that can give a prediction to a binary response (Y) with some explanatory variable (X). ...
7
votes
2answers
657 views

Preferred method for identifying curvilinear effect in multi-variable regression framework

Say some previous findings identified a curvilinear effect of X on Y, (specifically that X had a positive effect on Y, and that X^2 had a negative effect). You want to see if the same holds for your ...
9
votes
3answers
10k views

Linear regression effect sizes when using transformed variables

When performing linear regression, it is often useful to do a transformation such as log-transformation for the dependent variable to achieve better normal distribution conformation. Often it is also ...
89
votes
10answers
250k views

What's the difference between correlation and simple linear regression?

In particular, I am referring to the Pearson product-moment correlation coefficient.
9
votes
3answers
2k views

Computing best subset of predictors for linear regression

For the selection of predictors in multivariate linear regression with $p$ suitable predictors, what methods are available to find an 'optimal' subset of the predictors without explicitly testing all $...
26
votes
6answers
26k views

How do I decide what span to use in LOESS regression in R?

I am running LOESS regression models in R, and I want to compare the outputs of 12 different models with varying sample sizes. I can describe the actual models in more details if it helps with ...
4
votes
2answers
122 views

Regression specification choices

I am studying a population of individuals who all begin with a measureable score of interest (ranging from -2 to 2) [call it "old"], then they all undergo a change to a new score (also ranging from -2 ...
42
votes
6answers
25k views

What algorithm is used in linear regression?

I usually hear about "ordinary least squares". Is that the most widely used algorithm used for linear regression? Are there reasons to use a different one?
-2
votes
1answer
281 views

Shall I trust AIC (non-full model) or slope (full model)?

The purpose to run regressions for butterfly richness again 5 environmental variables is to show the importance rank of the independent variables mainly by AIC. In non-full models, they reveal that ...
5
votes
2answers
4k views

Method to compare variable coefficient in two regression models

I am regressing two butterfly richness variables (summer and winter) against a set of environmental variables separately. (variables with continuous numbers) Environmental variables are identitcal ...
42
votes
5answers
42k views

If the t-test and the ANOVA for two groups are equivalent, why aren't their assumptions equivalent?

I'm sure I've got this completely wrapped round my head, but I just can't figure it out. The t-test compares two normal distributions using the Z distribution. That's why there's an assumption of ...
28
votes
3answers
40k views

Regression coefficients that flip sign after including other predictors

Imagine You run a linear regression with four numeric predictors (IV1, ..., IV4) When only IV1 is included as a predictor the standardised beta is +.20 When you ...
10
votes
1answer
1k views

What type of post-fit analysis of residuals do you use?

When carrying out OLS multiple linear regression, rather than plot the residuals against fitted values, I plot the (internal) Studentized residuals against fitted values (ditto for covariates). These ...
21
votes
3answers
11k views

Coefficient of Determination ($r^2$): I have never fully grasped the interpretation

I want to fully grasp the notion of $r^2$ describing the amount of variation between variables. Every web explanation is a bit mechanical and obtuse. I want to "get" the concept, not just mechanically ...
6
votes
3answers
1k views

Mixed regression models and custom link functions in R?

It seems like the current revision of lmer does not allow for custom link functions. If one needs to fit a logistic linear mixed effect model with a custom link function what options are available ...
12
votes
5answers
5k views

What do ROC curves tell you that traditional inference wouldn't?

When would you tend to use ROC curves over some other tests to determine the predictive ability of some measurement on an outcome? When dealing with discrete outcomes (alive/dead, present/absent), ...
79
votes
9answers
28k views

Is there an intuitive explanation why multicollinearity is a problem in linear regression?

The wiki discusses the problems that arise when multicollinearity is an issue in linear regression. The basic problem is multicollinearity results in unstable parameter estimates which makes it very ...
4
votes
1answer
1k views

Regression-multiple observations per subject

I have data for about 1 year, 100 observations, multiple observations per subject, transactions occur on weekly basis but have 6-12 subjects per week, there is no order to this. There is a policy ...
-1
votes
3answers
3k views

Can I predict percentage change in earnings from percentage change in produced and percentage changed in price? [closed]

I have computed percentage change from time1 to time2 for several variables. Can I predict percentage change in earnings from percentage change in produced and percentage changed in price? When I ...
9
votes
5answers
6k views

Is a variable significant in a linear regression model?

I've got a linear regression model with the sample and variable observations and I want to know: Whether a specific variable is significant enough to remain included in the model. Whether another ...
4
votes
8answers
681 views

Relationships between two variables

Comparing two variables, I came up with the following chart. the x, y pairs represent independent observations of data on the field. I've doen Pearson correlation on it and have found one of 0.6. ...