Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
1
vote
2answers
9k views

How to calculate margin of error in linear regression?

As a basic question in Regression Analysis, I wanted to ask how can I calculate Margin of Error when I fit a straight line to a set of data. Assume that I have variation of parameter $A$ as a function ...
5
votes
2answers
10k views

Is automatic linear modelling in SPSS a good or bad thing?

In SPSS Version 19 there seems to be a new feature called Automatic Linear Modelling. It creates a 'Model' (which is new to me) and the function seems to combine a number of the functions that is ...
6
votes
1answer
16k views

How to graphically compare predicted and actual values from multivariate regression in R?

I'm trying to write a function to graphically display predicted vs. actual relationships in a linear regression. What I have so far works well for linear models, but I'd like to extend it in a few ...
5
votes
2answers
1k views

What does plotting residuals from one regression against the residuals from another regression give us?

I am working with the dataset of some heights and weights at different ages. My professor wants me to plot the residuals from regression of soma.WT9 against the ...
20
votes
2answers
10k views

Estimating R-squared and statistical significance from penalized regression model

I am using the R package penalized to obtain shrunken estimates of coefficients for a dataset where I have lots of predictors and little knowledge of which ones are important. After I've picked tuning ...
12
votes
2answers
2k views

GLM after model selection or regularization

I would like to pose this question in two parts. Both deal with a generalized linear model, but the first deals with model selection and the other deals with regularization. Background: I utilize ...
2
votes
2answers
7k views

Regression with Pooled data in SPSS

Context: I have "pooled data" with time and cross section dimensions. It is unbalanced data without a full range of time observations for each cross section of observations. It is kind of like ...
8
votes
1answer
5k views

Multiple regression with small data sets

I have a dataset of project case studies for a new type of research method for Government agencies to support decision making activities. My task is to develop an estimation method based on past ...
41
votes
1answer
44k views

When and how to use standardized explanatory variables in linear regression

I have 2 simple questions about linear regression: When is it advised to standardize the explanatory variables? Once estimation is carried out with standardized values, how can one predict with new ...
3
votes
1answer
3k views

Bayesian additive regression trees (BART) for classification analysis of gene expression data

I am interested in applying Bayesian additive regression trees (BART) for classification analysis of gene expression data. I am relatively new to R (and Bioconductor packages) and I am unable to find ...
14
votes
3answers
9k views

GLMNET or LARS for computing LASSO solutions?

I would like to get the coefficients for the LASSO problem $$||Y-X\beta||+\lambda ||\beta||_1.$$ The problem is that glmnet and lars functions give different answers. For the glmnet function I ask ...
11
votes
2answers
13k views

Using poisson regression for continuous data?

Can the poisson distribution be used to analyze continuous data as well as discrete data? I have a few data sets where response variables are continuous, but resemble a poisson distribution rather ...
5
votes
3answers
2k views

Effect of missing data and outliers on least square estimation

Why is it that "missing data" and "outliers" can affect the performance of least square estimation?
4
votes
2answers
606 views

Logarithmic regression with individual errors

If I have a dataset of $N$ pairs ($x_i$,$y_i$) where each $y_i$ has an individual error $\sigma_{y_i}$, I can incorporate this into regression by using the inverse of this as weights. If I now do ...
2
votes
1answer
773 views

Aggregating results from Arima runs R

/edit: To clarify: The mtable function from the memisc package does exactly what I need, but unfortunately does not work with arima models. Similar to this question: I have multiple Arima models, ...
63
votes
6answers
17k views

Efficient online linear regression

I'm analysing some data where I would like to perform ordinary linear regression, however this is not possible as I am dealing with an on-line setting with a continuous stream of input data (which ...
1
vote
3answers
4k views

Displaying regression results in MATLAB

Along the same lines as this question, is there a nice way to display regression results in MATLAB from a single or many regressions in table or graph form?
17
votes
2answers
8k views

Aggregating results from linear model runs R

Since regression modeling is often more "art" than science, I often find myself testing many iterations of a regression structure. What are some efficient ways to summarize the information from these ...
9
votes
1answer
2k views

Least angle regression keeps the correlations monotonically decreasing and tied?

I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing). Consider a regression ...
3
votes
1answer
243 views

Assessing conditional independence of genes in Trans-eQTL cluster

I've identified a Single Nucleotide Polymorphism (SNP) that is associated with a variable of interest (lets just call it height) using a Genome Wide Association Study (GWAS). The SNP can be of the ...
10
votes
1answer
4k views

Using Singular Value Decomposition to Compute Variance Covariance Matrix from linear regression model

I have a design matrix of p regressors, n observations, and I am trying to compute the sample variance-covariance matrix of the parameters. I am trying to directly calculate it using svd. I am using ...
6
votes
2answers
8k views

Regression selection using all possible subsets selection and automatic selection techniques

Given the dataset cars.txt, we want to formulate a good regression model for the Midrange Price using the variables Horsepower, Length, Luggage, Uturn, Wheelbase, and Width. Both: using all possible ...
14
votes
2answers
1k views

Sane stepwise regression?

Suppose I want to build a binary classifier. I have several thousand features and only a few 10s of samples. From domain knowledge, I have a good reason to believe that the class label can be ...
4
votes
2answers
17k views

How can one use the predict function on a lm object where the IVs have been dynamically scaled?

Note: I've updated the example case code, there were some errors in the previous version Cross posted to R-help, because I half suspect this is 'unexpected behaviour'. I want to predict values from ...
5
votes
1answer
2k views

Composite dependent variable

I have been running a linear regression where my dependent variable is a composite. By this I mean that it is built up of components that are added and multiplied together. Specifically, for the ...
5
votes
2answers
2k views

Derivative of a linear model

Question Is there such concept in econometrics/statistics as a derivative of parameter $\hat{b_{p}}$ in a linear model with respect to some observation $X_{ij}$? By derivative I mean $\frac{\partial \...
11
votes
2answers
18k views

How can one plot continuous by continuous interactions in ggplot2? [closed]

Let's say I have data: ...
7
votes
3answers
7k views

Predicting from a simple linear model with lags in R

I have a dataset that I want to fit a simple linear model to, but I want to include the lag of the dependent variable as one of the regressors. Then I want to predict future values of this time series ...
7
votes
2answers
6k views

Predicting daily electricity load - fitting time series

I want to predict inter-day electricity load. My data are electricity loads for 11 months, sampled in 30 minute intervals. I also got the weather-specific data from a meteorological station (...
18
votes
1answer
5k views

LASSO assumptions

In a LASSO regression scenario where $y= X \beta + \epsilon$, and the LASSO estimates are given by the following optimization problem $ \min_\beta ||y - X \beta|| + \tau||\beta||_1$ Are there any ...
5
votes
2answers
330 views

Can statistical prediction be asymmetric?

Thus can we have random variables $X_1 , ... , X_n$ , each with zero mean and unit variance, such that for a sizeable representative sample: (1) in the least-squares regression equation for $X_1$ ...
4
votes
1answer
1k views

Question on AIC and stepAIC

AIC(lm(Fertility ~ ., data=swiss)) [1] 326.0716 ok, since AIC is calculated as -2*logLik(lm(Fertility ~ ., data=swiss)) + 2*7 ...
1
vote
1answer
518 views

Regression with repeated data

I have a question regarding regression analysis on a dataset were the input values generate different results over time: e.g. 1 2 2 2 3 5 4 1 2 5 3 8 How would ...
12
votes
5answers
1k views

Measuring Regression to the Mean in Hitting Home Runs

Anyone that follows baseball has likely heard about the out-of-nowhere MVP-type performance of Toronto's Jose Bautista. In the four years previous, he hit roughly 15 home runs per season. Last year he ...
6
votes
1answer
2k views

Non-parametric regression

I am conducting a mulitple first order regression analysis of genetic data. The vectors of y-values do not all follow a normal distribution, therefore I need to implement a non-parametric regression ...
12
votes
3answers
48k views

Can I use multiple regression when I have mixed categorical and continuous predictors?

It looks like you can use coding for one categorical variable, but I have two categorical and one continuous predictor variable. Can i use multiple regression for this in SPSS and if so how? thanks!
6
votes
1answer
8k views

Splitting one variable according to bins from another variable

I have continuous data "A", binary categorical data "O", gender/sex and age for several participants in a study. A linear model in R shows no correlation between A and age. I would now like to ...
11
votes
4answers
734 views

How to present the gain in explained variance thanks to the correlation of Y and X?

I'm searching how to (visually) explain simple linear correlation to first year students. The classical way to visualize would be to give an Y~X scatter plot with a straight regression line. ...
3
votes
3answers
6k views

What does RMS stand for?

A physics application I'm using reports for a first order fit of the three points below as $11.388612x - 301.878$. x, y 35, 0 430, 4861 656, 7000 It ...
17
votes
1answer
14k views

Fitting models in R where coefficients are subject to linear restriction(s)?

How should I define a model formula in R, when one (or more) exact linear restrictions binding the coefficients is available. As an example, say that you know that b1 = 2*b0 in a simple linear ...
15
votes
5answers
11k views

How to model prices?

I asked this question on the matemathics stackexchange site and was recommended to ask here. I'm working on a hobby project and would need some help with the following problem. A bit of context Let'...
6
votes
2answers
22k views

Can the multiple linear correlation coefficient be negative?

I am using IDL regression function to compute the multiple linear correlation coefficient... ...
5
votes
1answer
2k views

What is the prediction error while using deming regression (weighted total least squares)

Deming Regression is a regression technique taking into account uncertainty in both the explanatory and dependent variable. Although I have found some interesting references on the calculation of ...
3
votes
1answer
2k views

Why are my constraints getting dropped?

I'm currently applying the Roy Zelner test of poolability as shown in the excellent article of Andrea Vaona, in fact I'm working with panel N=17 T=5, and my model looks like this : $$Y_{it}= a_0+...
5
votes
2answers
5k views

R-squared result in linear regression and “unexplained variance”

I did a linear regression in R and got the following result: ...
2
votes
1answer
2k views

Interpretation of log likelihood and covariate significance in Cox regression

Another newbie question here (probably piece of cake for you guys). When I run a Cox Regression and one of my covariates come out as significant: ...
2
votes
1answer
734 views

How to handle age at measure-start in Cox regression?

I'm a statistics newbie (medical student) trying my luck with a Cox regression for a survival analysis on the outcome of a specific type of operation. And I'm trying to determine which variables to ...
101
votes
3answers
101k views

Does an unbalanced sample matter when doing logistic regression?

Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables. My problem is the following: ...
4
votes
2answers
6k views

Evaluating effect sizes of interactions in multiple regression

I have been running 3-level multilevel models with HLM, and my main interest is in some cross-level interaction effects that I am finding. My concern is that the effect sizes of these interactions ...
20
votes
2answers
80k views

How do I interpret Exp(B) in Cox regression?

I'm a medical student trying to understand statistics(!) - so please be gentle! ;) I'm writing an essay containing a fair amount of statistical analysis including survival analysis (Kaplan-Meier, Log-...