Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
6
votes
1answer
1k views

Inverse logistic regression vs. repeated-measures vs. latent class?

As the title suggests, I'm pretty well befuddled about which approach makes the most sense for my data. Let me try to succinctly explain the problem. I have binary choice data representing whether a ...
17
votes
2answers
3k views

Change point analysis using R's nls()

I'm trying to implement a "change point" analysis, or a multiphase regression using nls() in R. Here's some fake data I've made. The formula I want to use to fit ...
21
votes
2answers
96k views

How to use weights in function lm in R?

Could anyone offer some pointers on how to use the weights argument in R's lm function? Say, for instance you were trying to ...
1
vote
2answers
1k views

How to test for statistical significance of regression coefficients in multivariate multiple regression?

How can I test the statistical significance of regression coefficients in multivariate multiple regression?
13
votes
1answer
3k views

Root mean square vs average absolute deviation?

Both Root Mean Square and Average absolute deviation seem like the measures of the magnitude of variability (especially when the variates are both +ve and -ve). What are the rules of thumb to choose ...
1
vote
2answers
9k views

How to calculate margin of error in linear regression?

As a basic question in Regression Analysis, I wanted to ask how can I calculate Margin of Error when I fit a straight line to a set of data. Assume that I have variation of parameter $A$ as a function ...
5
votes
2answers
10k views

Is automatic linear modelling in SPSS a good or bad thing?

In SPSS Version 19 there seems to be a new feature called Automatic Linear Modelling. It creates a 'Model' (which is new to me) and the function seems to combine a number of the functions that is ...
6
votes
1answer
15k views

How to graphically compare predicted and actual values from multivariate regression in R?

I'm trying to write a function to graphically display predicted vs. actual relationships in a linear regression. What I have so far works well for linear models, but I'd like to extend it in a few ...
5
votes
2answers
1k views

What does plotting residuals from one regression against the residuals from another regression give us?

I am working with the dataset of some heights and weights at different ages. My professor wants me to plot the residuals from regression of soma.WT9 against the ...
20
votes
2answers
10k views

Estimating R-squared and statistical significance from penalized regression model

I am using the R package penalized to obtain shrunken estimates of coefficients for a dataset where I have lots of predictors and little knowledge of which ones are important. After I've picked tuning ...
12
votes
2answers
2k views

GLM after model selection or regularization

I would like to pose this question in two parts. Both deal with a generalized linear model, but the first deals with model selection and the other deals with regularization. Background: I utilize ...
2
votes
2answers
7k views

Regression with Pooled data in SPSS

Context: I have "pooled data" with time and cross section dimensions. It is unbalanced data without a full range of time observations for each cross section of observations. It is kind of like ...
8
votes
1answer
5k views

Multiple regression with small data sets

I have a dataset of project case studies for a new type of research method for Government agencies to support decision making activities. My task is to develop an estimation method based on past ...
41
votes
1answer
44k views

When and how to use standardized explanatory variables in linear regression

I have 2 simple questions about linear regression: When is it advised to standardize the explanatory variables? Once estimation is carried out with standardized values, how can one predict with new ...
3
votes
1answer
3k views

Bayesian additive regression trees (BART) for classification analysis of gene expression data

I am interested in applying Bayesian additive regression trees (BART) for classification analysis of gene expression data. I am relatively new to R (and Bioconductor packages) and I am unable to find ...
14
votes
3answers
9k views

GLMNET or LARS for computing LASSO solutions?

I would like to get the coefficients for the LASSO problem $$||Y-X\beta||+\lambda ||\beta||_1.$$ The problem is that glmnet and lars functions give different answers. For the glmnet function I ask ...
11
votes
2answers
12k views

Using poisson regression for continuous data?

Can the poisson distribution be used to analyze continuous data as well as discrete data? I have a few data sets where response variables are continuous, but resemble a poisson distribution rather ...
5
votes
3answers
2k views

Effect of missing data and outliers on least square estimation

Why is it that "missing data" and "outliers" can affect the performance of least square estimation?
4
votes
2answers
596 views

Logarithmic regression with individual errors

If I have a dataset of $N$ pairs ($x_i$,$y_i$) where each $y_i$ has an individual error $\sigma_{y_i}$, I can incorporate this into regression by using the inverse of this as weights. If I now do ...
2
votes
1answer
766 views

Aggregating results from Arima runs R

/edit: To clarify: The mtable function from the memisc package does exactly what I need, but unfortunately does not work with arima models. Similar to this question: I have multiple Arima models, ...
62
votes
6answers
16k views

Efficient online linear regression

I'm analysing some data where I would like to perform ordinary linear regression, however this is not possible as I am dealing with an on-line setting with a continuous stream of input data (which ...
1
vote
3answers
4k views

Displaying regression results in MATLAB

Along the same lines as this question, is there a nice way to display regression results in MATLAB from a single or many regressions in table or graph form?
17
votes
2answers
8k views

Aggregating results from linear model runs R

Since regression modeling is often more "art" than science, I often find myself testing many iterations of a regression structure. What are some efficient ways to summarize the information from these ...
9
votes
1answer
2k views

Least angle regression keeps the correlations monotonically decreasing and tied?

I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing). Consider a regression ...
3
votes
1answer
242 views

Assessing conditional independence of genes in Trans-eQTL cluster

I've identified a Single Nucleotide Polymorphism (SNP) that is associated with a variable of interest (lets just call it height) using a Genome Wide Association Study (GWAS). The SNP can be of the ...
10
votes
1answer
4k views

Using Singular Value Decomposition to Compute Variance Covariance Matrix from linear regression model

I have a design matrix of p regressors, n observations, and I am trying to compute the sample variance-covariance matrix of the parameters. I am trying to directly calculate it using svd. I am using ...
6
votes
2answers
8k views

Regression selection using all possible subsets selection and automatic selection techniques

Given the dataset cars.txt, we want to formulate a good regression model for the Midrange Price using the variables Horsepower, Length, Luggage, Uturn, Wheelbase, and Width. Both: using all possible ...
14
votes
2answers
1k views

Sane stepwise regression?

Suppose I want to build a binary classifier. I have several thousand features and only a few 10s of samples. From domain knowledge, I have a good reason to believe that the class label can be ...
4
votes
2answers
17k views

How can one use the predict function on a lm object where the IVs have been dynamically scaled?

Note: I've updated the example case code, there were some errors in the previous version Cross posted to R-help, because I half suspect this is 'unexpected behaviour'. I want to predict values from ...
5
votes
1answer
2k views

Composite dependent variable

I have been running a linear regression where my dependent variable is a composite. By this I mean that it is built up of components that are added and multiplied together. Specifically, for the ...
5
votes
2answers
2k views

Derivative of a linear model

Question Is there such concept in econometrics/statistics as a derivative of parameter $\hat{b_{p}}$ in a linear model with respect to some observation $X_{ij}$? By derivative I mean $\frac{\partial \...
11
votes
2answers
18k views

How can one plot continuous by continuous interactions in ggplot2? [closed]

Let's say I have data: ...
7
votes
3answers
7k views

Predicting from a simple linear model with lags in R

I have a dataset that I want to fit a simple linear model to, but I want to include the lag of the dependent variable as one of the regressors. Then I want to predict future values of this time series ...
7
votes
2answers
6k views

Predicting daily electricity load - fitting time series

I want to predict inter-day electricity load. My data are electricity loads for 11 months, sampled in 30 minute intervals. I also got the weather-specific data from a meteorological station (...
18
votes
1answer
5k views

LASSO assumptions

In a LASSO regression scenario where $y= X \beta + \epsilon$, and the LASSO estimates are given by the following optimization problem $ \min_\beta ||y - X \beta|| + \tau||\beta||_1$ Are there any ...
5
votes
2answers
330 views

Can statistical prediction be asymmetric?

Thus can we have random variables $X_1 , ... , X_n$ , each with zero mean and unit variance, such that for a sizeable representative sample: (1) in the least-squares regression equation for $X_1$ ...
4
votes
1answer
1k views

Question on AIC and stepAIC

AIC(lm(Fertility ~ ., data=swiss)) [1] 326.0716 ok, since AIC is calculated as -2*logLik(lm(Fertility ~ ., data=swiss)) + 2*7 ...
1
vote
1answer
484 views

Regression with repeated data

I have a question regarding regression analysis on a dataset were the input values generate different results over time: e.g. 1 2 2 2 3 5 4 1 2 5 3 8 How would ...
11
votes
5answers
1k views

Measuring Regression to the Mean in Hitting Home Runs

Anyone that follows baseball has likely heard about the out-of-nowhere MVP-type performance of Toronto's Jose Bautista. In the four years previous, he hit roughly 15 home runs per season. Last year he ...
6
votes
1answer
2k views

Non-parametric regression

I am conducting a mulitple first order regression analysis of genetic data. The vectors of y-values do not all follow a normal distribution, therefore I need to implement a non-parametric regression ...
12
votes
3answers
48k views

Can I use multiple regression when I have mixed categorical and continuous predictors?

It looks like you can use coding for one categorical variable, but I have two categorical and one continuous predictor variable. Can i use multiple regression for this in SPSS and if so how? thanks!
6
votes
1answer
8k views

Splitting one variable according to bins from another variable

I have continuous data "A", binary categorical data "O", gender/sex and age for several participants in a study. A linear model in R shows no correlation between A and age. I would now like to ...
11
votes
4answers
730 views

How to present the gain in explained variance thanks to the correlation of Y and X?

I'm searching how to (visually) explain simple linear correlation to first year students. The classical way to visualize would be to give an Y~X scatter plot with a straight regression line. ...
3
votes
3answers
6k views

What does RMS stand for?

A physics application I'm using reports for a first order fit of the three points below as $11.388612x - 301.878$. x, y 35, 0 430, 4861 656, 7000 It ...
17
votes
1answer
14k views

Fitting models in R where coefficients are subject to linear restriction(s)?

How should I define a model formula in R, when one (or more) exact linear restrictions binding the coefficients is available. As an example, say that you know that b1 = 2*b0 in a simple linear ...
15
votes
5answers
11k views

How to model prices?

I asked this question on the matemathics stackexchange site and was recommended to ask here. I'm working on a hobby project and would need some help with the following problem. A bit of context Let'...
6
votes
2answers
21k views

Can the multiple linear correlation coefficient be negative?

I am using IDL regression function to compute the multiple linear correlation coefficient... ...
5
votes
1answer
2k views

What is the prediction error while using deming regression (weighted total least squares)

Deming Regression is a regression technique taking into account uncertainty in both the explanatory and dependent variable. Although I have found some interesting references on the calculation of ...
3
votes
1answer
2k views

Why are my constraints getting dropped?

I'm currently applying the Roy Zelner test of poolability as shown in the excellent article of Andrea Vaona, in fact I'm working with panel N=17 T=5, and my model looks like this : $$Y_{it}= a_0+...
5
votes
2answers
5k views

R-squared result in linear regression and “unexplained variance”

I did a linear regression in R and got the following result: ...