Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

0
votes
0answers
2 views

How does bootstrapping (wild) calculate a t-statistic?

I am using the wildbootstrap. Intuitively how do I calculate a t-critical values? Do I use original ones or do I somehow generate them?
0
votes
0answers
30 views

Calculating MSE for multiple regression

A study was carried out to examine the relationship between cholesterol level in the body (y) and two variables variables 𝑥1:= amount of daily walking (in steps) and 𝑥2:= age. A total number of 63 ...
7
votes
3answers
4k views

Is it valid to use a difference score as an independent variable in a regression analysis

I would like to see if the difference in the number of BPD symptoms from baseline to follow-up two years later can predict psychosocial functioning at the 2 year follow-up. So I wanted to do a linear ...
37
votes
8answers
35k views

Is it valid to include a baseline measure as control variable when testing the effect of an independent variable on change scores?

I am attempting to run an OLS regression: DV: Change in weight over a year (initial weight - end weight) IV: Whether or not you exercise. However, it seems reasonable that heavier people will lose ...
0
votes
0answers
3 views

Categorical control variables PROCESS

At the moment I want to perform a mediated regression analysis using PROCESS. My Independent value (IV), Dependent value (DV) and Mediator (M) are all numerical data. I analyze a simple mediation and ...
2
votes
0answers
93 views

Multivariate Regression with Uniform Errors

I'd like to know if there is a multivariate regression model with uniform errors. More specifically, I'd like to find the Maximum Likelihood Estimators for $\beta^a$ and $\beta^b$. The model is given ...
65
votes
2answers
37k views

Solving for regression parameters in closed-form vs gradient descent

In Andrew Ng's machine learning course, he introduces linear regression and logistic regression, and shows how to fit the model parameters using gradient descent and Newton's method. I know gradient ...
0
votes
0answers
6 views

Correlation between external variables and model coefficients over time

Are there any techniques which can allow me to test for correlations between a set of variables (e.g. population, disposable income) and the time-varying coefficients of a model? I would like to ...
0
votes
0answers
14 views

How is the correlation matrix in table 3.5 in the book ISLR calculated? What does jt signify?

I was reading the book Introduction to statistical learning by Hastie and Tibshirani and there was this relationship between newspaper, radio & tv ads and their effect on sales in multiple linear ...
0
votes
0answers
4 views

question about interpretation of results seemingly unrelated estimation (suest) method stata probit model

i use seemingly unrelated estimation (suest) method with probit model in stata but i can't interpret the results i would like to interpret on any group the relation with dependent variables and the ...
33
votes
2answers
27k views

What is elastic net regularization, and how does it solve the drawbacks of Ridge ($L^2$) and Lasso ($L^1$)?

Is elastic net regularization always preferred to Lasso & Ridge since it seems to solve the drawbacks of these methods? What is the intuition and what is the math behind elastic net?
0
votes
0answers
15 views

How to prove that X'ε=0 in regression models? [duplicate]

If X is a nxn matrix of some data exogenous to and ε is a 1xn matrix of residuals that sum up to zero, why is X'ε=0? EDIT: It's clear to me why ε sum to zero, but I see no reason why this should ...
1
vote
0answers
52 views
+50

Covariance matrix and persistence of excitation of input

Assume that a discrete-time system can be described by the following state-space equations $x(k+1)=Ax(k)+Bu(k)+w(k)$ where the input signal $u(k)$ is stationary and ergodic with $\mathbf{E}[u(k)]=0$....
0
votes
0answers
10 views

Application of oil statics in oil and gas industry [on hold]

How can I determine relationship between number of wells and annual oil production by using statics?
0
votes
1answer
21 views

Adding a 2nd interaction term makes 1st interaction term and the 2nd interaction term insignificant

I'm running a multiple linear OLS regression (X => Y) on a sample with 125 cases. My regression has two moderator variables (Z and W). Z and W correlate at about .4 but Z and W are two distinct/...
0
votes
1answer
15 views

How to check for the presence of constant difference within dummy variable?

In econometrics, should two subgroups of the data (for dummy=0 and dummy=1) have some variation, that can be explained also with a constant difference, how would one test for it? And is it a problem (...
0
votes
0answers
23 views

How do I compare the predictive power of two predictors within a single (logistic) regression?

I've read about how F-tests can be used to compare models and to decide whether an additional variable should be included in the regression. However, I want to test whether A vs. B are better ...
0
votes
0answers
9 views

logistic regression manipulation [on hold]

hello guys i have a work to do of logistic regression and i'm little bit confused by the way that i must use my data i have about 19 variable and are all factor with labels ofen like yes or no so the ...
0
votes
0answers
6 views

Is it right of selecting model based on Goodness of fit R2 computed for whole data set

In my regression model building process, I split the data as training and testing set and used all my testing data as external data set and did not use it for training procedure and I have ended up ...
0
votes
1answer
2k views

Multiplicative regression coefficient

The expected model is: $$ Y = X1*X2*X3 $$ Taking logs we should get a coefficient of $1$ on all three when we regress. That is: $$ \ln (Y) = b + b1 \ln(X1) + b2 \ln(X2) + b3 \ln(X3), \\ \text{gives: ...
1
vote
0answers
41 views

Logistic regression with rare events

Could somebody provide some references on the application of logistic regression for modelling rare events? So far I've come across three methods with the main references below: Exact logistic ...
0
votes
1answer
232 views

How can I see a covariance matrix, as a standard regression y-Bx

ok so I have 2 assets, asset A and asset B, this assets have a vector of returns , 30 observations each. I calculate the estimated return as mean of the asset vector A and for B as a mean of asset ...
2
votes
2answers
55 views

Modeling Opioid Mortality Rates using Poisson Regression

This is a general statistics question about Poisson Regression. I have age-adjusted and crude rates for opioid mortality for the period 2014-2016. I want to use Poisson regression, but I am not sure ...
0
votes
1answer
320 views

How to use categorical explanatory variables in a multiple regression having an ordinal “score” response variable?

I have limited statistical experience from my coursework in undergrad running simple linear regressions and performing chi-square tests. I have some data, ~5000 survey results on individuals, each ...
0
votes
1answer
44 views

Indentify subset of data that follow linear regression

I have a dataset that is split into two different behavior, a first part of the data follow a linear model and the second part of the data follows a log model. My problem is that I need to get the ...
1
vote
1answer
23 views

Making the mindset transition - projection in OLS regression [duplicate]

For example, the space spanned by the columns in $$\mathbf{X} = \begin{bmatrix} 0 & 0 \\ 1 & 0 \\ 0 & 1 \end{bmatrix}$$ is the y-z plane. Further, $$\mathbf{X'X} = \begin{bmatrix} 1 &...
0
votes
0answers
14 views

Solution of an endogeneity model without instrumental variable, where $E[\varepsilon |x]\neq0$

I know that using Instrumental Variable (IV) solves endogeneity, feedback, unbiased problems and other situations. I was wondering, besides IV, what other method comes to your minds to solve a model ...
1
vote
0answers
19 views

Logistic regression for complaints on insurance claims

Suppose you have a set of insurance claims and you want to predict the probability that a claim will give rise to a complaint from some features of the claim at a certain point in time such as time ...
0
votes
1answer
364 views

percent deviation for linear regression

I have 2 sets of experimental data to which I applied a linear fit using Matlab. I can use the slope value to compare between both of them. My question is: can I use the following percent deviation ...
1
vote
1answer
406 views

Interpreting how much my linear model has improved after Box-Cox transformation

I am working on a linear regression project where I first removed insignificant variables, then looked at a possible transform of the data. I performed the variable selection smoothly, however am ...
2
votes
1answer
38 views

Covariance of an estimate from optimization

Consider a standard linear regression model, $\boldsymbol y = X \boldsymbol \beta + \boldsymbol \epsilon$. $\boldsymbol y$ is a vector of $m$ responses, $X$ is a design matrix with $m$ rows and $p$ ...
2
votes
1answer
64 views

Inclusion of standard error in regression equation

$$ \begin{alignat}{14} P = 11&.32 &+ 0&.71 \,\text{PASN} &+ 1&.54 \,\text{DIS} &- 1&.02 \,\text{DIS}^2 &+ 3&.44 \,\text{FUEL} &+ 1&.36 \,\text{FIRST} &...
1
vote
3answers
32 views

Commonly used example data sets

Is there a place where one can get standard example data for various statistics tools to try on? For example, if one is learning about ARIMA models, where would one get data that can be modelled well ...
68
votes
2answers
15k views

Why isn't Logistic Regression called Logistic Classification?

Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
0
votes
0answers
10 views

Use of adjusted R^2 when regressing using overlapping data

Context: Taking time series levels and converting to 10 day changes in levels. Then doing a linear regression on the 10 day changes over a period of time eg 500 overlapping samples, where Y is the ...
2
votes
1answer
195 views

Bias-variance docomposition of linear model fit in 'The Elements of Statistical Learning'

In section 7.3 of 'The Elements of Statistical Learning', the authors have shown the expression for bias-variance decomposition of linear model fit: However, I get a slightly different expression for ...
6
votes
2answers
854 views

a regression through the origin

Why do a pair of variables with no significant correlation and no significant regression intercept and slope, have a highly significant regression with high adjusted $R^2$ when the regression is ...
3
votes
1answer
719 views

Polynomial fit: removing outliers

I want to fit a scatter plot with a polynomial, and find the correlation between two variables. 1) How can I define and remove outliers from data points? (in the figure the outliers on the right ...
7
votes
2answers
102 views

What useful properties does the canonical link function have?

So here I am studying generalized linear models. I know this question is quite naive and simple, but I do not exactly know why the link canonical function is so useful. Could someone provide me an ...
0
votes
1answer
23 views

Bootstrap Confidence Bands for Linear Regression (in R)

I am looking for a way to implement non-parametric bootstrap to confidence bands around my regression line for my linear regression model. I am, however, new to bootstrap, therefore I am unsure how to ...
0
votes
0answers
9 views

Cross-validation error of ridge regression

Problem In order to find the optimal parameter $\lambda$, each individual observation is taken out from design matrix $\mathbf{X}$ and solves $$ \text{minimize}_{\beta} \frac{1}{2}\Vert \mathbf{y}_{-...
0
votes
0answers
8 views

Comparing regression/correlation values within-subjects? Repeated Measures ANOVA afters Fishers Transform?

I was interested in the most appropriate way to assess whether there is a difference in the R values from a regression analysis in a repeated measures design. I transformed my R values using a ...
0
votes
1answer
29 views

Coefficient Interpretation when dependent and independent variables are percentages

I have built a linear mixed regression model with fund returns (measured in percentage ie. 0.01 denotes one percent) as the dependent variable. For the independent variables I have percentage level of ...
0
votes
1answer
50 views

Computationally verifying the equivalence of ridge regression estimates and Bayesian regression estimates

I'm trying to show that the numerical estimates of ridge regression's parameter estimates are the same as the MAP parameter estimates of a Bayesian regression model with normal prior distributions. So ...
37
votes
6answers
12k views

Least-angle regression vs. lasso

Least-angle regression and the lasso tend to produce very similar regularization paths (identical except when a coefficient crosses zero.) They both can be efficiently fit by virtually identical ...
0
votes
0answers
23 views

Developing an appropriate volatility variable to predict stock returns based on past month

I am doing a project about the predictability of stock returns. I am using following regression model: \begin{equation} r_{t} = \alpha+\beta X_{t-1}+\epsilon_{t}, \end{equation} where $r_{t}$ is the ...
2
votes
1answer
135 views

Nonnegative generalized linear model

Is it possible that all the parameters of a generalized linear model are constrained to be non-negative? If so, when? Any examples?
2
votes
0answers
797 views

How to get an Effect size for Negative Binomial Regression?

Typically I use the confidence interval as an indicator of effect size for a negative binomial model, however, I have been told to provide an actual effect size (e.g., pseudo $R^2$). I have looked ...
0
votes
0answers
10 views

Interpreting regression coefficients with cube root transformation of both dependent and independent variables

I have used the cube root $(1/3)$ transformation on both my dependent and one of my independent variables. I would have preferred a log-transformation but my data has both negative values and zeroes. ...