Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

4
votes
2answers
42 views

In linear regression, why are raw least squares residuals heteroskedastic?

In my course notes on a regression course with regards to the detection of heteroskedasticity there's the following quote: "Because the least-squares residuals have unequal variances even in the ...
3
votes
1answer
577 views

Dealing with zeros in a poisson regression

Our code goes through multiple stages of review. I wish to use the number of defects at an earlier stage of review as a "defect density" estimate for later stages. It sometimes happens that code has ...
0
votes
0answers
8 views

optimism error rate calculation

As posted here in the book Elements of Statistical Learning we drive to the following: $$ w = \frac{2}{N}\sum_1^Ncov(y_i, \hat{y_i}) $$ The derivation is based on the following properties $E_y E_{Y^...
0
votes
0answers
7 views

What is the risk of not oversampling randomly?

Random Oversampling involves supplementing the training data with multiple copies of some of the minority classes.Instead of duplicating every sample in the minority class, some of them may be ...
3
votes
1answer
137 views

Statistical analysis applied to methods coming out of Machine Learning

Most of the recent famous methods coming out of the machine learning, are supervised learning methods like Decision Trees, Random Forests, Deep Learning, SVMs. The more traditional supervised ...
0
votes
2answers
22 views

How to perform a regression for a multi-function?

How would one perform a regression analysis with a multi-function? That is, where a single X-value can correspond to a number of Y-values? Here is an example: Note that in this example, it is not ...
1
vote
1answer
22 views

Inconsistency between multivariate logistic regression and independent t test

I have data in which there are 1 dependent continuous variable ($Y$) and 20 independent continuous variables ($X$s). I discretized $Y$ into two categories ($low$ and $high$). Then, I applied ...
-1
votes
1answer
16 views

Effect size for linear models

I´m running simple linear models in R y~x but always got a really small R², although the p-Value always shows significance!. Now I wanted to calculate an effect size because I want to know whether ...
8
votes
3answers
3k views

Comparing nested binary logistic regression models when $n$ is large

To better ask my question, I have provided some of the outputs from both a 16 variable model (fit) and a 17 variable model (fit2)...
1
vote
1answer
2k views

Evaluating/Tuning hyper parameters of a neural network regression model

I have a fairly reasonable understanding of the theory behind neural networks, regularization, and cross-validation, but I am lacking in the actual experience department. In a nutshell, I am using a ...
2
votes
0answers
54 views
+100

Failure to replicate calculation of PCA residuals in linear regression with heteroscedasticity

In their preprint, Rocha et al. suggest a new type of residual for linear regression models with heteroscedasticity. They call their new residual PCA residuals. I have tried to replicate some of their ...
2
votes
1answer
25 views

Building a Logistic regression with multiple dummies

I am building a model that has 10 dummy variables for a category called operator. The operator values are string, so I created binary variables to make sure each operator is within the model. I am ...
0
votes
1answer
828 views

Formal errors from non-negative least-squares?

I am computing a standard linear regression subject to a positivity constraint using non-negative least squares (lsqnonneg in Matlab, actually). Is it possible to ...
0
votes
0answers
70 views

Term for two variables that are “too close for control”

Sometimes we are tempted to assess a relationship of X1 with Y while controlling for X2, but it would be a mistake, because X2 is not merely correlated with Y -- it is more closely associated than ...
2
votes
2answers
22 views

Can censored data act as the dependent variable for a logistic regression?

I need to do the research about the risk factors that contribute to heart failure. The data I have just the censored data with some risk factors of heart failure. And I need to use logistic ...
0
votes
0answers
11 views

How to prove the estimator is consistent when there's a multiple regression without intercept? [on hold]

yi=B1*x1i+B2*x2i+ui I've tried many ways to prove B1 is consistent when we know x2i is observable and is uncorrelated with x1i.
0
votes
1answer
14 views

What type of regression for two groups of data?

I have data from 500 school children who took a test. 250 of the children have a certain type of disability (group A). Each child in group A was matched to a child on the basis of age, gender to a ...
0
votes
0answers
41 views

How to read lm() plots for models with factors

I'm trying to get a handle on how to read the Residuals vs Fitted and Scale-Location plots of lm() objects when the predictors are a mix of continuous and factor ...
1
vote
0answers
13 views

How would you see a trend a sales in years even there are fluctuations? [on hold]

How would you see a trend a sales in years even there are fluctuations? Please provide answer with details.
1
vote
1answer
2k views

Calculation of variance of prediction

Could anybody show me how @Rob Hyndman calculates the variance of $\hat{y}$ in the following link Obtaining a formula for prediction limits in a linear model : EDIT: Basically I don't understand how ...
0
votes
0answers
13 views

Logistic Regression: How to Detect Complete AND Quasi-Separation of Data Points

I have written a logistic regression routine using the Newton-Ralphson algorithm in VBA for use in a class that I teach which uses primarily EXCEL. I want the algorithm to test for complete and quasi-...
3
votes
1answer
636 views

How to calculate MSE in a quantile regression simulation study

I am working on a simulation study on quantile regression. So what I did is to simulate data based on a given model, which is different from the true underlying model of the data, in other words, a ...
2
votes
1answer
51 views

What is actually being modeled in binomial logistic regression?

One thing I've been struggling with for a while is this: When the binomial logistic regression model includes different number of trials across the observations, what are we estimating at the end of ...
4
votes
4answers
111 views

Why we try to capture variability?

I am new to Statistics and I have a Mathematics background. In Statistics, particularly in Linear Regression and Principal Component Analysis (PCA) so far what I have understood is that the main idea ...
3
votes
4answers
896 views

Multicollinearity and categorical predictor with three levels

If I have a continuous Dependent Variable and two Independent Variables, where one is categorical with three levels and the other is continuous, what assumptions do I need to check for multiple ...
0
votes
0answers
21 views

Logistic Regression - Coefficients not defined because of singularities

I am running a regression model to predict dropout from an online program. People have to take 5 classes but some people dropped before taking the 5 courses. So I am using a dummy variables that is 1 ...
0
votes
0answers
11 views

Random effect VS Pooled OLS

I am a second year MSc ACFN student at Addis Ababa University, Ethiopia. Now, I am conducting my research on the "effect of leverage on profitability and I use a panel data". When I was trying to ...
0
votes
0answers
14 views

Calculating predicted value from polynomial regression by hand

I have the following output from from R. I know we can calculate predicted values using the predict() function. How can we calculate the predicted value by hand if X=2 or 3? I'm using the below ...
10
votes
2answers
401 views

Showing the Equivalence Between the $ {L}_{2} $ Norm Regularized Regression and $ {L}_{2} $ Norm Constrained Regression Using KKT

According to the references Book 1, Book 2 and paper. It has been mentioned that there is an equivalence between the regularized regression (Ridge, LASSO and Elastic Net) and their constraint ...
0
votes
0answers
6 views

Correlations and Hierachical Regression

For my dissertation I've carried out a hierarchical regression. My results have shown: -correlations. Neuroticism is correlated non significantly .000 on SPSS to my dependent variable. Mood is non ...
0
votes
0answers
24 views

How to interpret the Z Value in probit regression summary

I am running a Probit model in R and find trouble interpreting the Z Value of -9.974. Obviously, it is not the same as the Z-score. So how do I interpret it and is t the same as the t-statistic on my ...
1
vote
3answers
34 views

How to detect Heteroscedasticity in a residual plot?

In this residual plot, both the increase and the decrease in the y variables are observed. In this case, how do you conclude whether heteroscedasticity exist or not? I am not sure if I can just simply ...
1
vote
0answers
18 views

G efficiency in Optimal designs

I am studying Optimal Designs and found a very interesting article by Peter Goos. In the article he provides an example in the form of an Excel document of generation of Optimal Designs for various ...
0
votes
0answers
7 views

Proof for multicollinearity consequence

I came across this statment "Even extreme multicollinearity (so long as it is not perfect) does not violate OLS assumptions. OLS estimates are still unbiased and BLUE (Best Linear Unbiased Estimators)"...
1
vote
1answer
176 views

Transforming functions for OLS regression help [on hold]

I was wondering if you would be able to help me with an econometric problem regarding OLS regression. Part of my research is transforming functions in order to be used in stata example: Q = e^(B0 +B1(...
2
votes
1answer
664 views

loan default model

I have a loan dataset that includes all the loans originated from 2000 through the most recent quarter. For each loan, available are information at origination, such as loan size, FICO, LTV, LTI etc......
0
votes
1answer
13 views

Analysis crossover clinical trial

I have data from a six-sequence 3-drug 3-phase crossover trial that had a 7-day baseline period pre-study (used to generate mean baseline value). My outcome is a non-parametric continuous variable ...
1
vote
1answer
63 views

Why divide by 1-leverage?

I'm reading about resampling methods, and specifically leave-one-out cross-validation. I understood the method, and how to calculate the estimate of the test MSE (Mean squared error): In the setup ...
0
votes
0answers
20 views

Regression percentage versus absolute levels

I regressed interest rates using the percentage changes for each respective rate. The r^2 was significantly lower, .5, versus .85 for regression pure rate values--not transforming them into ...
2
votes
0answers
34 views

Bias Variance Decomposition 2.7 in Elements of Statistical Inference

I try to derive 2.7 from the book. I expose my demonstration $E_\tau[(y_0-\hat{y}_0)^2]=E_\tau[y_0^2]-2E_{\tau}[y_{0}\hat{y_{0}}]+E_{\tau}[\hat{y_{0}}^{2}]$ $= y_{...
3
votes
1answer
24 views

How to estimate the “effect of an effect” identified through regression?

Suppose I estimate the effect of some variable $X_t$ on $Y_t$. Let's say that $X_t$ is some firm variable (age of CEO in year $t$ or something like that), and $Y_t$ is a firm's change in investment in ...
0
votes
0answers
11 views

Bayesian parameter estimation with varying subsets of data?

I'm currently working on a model in which I have 2 measurements, taken at different temperatures. The covariance between these measurements with temperature is assumed to be linear and therefore a ...
0
votes
0answers
7 views

Cost function increases dramatically at first then starts decreasing

I am training a simple linear regression algorithm using stochastic gradient descent. When plotting cost (MSE) vs number of iterations I get a plot which looks very strange: What would be the ...
0
votes
1answer
39 views

How do I implement stochastic gradient descent correctly?

I'm trying to implement stochastic gradient descent in MATLAB however I am not seeing any convergence. Mini-batch gradient descent worked as expected so I think that the cost function and gradient ...
2
votes
0answers
10 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
0
votes
0answers
18 views

Deriving covariance from supply and demand equations

I'm working on a problem from the textbook and I'm given two equations. $$ ln Q_i = \beta_0 + \beta_1 lnP_i +u_i $$ $$ ln P_i = \gamma_0 + \gamma_1lnQ_i + v_i $$ Show $$ cov(ln P_i, u_i) = \frac{\...
3
votes
1answer
49 views

What is the relationship between the sum of squares of all weights and lambda in the ridge regression [duplicate]

Currently I am reading chapter 8, regression. And I feel quite confused about the following paragraph(see picture below), does it mean in ridge algorithm, the sum of all weights will be less than ...
0
votes
1answer
229 views

Perfect separation in logistic regression and data transformation -> can it help?

first of all, I am super happy that I found this great community. I am currently having trouble in my logistic regression analysis in that I get the error message display ...
2
votes
1answer
62 views

My data has overdispersion but the Hurdle model estimated theta is 0. What am I doing wrong?

I am confused by the dispersion parameter from my model. My data fails the overdispersion test. It's mean is 28.7, the variance is 18655.27. N=2916 of which 32% are zeros. How can theta equal 0 in ...
0
votes
0answers
21 views

Thin Plate Regression Splines mgcv

I am struggling on the understanding of thin plate regression splines. I already found a very helpful answer here in cross-validated: smoothing methods for gam in mgcv package but I still have some ...