Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

0
votes
0answers
4 views

Random effect VS Pooled OLS

I am a second year MSc ACFN student at Addis Ababa University, Ethiopia. Now, I am conducting my research on the "effect of leverage on profitability and I use a panel data". When I was trying to ...
0
votes
0answers
4 views

How to calculate effect size estimates and 95% CI for regression parameters for a GLM?

I have a GLM with formula model_1 <- glm.nb(Number ~ WL, data = data_1, link = "identity") My sample size for this model is 30. The summary of the model is ...
0
votes
0answers
4 views

Proof for multicollinearity consequence

I came across this statment "Even extreme multicollinearity (so long as it is not perfect) does not violate OLS assumptions. OLS estimates are still unbiased and BLUE (Best Linear Unbiased Estimators)"...
0
votes
0answers
9 views

Logistic Regression - Coefficients not defined because of singularities

I am running a regression model to predict dropout from an online program. People have to take 5 classes but some people dropped before taking the 5 courses. So I am using a dummy variables that is 1 ...
1
vote
3answers
28 views

How to detect Heteroscedasticity in a residual plot?

In this residual plot, both the increase and the decrease in the y variables are observed. In this case, how do you conclude whether heteroscedasticity exist or not? I am not sure if I can just simply ...
0
votes
0answers
13 views

Regression percentage versus absolute levels

I regressed interest rates using the percentage changes for each respective rate. The r^2 was significantly lower, .5, versus .85 for regression pure rate values--not transforming them into ...
3
votes
1answer
17 views

How to estimate the “effect of an effect” identified through regression?

Suppose I estimate the effect of some variable $X_t$ on $Y_t$. Let's say that $X_t$ is some firm variable (age of CEO in year $t$ or something like that), and $Y_t$ is a firm's change in investment in ...
3
votes
4answers
95 views

Why we try to capture variability?

I am new to Statistics and I have a Mathematics background. In Statistics, particularly in Linear Regression and Principal Component Analysis (PCA) so far what I have understood is that the main idea ...
0
votes
0answers
10 views

Bayesian parameter estimation with varying subsets of data?

I'm currently working on a model in which I have 2 measurements, taken at different temperatures. The covariance between these measurements with temperature is assumed to be linear and therefore a ...
0
votes
0answers
6 views

Cost function increases dramatically at first then starts decreasing

I am training a simple linear regression algorithm using stochastic gradient descent. When plotting cost (MSE) vs number of iterations I get a plot which looks very strange: What would be the ...
2
votes
0answers
10 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
0
votes
0answers
13 views

Building a Logistic regression with multiple dummies

I am building a model that has 10 dummy variables for a category called operator. The operator values are string, so I created binary variables to make sure each operator is within the model. I am ...
2
votes
0answers
30 views

Bias Variance Decomposition 2.7 in Elements of Statistical Inference

I try to derive 2.7 from the book. I expose my demonstration $E_\tau[(y_0-\hat{y}_0)^2]=E_\tau[y_0^2]-2E_{\tau}[y_{0}\hat{y_{0}}]+E_{\tau}[\hat{y_{0}}^{2}]$ $= y_{...
0
votes
1answer
38 views

How do I implement stochastic gradient descent correctly?

I'm trying to implement stochastic gradient descent in MATLAB however I am not seeing any convergence. Mini-batch gradient descent worked as expected so I think that the cost function and gradient ...
0
votes
0answers
5 views

How to compute measures of dispersion and statistical significance of the impacts of a panel Spatial Durbin model (SDM)?

I ran a panel Spatial Durbin model (SDM) and computed the summary measures of impacts (direct, indirect and total). Now, I would like to get measures of dispersion for the impacts estimates as well as ...
-1
votes
1answer
12 views

Finding Average Treatment Effects through R

I am trying to determine the average treatment effect of a data set. The data set already has the outcome for each unit both under treatment and under control. How can we find the effect through ...
0
votes
0answers
14 views

Deriving covariance from supply and demand equations

I'm working on a problem from the textbook and I'm given two equations. $$ ln Q_i = \beta_0 + \beta_1 lnP_i +u_i $$ $$ ln P_i = \gamma_0 + \gamma_1lnQ_i + v_i $$ Show $$ cov(ln P_i, u_i) = \frac{\...
0
votes
1answer
13 views

Analysis crossover clinical trial

I have data from a six-sequence 3-drug 3-phase crossover trial that had a 7-day baseline period pre-study (used to generate mean baseline value). My outcome is a non-parametric continuous variable ...
0
votes
0answers
15 views

OLS estimator - caculating mean and variance [duplicate]

Yi = a + Xi + ui (i=1,2,...,n) In this regression model(there is no b(Coefficient of Xi)), I calculated the estimator of a. a hat = ΣYi/ΣXi And then I tried to calculate E(a hat) and Var(a hat) ...
0
votes
0answers
8 views

Variable importance when performing zero-inflated Poisson regression in R?

In short, I need to get the importance of the variables after a zero-inflated regression, with all my predictors being dichotomous factors. I tried something like this: ...
1
vote
1answer
17 views

Difference between using a propensity score for matching vs. regression analysis

So I am confused on what the difference is if I match patients based on propensity scores vs. using the propensity score and then applying that into a multivariate regression analysis? Is there a ...
0
votes
0answers
18 views

Thin Plate Regression Splines mgcv

I am struggling on the understanding of thin plate regression splines. I already found a very helpful answer here in cross-validated: smoothing methods for gam in mgcv package but I still have some ...
0
votes
0answers
27 views

How should I deal with continuous independent variables in a regression for ordinal dependent variables?

I am doing a research for which I will perform a data-analysis in SPSS. My dependent variable is 'father involvement'. I have four different questions that have measured different forms of 'father ...
0
votes
0answers
17 views

Estimating Policy Effect with a Logit

So I am testing a policy which was introduced in a country trying to incentivise people to stay employed at older ages (beyond the retirement age of 65). As such, they introduced a bonus where people ...
0
votes
0answers
8 views

Yes/no GZLMM zero inflated

I have got a variable which I want to analyse using a GZLMM with binomial distribution as the variable is coded as yes/no(1/0). However, there are a lot of zeros and not many 1's.I was hoping to ...
0
votes
0answers
8 views

Negative Intercept & Negative Independent Variable (Logistic Regression)

My intercept is negative and my independent variable is negative, does this mean there's a positive relationship? When my independent variable increases, it increases the log odds of my dependent ...
1
vote
0answers
20 views

Failure to replicate calculation of PCA residuals in linear regression with heteroscedasticity

In their preprint, Rocha et al. suggest a new type of residual for linear regression models with heteroscedasticity. They call their new residual PCA residuals. I have tried to replicate some of their ...
0
votes
0answers
17 views

I cannot fit the coefficient, standard error, and p-level on a single page. What do I do? [on hold]

I have a big regression. 3 panels, 5 columns, and about 10 variables. STATA outputs the table with each cell like this: 150.234*** (23.4) which takes up so much space the regression fails to fit on ...
0
votes
0answers
20 views

finding regression coefficients and deviation with autocorrellated outlaws

I try to make regression analyses to vector of average month C02 concentration in the air. ...
1
vote
0answers
18 views

G efficiency in Optimal designs

I am studying Optimal Designs and found a very interesting article by Peter Goos. In the article he provides an example in the form of an Excel document of generation of Optimal Designs for various ...
0
votes
0answers
5 views

Is repeated propensity score matching over many 0-1-features a valid procedure?

I would like to do a simple linear model where the outcome $y$ is real-valued, but my data matrix $X$ consists of several hundred features that all are $0$-$1$-valued. The number of observations $n$ ...
0
votes
0answers
16 views

Can i add my own data to this model (multiple regression?)

Possibly (probably?) a stupid question, so forgive my ignorance! This is a results table reported in a paper i read where they are using several paramaters to predict dry matter intake in cattle. If ...
2
votes
2answers
22 views

Main effects flip sign when adding interaction term in ordinal logistic regression

I am running an ordinal logistic regression. DV: policy score (0-3). IV: all continuous scale (GDP, corruption perception, total number of mines) All IV have a positive correlation with DV. When ...
1
vote
0answers
23 views

Linear regression diagnostics

I spent years reading articles, text, etc about the use of residuals to determine model violation, but I have a hard time telling if they actually have occurred and how much the violation matters. I ...
1
vote
1answer
27 views

Linear Regression confidence interval bounds

When performing a linear regression we first get a slope and intercept that is the best fit. How do we compute the confidence interval for predicted values? Here's an example: ...
0
votes
0answers
4 views
0
votes
0answers
13 views

Why does my Gaussian Process Regression on multivariate time-series data work well on training, but predicts only the mean for test?

My data is an hourly multivariate time-series consisting a temperature ($y_t$) and 7 other weather features $\mathbf{x_t}$ (e.g pressure, humidity...). $\mathbf{W} = \{(\mathbf{x}_1,y_1),(\mathbf{x}...
2
votes
0answers
25 views

Linear autoencoder model structure

My book is outlining a question that I think wants to demonstrate the relation between PCA and linear autoencoders. I'll outline how it begins below: Assume we have the training vectors $\ (x_i)_{i=1}...
0
votes
0answers
11 views

Is there a test for omitted variable bias?

I study finance and economics and every time i study an econometric study with OLS regression i wonder how the author can be sure of the non existance of omitted variable bias. I guess that in almost ...
2
votes
1answer
57 views

My data has overdispersion but the Hurdle model estimated theta is 0. What am I doing wrong?

I am confused by the dispersion parameter from my model. My data fails the overdispersion test. It's mean is 28.7, the variance is 18655.27. N=2916 of which 32% are zeros. How can theta equal 0 in ...
2
votes
1answer
17 views

understanding derivatives of a regression spline

I am trying to understand why regression splines are continuous at their knots Suppose I am fitting a regression spline $$ E[Y|X] = \alpha + \beta_1 x + \beta_2 (x - t)^+ $$ where $(x - t)^+ = \...
2
votes
1answer
25 views

Sample selection bias and logistic regression

I am struggling with possible sample selection bias at the moment, and I was wondering whether someone has a methodological tip or possibly knows of fancy statistical/econometric tools I could use to ...
1
vote
0answers
28 views

In linear regression, why are raw least squares residuals heteroskedastic?

In my course notes on a regression course with regards to the detection of heteroskedasticity there's the following quote: "Because the least-squares residuals have unequal variances even in the ...
0
votes
0answers
24 views

Is it possible for a logistic regression to have negative r squared on its training dataset?

Let's say I train a logistic model on xs and ys, and then use that model to back-predict ys from the original xs and compute an $r^2$ value as $1-rss/tss$. Is it possible for that value to be ...
3
votes
1answer
49 views

What is the relationship between the sum of squares of all weights and lambda in the ridge regression [duplicate]

Currently I am reading chapter 8, regression. And I feel quite confused about the following paragraph(see picture below), does it mean in ridge algorithm, the sum of all weights will be less than ...
0
votes
0answers
27 views

Using longitudinal income data to predict cross-sectional outcome measure

I have the following data: income data measured yearly from 2004 to 2011 in households occupied by adolescents a single variable denoting households where parents have divorced (divorced vs non-...
3
votes
2answers
50 views

How is Logistic Regression related to Logistic Distribution?

We all know that logistic regression is used to calculate probabilities through the logistic function. For a dependent categorical random variable $y$ and a set of $n$ predictors $\textbf{X} = [X_1 \...
0
votes
1answer
22 views

How to calculate residual sum of squares?

$Y_i = a + bX_i + u_i$ I found the estimates of a and b from this simple regression model by using some given facts below. $\sum X_i=40, \sum y_i=60, \sum X_i^2=200, \sum y_i^2=2460,\sum X_iy_i=240, ...
0
votes
1answer
22 views

Approximation of non linear function with multiple linear functions

How can a non-linear function be approximated by an appropriate amount of linear functions? In the picture below, it would be quite easy to draw 10-15 linear functions to describe all data points ...
-1
votes
1answer
21 views

Hierarchical clustering in R

I have a dataset of around 25 observations and most of them being categorical. I have three questions for this. 1- Do the covariates I pick for hierarchical clustering matter or should I try and ...