# Questions tagged [leverage]

Leverage is a measure used in regression to highlight observations which are outlying in the space of the predictors.

74 questions
Filter by
Sorted by
Tagged with
20 views

### Understanding leverage and influence

there is a reference to the $i^{th}$ diagonal entry in $H$ where $H=X(X^TX)^{-1}X^T$ in the definitions of leverage and cook's distance. See: https://en.wikipedia.org/wiki/Leverage_(statistics) and ...
34 views

### PRESS from the hat matrix and numerical stability from statsmodels ols.fit()

Leave one out cross validation in the context of ordinary least squares regression can be done via the hat matrix: The "hat" or projection matrix $$H = X(X^T X)^{-1} X^T$$ many fit ...
7 views

### Why does high leverage point in SLR have unusual covariate values

As stated above and also wondering what a covariate value means, is it just covariance? IF not, then what it is and what is the difference between covariance and covariate
27 views

### Leverage Statistic (h) for Multiple Predictors [duplicate]

I'm working through Introduction to Statistical Learning and understand the leverage statistic for a simple linear regression. However the text says "There is a simple extension of h_i to the ...
57 views

### How to decide between different robust standard errors?

Specifying my model I ran into some very mild heteroscedasticity problems. Given its superior small-sample properties (my dataset contains 79 observations) I used the HC3 specification of the White ...
20 views

### Interpreting results from glm.diag.plots (package 'boot')

I'm having hard time interpreting results from glm.diag.plots (package 'boot'). I read through some questions and answers (this one was very helpful, thank you! Interpreting glm.diag.plots) but didn't ...
21 views

### Condition number and leverage

Condition number is defined using the eigenvalues of $X^\top X$ while leverage values are the diagonal elements of the projection matrix. How are the two related? How can rescaling help, because aren'...
26 views

### Leverage after Tobit Regression

In STATA, after running a TOBIT regression, I'm trying to calculate the leverage and Cook's Distance values. When I run: predict c, cooksd I get the error: 'option ...
136 views

### leverage() diagnostic test not supported for glmmTMB models in r

I am using a glmmTMB to look at the effect of numerous variables on how far individuals travel (distance). The example below is ...
25 views

### N size for not being considerated leverage

Suposse I have a dataset with two variables x and y, with the purpose to run a linear regression y ~ x. We have all x values equal and y varying between 1 and 10. For example (in R code): ...
22 views

### Relation between dyi/dei and its leverage (hii)

Althought I have tried in different ways, I have not been able to show that $$\frac{\mathrm{d}Y_i}{\mathrm{d}e_i} = \frac{1}{1 - l_{ii}}$$ $e_i$ is equal to $l_{ii}$ is the $i-th$ element of the ...
8 views

168 views

### Simple Linear Regression: Hat-Value $h_i$

I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the ...
147 views

### Individual significance of data points in correlation

In this question on stackoverflow, I asked about how it is possible to find the individual significance of each correlation coefficient of each node. I answered the question myself later stating that ...
2k views

### Proof about the diagonal element of the hat matrix

I'd really appreciate it if you could help me find the proof for the following formula: $$h_{ii}=1/n + \frac{(x_{i}-\bar{x})^2}{\sum(x_{j}-\bar{x})^2},$$ where $j=1,\ldots,n$. I don't really know ...
139 views

### Outliers and influential observations in elastic net logistic regression

My dataset has many biomarkers and the boxplots of these variables show the presence of many outliers. However, these 'outliers' are real data and not misread observations. I want to use elastic net ...
84 views

### Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
533 views

### Leverage and Influence

Is it possible that an outlier is neither influential nor does it have high leverage? Or can it happen that an observation with high leverage is not an outlier and is neither influential?
49 views

### How outliers influence your results? and what are good and bad leverage points? [duplicate]

I am confused between outliers and leverage points. And the difference between good and bad leverage points in time series analysis. Can somebody help me?
81 views

### Numerically Distinguish Between Real Correlation and Artifact

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these ...
346 views

### Why divide by 1-leverage?

I'm reading about resampling methods, and specifically leave-one-out cross-validation. I understood the method, and how to calculate the estimate of the test MSE (Mean squared error): In the setup ...
30 views

### Name for spurious linear Regression Plots [duplicate]

Yesterday I was at a medical conference in which a lot of plots of Point Clouds with linear fits were shown. In many cases the fit seemed (at least to me and colleagues) to be influenced mostly by ...
454 views

### Why uncentered hat matrix can be used to measure the distance from the center of data?

This question is motivated from here :Why leverage measure the distance of the ith observation from the center of the x space? which is the question related to the wonderful answer of this link :...
854 views

Sample data ...
2k views

### Influential observations and Outliers in Linear Regression model

What is the difference between influential observations and outliers in linear regression model?
106 views

### Is there a specific standard error for an individual residual?

On page 97 of Introduction to Statistical Learning book, there is a paragraph on studentized residuals within the context of looking for outliers. But in practice, it can be difficult to decide how ...
89 views

### News Impact curve shift

I computed the NIC for both vanilla GARCH and EGARCH model specifications (with t(4)-student distributed innovations). As we can see from the plot, no asymmetries are present, but the curve has ...
94 views

### How to solve a well fitted model - Model Misspecification

I am currently writing a paper, analysing the impact of goldprice movements on the capital structure of gold mining firms. My basic model is a simple OLS model with (y=leverage and x=ln(goldprice)). ...
137 views

### Sensitivity Measures for GEE Model

Is there any method (e.g. like Cooks D) implemented in R to identify leverage points for GEE Models? I used geepack to fit my models and would like to do a sensitivity analysis now. However, I don't ...
814 views

### Leverage formula/derivation and Hat matrix

1) So I know that $h_{ii}$ is just the ith row ith column of $H=X(X^TX)^{-1}X^T$. Intuitively, why is this the case? I understand that H is the projection matrix and leverage is measuring how far away ...
12k views

### Cook's distance vs. hat values

What exactly does Cook's distance measure? And how is this different from what hat values measure? I know hat values measure how distant a point it form its corresponding fitted point. I also know ...
1k views

### Leverage Statistic - equation explanation

I came across this equation for the leverage statistic in ISL. Does the denominator translate to the summation of all the squares of $(x_i - mean(x))^2$ excluding the current observation of x? In ...
543 views

198 views

### Influence function for glm?

I would like to calculate the influence of each sample on the coefficient under a logistic regression model. The R built-in function influence() is suppose to do ...
649 views

### How to extract/compute leverage and Cook's distances for Generalised Additive models

Similar to the following question, How to extract/compute leverage and Cook's distances for linear mixed effects models Is there any method in R to determine influential points in a GAM?
3k views

### What is .hat in regression output

The augment() function in the broom package for R creates a dataframe of predicted values from a regression model. Columns created include the fitted values, the ...
320 views

### leverage.plot() R - CAR package

I'm trying to complete a homework regarding added-variables and leverage plots using the CAR package. In the documentation of the ...
406 views