# Questions tagged [leverage]

Leverage is a measure used in regression to highlight observations which are outlying in the space of the predictors.

89 questions
Filter by
Sorted by
Tagged with
48 views

### Understanding the ridge leverage scores sampling from an arXiv paper

I give a try to read the arXiv paper Distributed Adaptive Sampling for Kernel Matrix Approximation, Calandriello et al. 2017. I got a code implementation where they compute ridge leverage scores ...
20 views

### a question about leverage and variance of the predictor variable [duplicate]

In a simple linear regression: Assume that the leverage of a specific data point equals 1 (maximum leverage), from my understanding, it also means that the variance of the estimated error (e) of that ...
35 views

### How do I go about rectifying violated assumptions when more than one is violated at the same time? [duplicate]

I am currently trying to run a model analyzing the duration of the egg stage of each sex of two species of insect across five different temperatures. All independent variables are categorical. My ...
351 views

110 views

### Removing high leverage point and then transforming response variable

I have a dataset and I fitted a linear model to the data. The model has satisfactory F-statistics and all the predictors have good p-values. Now I plot the residuals against the fitted values and ...
118 views

### Do leverage values make sense outside of linear models?

I've been looking at leverage plots, but it seems to me that they are always related to linear regression models. For instance, this explanation of a hat matrix considers a linear regression model: ...
1 vote
66 views

### Help connecting MSE and residual variance $(=σ^2/(1-h_{ii}$)) in linear regression

In linear regression, we assume errors are normally distributed with mean 0 and variance $σ^2$. We use MSE = (Residual sum of squares) / (n-p) as the estimator for σ, where n= number of observations ...
208 views

### Who originally defined leverage scores to be the diagonal elements of $X(X^TX)^{-1}X^T$?

A nice description of leverage in the sense that I am using it is given here so I will not repeat it. Who originally defined leverage scores to be the diagonal elements of $X(X^TX)^{-1}X^T$?
1 vote
645 views

### Identification High Leverage for Logistic Regression

As seen below from Piet De Jong, Generalized Linear Models for Insurance Data, for linear model, we can identify high leverage, if the value of leverage>2p/n (or hii > 2p/n) then the indicates ...
1 vote
73 views

Wikipedia's page for leverage defines the design matrix $X \in \mathbb{R}^{n \times p}$ where $n$ is the number of data and $p$ is the dimensionality of each datum, and then, under "Relation to ...
1 vote
279 views

### Developing Leverage Statistics Manually in R

I've asked this question on Stack Overflow but think it might be better here. I currently have the below data frame df with the regression equation ...
523 views

### Cook's distance, not plotting an observation with leverage one. What is this?

I am doing some regression diagnostics in R. I use plot() function and look at the four graphs. However, when I reach Cook's distance graph, I receive a warning ...
195 views

### Selecting a subset of observations that retains the most information

Is there a "best" way to select a subset of data (observations, not variables) to use in a multiple regression? The problem: A colleague of mine is planning a project for this field season. ...
556 views

### Logistic regression with rare events, all events have large residuals and are influential points

I am running a logistic regression model where the outcome is relatively rare (250 out of 5000). My main interest is to see if there are differences between age groups, sex, educational levels, income ...
67 views

### Why (or when) can we neglect residuals mutual correlation?

I've found in several books so far only a waving of hands explanation on why one can simply neglect the residuals mutual correlation, in a linear model context, and plot a QQ-graph to do a qualitative ...
1k views

### Understanding Residuals vs Leverage plot in terms of meeting regression assumptions

Can someone help me understand the Residuals vs Leverage plot in terms of meeting the assumption of independence/influence for multiple linear regression models? My understanding is that the ...
181 views

### PRESS from the hat matrix and numerical stability from statsmodels ols.fit()

Leave one out cross validation in the context of ordinary least squares regression can be done via the hat matrix: The "hat" or projection matrix $$H = X(X^T X)^{-1} X^T$$ many fit ...
80 views

### Why does high leverage point in SLR have unusual covariate values

As stated above and also wondering what a covariate value means, is it just covariance? IF not, then what it is and what is the difference between covariance and covariate
1 vote
319 views

### How to decide between different robust standard errors?

Specifying my model I ran into some very mild heteroscedasticity problems. Given its superior small-sample properties (my dataset contains 79 observations) I used the HC3 specification of the White ...
1 vote
138 views

### Interpreting results from glm.diag.plots (package 'boot')

I'm having hard time interpreting results from glm.diag.plots (package 'boot'). I read through some questions and answers (this one was very helpful, thank you! Interpreting glm.diag.plots) but didn't ...
480 views

### leverage() diagnostic test not supported for glmmTMB models in r

I am using a glmmTMB to look at the effect of numerous variables on how far individuals travel (distance). The example below is ...
27 views

### N size for not being considerated leverage

Suposse I have a dataset with two variables x and y, with the purpose to run a linear regression y ~ x. We have all x values equal and y varying between 1 and 10. For example (in R code): ...
389 views

### How to check for outliers and leverage values in SPSS when conducting a multinomial logistic regression?

I am conducting a multinomial logistic regression in SPSS. I want to check for the presence of outliers and high leverage values. How do I do that in SPSS?
35 views

### Relation between dyi/dei and its leverage (hii)

Althought I have tried in different ways, I have not been able to show that $$\frac{\mathrm{d}Y_i}{\mathrm{d}e_i} = \frac{1}{1 - l_{ii}}$$ $e_i$ is equal to $l_{ii}$ is the $i-th$ element of the ...
585 views

### Simple Linear Regression: Hat-Value $h_i$

I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the ...
488 views

### Individual significance of data points in correlation

In this question on stackoverflow, I asked about how it is possible to find the individual significance of each correlation coefficient of each node. I answered the question myself later stating that ...
4k views

### Proof about the diagonal element of the hat matrix

I'd really appreciate it if you could help me find the proof for the following formula: $$h_{ii}=1/n + \frac{(x_{i}-\bar{x})^2}{\sum(x_{j}-\bar{x})^2},$$ where $j=1,\ldots,n$. I don't really know ...
274 views

### Outliers and influential observations in elastic net logistic regression

My dataset has many biomarkers and the boxplots of these variables show the presence of many outliers. However, these 'outliers' are real data and not misread observations. I want to use elastic net ...
153 views

### Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
1 vote
1k views

### Leverage and Influence

Is it possible that an outlier is neither influential nor does it have high leverage? Or can it happen that an observation with high leverage is not an outlier and is neither influential?
57 views

### How outliers influence your results? and what are good and bad leverage points? [duplicate]

I am confused between outliers and leverage points. And the difference between good and bad leverage points in time series analysis. Can somebody help me?
1 vote
183 views

### Numerically Distinguish Between Real Correlation and Artifact

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these ...
907 views

### Why divide by 1-leverage?

I'm reading about resampling methods, and specifically leave-one-out cross-validation. I understood the method, and how to calculate the estimate of the test MSE (Mean squared error): In the setup ...
34 views

### Name for spurious linear Regression Plots [duplicate]

Yesterday I was at a medical conference in which a lot of plots of Point Clouds with linear fits were shown. In many cases the fit seemed (at least to me and colleagues) to be influenced mostly by ...
1 vote
1k views

### Why uncentered hat matrix can be used to measure the distance from the center of data?

This question is motivated from here :Why leverage measure the distance of the ith observation from the center of the x space? which is the question related to the wonderful answer of this link :...
1k views

Sample data ...
3k views

### Influential observations and Outliers in Linear Regression model

What is the difference between influential observations and outliers in linear regression model?
324 views

### Is there a specific standard error for an individual residual?

On page 97 of Introduction to Statistical Learning book, there is a paragraph on studentized residuals within the context of looking for outliers. But in practice, it can be difficult to decide how ...
264 views

### News Impact curve shift

I computed the NIC for both vanilla GARCH and EGARCH model specifications (with t(4)-student distributed innovations). As we can see from the plot, no asymmetries are present, but the curve has ...
133 views

### How to solve a well fitted model - Model Misspecification

I am currently writing a paper, analysing the impact of goldprice movements on the capital structure of gold mining firms. My basic model is a simple OLS model with (y=leverage and x=ln(goldprice)). ...
1 vote
251 views

### Sensitivity Measures for GEE Model

Is there any method (e.g. like Cooks D) implemented in R to identify leverage points for GEE Models? I used geepack to fit my models and would like to do a sensitivity analysis now. However, I don't ...
1k views

### Leverage formula/derivation and Hat matrix

1) So I know that $h_{ii}$ is just the ith row ith column of $H=X(X^TX)^{-1}X^T$. Intuitively, why is this the case? I understand that H is the projection matrix and leverage is measuring how far away ...
18k views

### Cook's distance vs. hat values

What exactly does Cook's distance measure? And how is this different from what hat values measure? I know hat values measure how distant a point it form its corresponding fitted point. I also know ...
1k views

### Leverage Statistic - equation explanation

I came across this equation for the leverage statistic in ISL. Does the denominator translate to the summation of all the squares of $(x_i - mean(x))^2$ excluding the current observation of x? In ...
977 views

### What are the leverage values for Ridge regression?

In linear least squares the parameter estimates are: $\hat{\beta} = \left(X^{\top}X\right)^{-1}X^{\top}y$. In Ridge regression the standardized parameter estimates are given by \$\hat{\beta}_{\Gamma} = ...
147 views

### which residual vs fitted plot is looking better here?

This is a data analysis project we have at school where we try different ways to correct a simple linear regression model. My first attempt is just trying to transform the response variable by taking ...
1 vote