Questions tagged [leverage]

Leverage is a measure used in regression to highlight observations which are outlying in the space of the predictors.

Filter by
Sorted by
Tagged with
2
votes
1answer
46 views

Proof about the diagonal element of the hat matrix

I'd really appreciate it if you could help me find the proof for the following formula: $h_{ii}=1/n + \frac{(x_{i}-\bar{x})^2}{\sum(x_{j}-\bar{x})^2}$ , where $j=1,...,n$. I don't really know where ...
1
vote
0answers
24 views

Outliers and influential observations in elastic net logistic regression

My dataset has many biomarkers and the boxplots of these variables show the presence of many outliers. However, these 'outliers' are real data and not misread observations. I want to use elastic net ...
5
votes
1answer
67 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
0
votes
0answers
22 views

Inferences about leverage score and Mahalanobis distance [duplicate]

I have some inference given below: Given the design matrix $\textbf{X}$, the leverage score is defined as $\textbf{H}_{ii}$ where $\textbf{H} = \textbf{X} (\textbf{X}^T \textbf{X})^{-1} \textbf{X}^T$ ...
0
votes
1answer
55 views

Leverage and Influence

Is it possible that an outlier is neither influential nor does it have high leverage? Or can it happen that an observation with high leverage is not an outlier and is neither influential?
0
votes
0answers
14 views

How to find influance on correlation of single value

I need to find possible errors or outliers in my data sets. I want to find most affecting values of my pearson correlation. For example let say we got two data sets for X and Y: ...
0
votes
0answers
51 views

Why would standard errors around each individual point of an ARIMA model converge to a single value?

I am using an AR(1) model to determine if a new data point is beyond some expected range. Typically, you could use a prediction interval, which would just be the predicted value, +/- some multiple (e....
0
votes
0answers
43 views

How outliers influence your results? and what are good and bad leverage points? [duplicate]

I am confused between outliers and leverage points. And the difference between good and bad leverage points in time series analysis. Can somebody help me?
1
vote
1answer
66 views

Numerically Distinguish Between Real Correlation and Artifact

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these ...
1
vote
1answer
76 views

Why divide by 1-leverage?

I'm reading about resampling methods, and specifically leave-one-out cross-validation. I understood the method, and how to calculate the estimate of the test MSE (Mean squared error): In the setup ...
1
vote
1answer
163 views

Why uncentered hat matrix can be used to measure the distance from the center of data?

This question is motivated from here :Why leverage measure the distance of the ith observation from the center of the x space? which is the question related to the wonderful answer of this link :...
6
votes
2answers
582 views

Identifying outliers in the data

Sample data ...
2
votes
1answer
704 views

Influential observations and Outliers in Linear Regression model

What is the difference between influential observations and outliers in linear regression model?
2
votes
1answer
45 views

Is there a specific standard error for an individual residual?

On page 97 of Introduction to Statistical Learning book, there is a paragraph on studentized residuals within the context of looking for outliers. But in practice, it can be difficult to decide how ...
0
votes
0answers
31 views

News Impact curve shift

I computed the NIC for both vanilla GARCH and EGARCH model specifications (with t(4)-student distributed innovations). As we can see from the plot, no asymmetries are present, but the curve has ...
0
votes
0answers
62 views

How to solve a well fitted model - Model Misspecification

I am currently writing a paper, analysing the impact of goldprice movements on the capital structure of gold mining firms. My basic model is a simple OLS model with (y=leverage and x=ln(goldprice)). ...
1
vote
0answers
91 views

Sensitivity Measures for GEE Model

Is there any method (e.g. like Cooks D) implemented in R to identify leverage points for GEE Models? I used geepack to fit my models and would like to do a sensitivity analysis now. However, I don't ...
2
votes
1answer
305 views

Leverage formula/derivation and Hat matrix

1) So I know that $h_{ii}$ is just the ith row ith column of $H=X(X^TX)^{-1}X^T$. Intuitively, why is this the case? I understand that H is the projection matrix and leverage is measuring how far away ...
3
votes
2answers
6k views

Cook's distance vs. hat values

What exactly does Cook's distance measure? And how is this different from what hat values measure? I know hat values measure how distant a point it form its corresponding fitted point. I also know ...
3
votes
1answer
527 views

Leverage Statistic - equation explanation

I came across this equation for the leverage statistic in ISL. Does the denominator translate to the summation of all the squares of $(x_i - mean(x))^2$ excluding the current observation of x? In ...
5
votes
1answer
294 views

What are the leverage values for Ridge regression?

In linear least squares the parameter estimates are: $\hat{\beta} = \left(X^{\top}X\right)^{-1}X^{\top}y$. In Ridge regression the standardized parameter estimates are given by $\hat{\beta}_{\Gamma} = ...
0
votes
0answers
65 views

which residual vs fitted plot is looking better here?

This is a data analysis project we have at school where we try different ways to correct a simple linear regression model. My first attempt is just trying to transform the response variable by taking ...
1
vote
0answers
46 views

How to Adapt Leverage vs Residual plot for Multi-variate Regression

After reading about What is the best way to identify outliers in multivariate data? I noticed that some heuristics are hard to adapt to mult-variate analysis. I'm hoping there is a way to at least ...
1
vote
0answers
23 views

How to check if overall results from a survey are significantly affected by participation from a large company(ies)?

Suppose there is some sort of survey where people from various companies answer some questions. Is it possible to determine whether overall results from the survey are significantly affected by ...
2
votes
1answer
314 views

Is it OK if I have some high leverage points in my experimental plots and they weren't outliers?

I've analyzed effects of temperature and sample thickness on drying process via response surface methodology. I've faced to four high leverage points (my four axial runs) that stick together on normal ...
1
vote
1answer
30 views

Using low coverage predictors in models?

The conventional wisdom I run up against is to drop predictors with low coverage without given them much consideration. By low coverage predictors: I meant predictors whose values are mostly missing. ...
0
votes
0answers
57 views

Linear model and mean value of covariates

Suppose a linear model $$Y_i=\beta_0+x_{i1}\beta_1+\ldots+x_{ip}\beta_p+e_i,\quad e_i\sim N(0,\sigma^2),\quad i=1,\ldots,n,$$ and its hat matrix $P=X(X^TX)^{-1}X^T$, where $Y=X\beta+e$. Let $\bar{x}...
1
vote
0answers
109 views

Influence function for glm?

I would like to calculate the influence of each sample on the coefficient under a logistic regression model. The R built-in function influence() is suppose to do ...
0
votes
1answer
508 views

How to extract/compute leverage and Cook's distances for Generalised Additive models

Similar to the following question, How to extract/compute leverage and Cook's distances for linear mixed effects models Is there any method in R to determine influential points in a GAM?
6
votes
1answer
2k views

What is .hat in regression output

The augment() function in the broom package for R creates a dataframe of predicted values from a regression model. Columns created include the fitted values, the ...
3
votes
0answers
241 views

leverage.plot() R - CAR package

I'm trying to complete a homework regarding added-variables and leverage plots using the CAR package. In the documentation of the ...
1
vote
1answer
226 views

Linear regression question on Idempotent matrix and leverage points

I am considering a linear regression model $Y_i = X_i^T \beta + \epsilon_i, i = 1,2,\dots,n$. where $X_i \in \mathbb{R}^p$. $\epsilon_i$'s are independent copies of random error $\epsilon \in \mathbb{...
3
votes
1answer
133 views

Leverages and effect of leverage points

I just got some question about the hat matrix in linear models. My first question is: Why in a balanced one-way layout $(n_1=...=n_c=n_0)$, all leverages $h_{ii}$ have the same value $\frac{1}{n_0}$? ...
2
votes
0answers
58 views

Why leverage measure the distance of the ith observation from the center of the x space? [duplicate]

I know the definition of leverage points in regression, that is $h_{ii}=x_{i}'(X'X)^{-1}x_{i}. $ In many places and text books, they always say that leverage is a standardized measure of the distance ...
2
votes
1answer
93 views

How does centering affect leverage?

For a linear regression model $Y = β_0 + β_1X$, consider the matrix $X$ and $X_c$ with centering the mean. How do you use algebra to show directly that centering does not change the leverage? I have ...
5
votes
1answer
8k views

Hat matrix and leverages in classical multiple regression

What is Hat matrix and leverages in classical multiple regression? What are their roles? And Why do use them? Please explain them or give satisfactory book/ article references to understand them.
1
vote
1answer
125 views

variance decreases when x gets farther from the average x?

I just read the description of Studentized residual on Wikipeida. I'm confused about what it says about variance, it says that "the residuals, unlike the errors, do not all have the same variance ...
1
vote
0answers
570 views

Use linear regression to detect outliers and leverage points

I want to use linear regression to pre-process the data (e.g find outliers) so that I can use techniques like ANOVA to analyze the data. The goal is not to fit a regression model. I saw two posts ...
1
vote
0answers
39 views

A little help in confirming an interpretation on an output

How can we interpret the results in the graph below? Does the R command lm() uses Heteroskedasticity-consistent covariance matrices(HCCM) estimators, or HAC estimators for the inference tests? The ...
3
votes
0answers
95 views

Multiple Regression Assumptions

This may seem like a basic question, but I'm verifying the assumptions for a multiple regression and have some trouble wrapping my head around homoscedasticity. I have a few questions listed below: 1)...
12
votes
1answer
2k views

Prove the relation between Mahalanobis distance and Leverage?

I have seen formulas on Wikipedia. that relate Mahalanobis distance and Leverage: Mahalanobis distance is closely related to the leverage statistic, $h$, but has a different scale: $$D^2 = (N - 1)(...
7
votes
1answer
3k views

How to identify outliers and do model diagnostics for an lme4 model?

I need to identify outliers and high leverage points, and perform model diagnostics, in an lme4 model. For outliers and high leverage points, simply making a plot ...
2
votes
1answer
149 views

Multiple regression - High adjusted R^2 but the residuals are high

I used the multiple regression models to derive the outcomes after using AIC pairwise comparison and deleted the outliers, high leverage points. And it seems good, the adjusted R^2 acheived 0.9543, ...
2
votes
2answers
4k views

R not plotting observations with leverage one

I am building a really simple linear model. I want to test if the frass I got over 3 days from butterfly larvae depend upon the food they ate (diet), the butterfly family (the mother line) and ...
3
votes
0answers
291 views

Is it possible to derive Leverage figures without a Hat Matrix?

I ran into an impasse while attempting to write code for Cook's Distance: when a regression model reaches only a moderate size, I can't derive a Hat Matrix through my normal matrix math routines ...
0
votes
1answer
50 views

Whats it called when you fit a linear regression to data with outliers at the end point that influence your regression

So if you have outliers in the middle of your sample it doesn't influence your regression much but if they are at either end of your sample they do.
2
votes
0answers
151 views

How to distinguish suspicious leverages?

Given a linear model and the following hatvalues and influence.measures, how can I say which measurements are suspicious? I mean ...
1
vote
0answers
232 views

Calculating leverage/cook's distance for a Weighted Spatial Simultaneous Autoregression Model

I am estimating a Weighted Spatial Simultaneous Autoregression Model (spdep::spautolm --> Link) in R and I would like to do some residual analysis. Unfortunately ...
4
votes
0answers
220 views

what to do with ridiculous but valid leverage points

So I'm having some difficulty fitting a linear model to the data (see other post here glm model fit - can't find a family/link combination that produces good fit). In particular, I'm worried ...
6
votes
1answer
1k views

Interpreting case influence statistics (leverage, studentized residuals, and Cook's distance)

I just wanted to clarify some things about leverage, studentized residuals, and Cook's distance: Does a large (in absolute value) studentized residual mean that a case is an outlier? Does a large ...