Questions tagged [leverage]

Leverage is a measure used in regression to highlight observations which are outlying in the space of the predictors.

Filter by
Sorted by
Tagged with
0
votes
0answers
44 views

Cooks Distance for multiple observations

I would like to determine the combined influence of a group of observations on a linear regression model, but I am not entirely sure how to compute the Cook's Distance for that. I know that for a ...
1
vote
1answer
48 views

Simple Linear Regression: Hat-Value $h_i$

I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the ...
3
votes
1answer
74 views

Individual significance of data points in correlation

In this question on stackoverflow, I asked about how it is possible to find the individual significance of each correlation coefficient of each node. I answered the question myself later stating that ...
2
votes
1answer
355 views

Proof about the diagonal element of the hat matrix

I'd really appreciate it if you could help me find the proof for the following formula: $$h_{ii}=1/n + \frac{(x_{i}-\bar{x})^2}{\sum(x_{j}-\bar{x})^2},$$ where $j=1,\ldots,n$. I don't really know ...
2
votes
0answers
78 views

Outliers and influential observations in elastic net logistic regression

My dataset has many biomarkers and the boxplots of these variables show the presence of many outliers. However, these 'outliers' are real data and not misread observations. I want to use elastic net ...
5
votes
1answer
72 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
0
votes
0answers
22 views

Inferences about leverage score and Mahalanobis distance [duplicate]

I have some inference given below: Given the design matrix $\textbf{X}$, the leverage score is defined as $\textbf{H}_{ii}$ where $\textbf{H} = \textbf{X} (\textbf{X}^T \textbf{X})^{-1} \textbf{X}^T$ ...
0
votes
1answer
273 views

Leverage and Influence

Is it possible that an outlier is neither influential nor does it have high leverage? Or can it happen that an observation with high leverage is not an outlier and is neither influential?
0
votes
0answers
24 views

How to find influance on correlation of single value

I need to find possible errors or outliers in my data sets. I want to find most affecting values of my pearson correlation. For example let say we got two data sets for X and Y: ...
0
votes
0answers
80 views

Why would standard errors around each individual point of an ARIMA model converge to a single value?

I am using an AR(1) model to determine if a new data point is beyond some expected range. Typically, you could use a prediction interval, which would just be the predicted value, +/- some multiple (e....
0
votes
0answers
45 views

How outliers influence your results? and what are good and bad leverage points? [duplicate]

I am confused between outliers and leverage points. And the difference between good and bad leverage points in time series analysis. Can somebody help me?
1
vote
1answer
72 views

Numerically Distinguish Between Real Correlation and Artifact

I'm looking at correlation for a large number of vectors, and many (about 3000) of these pairwise comparisons appear to have a significant correlation even after Bonferroni correction. Plotting these ...
1
vote
1answer
148 views

Why divide by 1-leverage?

I'm reading about resampling methods, and specifically leave-one-out cross-validation. I understood the method, and how to calculate the estimate of the test MSE (Mean squared error): In the setup ...
1
vote
1answer
258 views

Why uncentered hat matrix can be used to measure the distance from the center of data?

This question is motivated from here :Why leverage measure the distance of the ith observation from the center of the x space? which is the question related to the wonderful answer of this link :...
6
votes
2answers
670 views

Identifying outliers in the data

Sample data ...
2
votes
1answer
1k views

Influential observations and Outliers in Linear Regression model

What is the difference between influential observations and outliers in linear regression model?
2
votes
1answer
64 views

Is there a specific standard error for an individual residual?

On page 97 of Introduction to Statistical Learning book, there is a paragraph on studentized residuals within the context of looking for outliers. But in practice, it can be difficult to decide how ...
0
votes
0answers
44 views

News Impact curve shift

I computed the NIC for both vanilla GARCH and EGARCH model specifications (with t(4)-student distributed innovations). As we can see from the plot, no asymmetries are present, but the curve has ...
0
votes
0answers
73 views

How to solve a well fitted model - Model Misspecification

I am currently writing a paper, analysing the impact of goldprice movements on the capital structure of gold mining firms. My basic model is a simple OLS model with (y=leverage and x=ln(goldprice)). ...
1
vote
0answers
102 views

Sensitivity Measures for GEE Model

Is there any method (e.g. like Cooks D) implemented in R to identify leverage points for GEE Models? I used geepack to fit my models and would like to do a sensitivity analysis now. However, I don't ...
2
votes
1answer
455 views

Leverage formula/derivation and Hat matrix

1) So I know that $h_{ii}$ is just the ith row ith column of $H=X(X^TX)^{-1}X^T$. Intuitively, why is this the case? I understand that H is the projection matrix and leverage is measuring how far away ...
3
votes
2answers
9k views

Cook's distance vs. hat values

What exactly does Cook's distance measure? And how is this different from what hat values measure? I know hat values measure how distant a point it form its corresponding fitted point. I also know ...
3
votes
1answer
786 views

Leverage Statistic - equation explanation

I came across this equation for the leverage statistic in ISL. Does the denominator translate to the summation of all the squares of $(x_i - mean(x))^2$ excluding the current observation of x? In ...
6
votes
1answer
349 views

What are the leverage values for Ridge regression?

In linear least squares the parameter estimates are: $\hat{\beta} = \left(X^{\top}X\right)^{-1}X^{\top}y$. In Ridge regression the standardized parameter estimates are given by $\hat{\beta}_{\Gamma} = ...
0
votes
0answers
72 views

which residual vs fitted plot is looking better here?

This is a data analysis project we have at school where we try different ways to correct a simple linear regression model. My first attempt is just trying to transform the response variable by taking ...
1
vote
0answers
50 views

How to Adapt Leverage vs Residual plot for Multi-variate Regression

After reading about What is the best way to identify outliers in multivariate data? I noticed that some heuristics are hard to adapt to mult-variate analysis. I'm hoping there is a way to at least ...
1
vote
0answers
23 views

How to check if overall results from a survey are significantly affected by participation from a large company(ies)?

Suppose there is some sort of survey where people from various companies answer some questions. Is it possible to determine whether overall results from the survey are significantly affected by ...
2
votes
1answer
560 views

Is it OK if I have some high leverage points in my experimental plots and they weren't outliers?

I've analyzed effects of temperature and sample thickness on drying process via response surface methodology. I've faced to four high leverage points (my four axial runs) that stick together on normal ...
1
vote
1answer
30 views

Using low coverage predictors in models?

The conventional wisdom I run up against is to drop predictors with low coverage without given them much consideration. By low coverage predictors: I meant predictors whose values are mostly missing. ...
0
votes
1answer
49 views

Deleting Influential Points Leading to Non-constant Variance

http://archive.ics.uci.edu/ml/datasets/Wine+Quality Using this data set, I am regressing chlorides on ten predictors (all except quality). After Box-Cox transforming chlorides, I get the following Q-Q ...
0
votes
0answers
63 views

Linear model and mean value of covariates

Suppose a linear model $$Y_i=\beta_0+x_{i1}\beta_1+\ldots+x_{ip}\beta_p+e_i,\quad e_i\sim N(0,\sigma^2),\quad i=1,\ldots,n,$$ and its hat matrix $P=X(X^TX)^{-1}X^T$, where $Y=X\beta+e$. Let $\bar{x}...
1
vote
0answers
136 views

Influence function for glm?

I would like to calculate the influence of each sample on the coefficient under a logistic regression model. The R built-in function influence() is suppose to do ...
0
votes
1answer
560 views

How to extract/compute leverage and Cook's distances for Generalised Additive models

Similar to the following question, How to extract/compute leverage and Cook's distances for linear mixed effects models Is there any method in R to determine influential points in a GAM?
6
votes
1answer
2k views

What is .hat in regression output

The augment() function in the broom package for R creates a dataframe of predicted values from a regression model. Columns created include the fitted values, the ...
3
votes
0answers
278 views

leverage.plot() R - CAR package

I'm trying to complete a homework regarding added-variables and leverage plots using the CAR package. In the documentation of the ...
1
vote
1answer
297 views

Linear regression question on Idempotent matrix and leverage points

I am considering a linear regression model $Y_i = X_i^T \beta + \epsilon_i, i = 1,2,\dots,n$. where $X_i \in \mathbb{R}^p$. $\epsilon_i$'s are independent copies of random error $\epsilon \in \mathbb{...
3
votes
1answer
195 views

Leverages and effect of leverage points

I just got some question about the hat matrix in linear models. My first question is: Why in a balanced one-way layout $(n_1=...=n_c=n_0)$, all leverages $h_{ii}$ have the same value $\frac{1}{n_0}$? ...
2
votes
0answers
63 views

Why leverage measure the distance of the ith observation from the center of the x space? [duplicate]

I know the definition of leverage points in regression, that is $h_{ii}=x_{i}'(X'X)^{-1}x_{i}. $ In many places and text books, they always say that leverage is a standardized measure of the distance ...
2
votes
1answer
118 views

How does centering affect leverage?

For a linear regression model $Y = β_0 + β_1X$, consider the matrix $X$ and $X_c$ with centering the mean. How do you use algebra to show directly that centering does not change the leverage? I have ...
6
votes
1answer
11k views

Hat matrix and leverages in classical multiple regression

What is Hat matrix and leverages in classical multiple regression? What are their roles? And Why do use them? Please explain them or give satisfactory book/ article references to understand them.
1
vote
1answer
136 views

variance decreases when x gets farther from the average x?

I just read the description of Studentized residual on Wikipeida. I'm confused about what it says about variance, it says that "the residuals, unlike the errors, do not all have the same variance ...
1
vote
0answers
682 views

Use linear regression to detect outliers and leverage points

I want to use linear regression to pre-process the data (e.g find outliers) so that I can use techniques like ANOVA to analyze the data. The goal is not to fit a regression model. I saw two posts ...
1
vote
0answers
40 views

A little help in confirming an interpretation on an output

How can we interpret the results in the graph below? Does the R command lm() uses Heteroskedasticity-consistent covariance matrices(HCCM) estimators, or HAC estimators for the inference tests? The ...
3
votes
0answers
97 views

Multiple Regression Assumptions

This may seem like a basic question, but I'm verifying the assumptions for a multiple regression and have some trouble wrapping my head around homoscedasticity. I have a few questions listed below: 1)...
12
votes
1answer
2k views

Prove the relation between Mahalanobis distance and Leverage?

I have seen formulas on Wikipedia. that relate Mahalanobis distance and Leverage: Mahalanobis distance is closely related to the leverage statistic, $h$, but has a different scale: $$D^2 = (N - 1)(...
7
votes
1answer
4k views

How to identify outliers and do model diagnostics for an lme4 model?

I need to identify outliers and high leverage points, and perform model diagnostics, in an lme4 model. For outliers and high leverage points, simply making a plot ...
2
votes
1answer
162 views

Multiple regression - High adjusted R^2 but the residuals are high

I used the multiple regression models to derive the outcomes after using AIC pairwise comparison and deleted the outliers, high leverage points. And it seems good, the adjusted R^2 acheived 0.9543, ...
3
votes
2answers
5k views

R not plotting observations with leverage one

I am building a really simple linear model. I want to test if the frass I got over 3 days from butterfly larvae depend upon the food they ate (diet), the butterfly family (the mother line) and ...
3
votes
0answers
312 views

Is it possible to derive Leverage figures without a Hat Matrix?

I ran into an impasse while attempting to write code for Cook's Distance: when a regression model reaches only a moderate size, I can't derive a Hat Matrix through my normal matrix math routines ...
0
votes
1answer
50 views

Whats it called when you fit a linear regression to data with outliers at the end point that influence your regression

So if you have outliers in the middle of your sample it doesn't influence your regression much but if they are at either end of your sample they do.