# Accounting for biases in the data (normalizing? detrending?)

What's it called when I'm trying to remove the effect of a variable on another variable? Am I in the right ballpark using the terms "detrending" or "normalizing" or "controlling for Y on X"? What's the general process for doing this?

The problem I'm working with currently is as follows:

I work at a mine that loads trains that are 160 wagons in length. After the trains are filled, they leave to get unloaded and then later return to the mine. Sometimes the wagons end up in different positions in the train. Our aim is to maximise tonnes in every wagon.

We've noticed two things

1. some wagons IDs load consistently low in tonnes
2. wagons in poisitions near the start or end of the train load consistently low. i.e. tonnes go down as distance from middle position goes up

I've got a linear model fitted in R like

    > lm(df$tonnes ~ df$dist_fr_middle)
Coefficients:
(Intercept)    df$dist_fr_middle 113.92001 -0.03915  How do I go about creating a new column in my dataframe for distance-from-middle-adjusted tonnes? Thanks for your help! - In R df is used for the density function of the F distribution. Assigning new values to built in names is possible, but it may lead to unpredictable results. – GaBorgulya Apr 28 '11 at 10:58 I do not understand what such adjusted data will be used for, it will not provide the solution that maximizes tonnes in every wagon? Residuals, as GaBorgulya suggested, shows just the deviations from the average. Well it is not what you are actually looking for, isn;t it? I think, you have to play with some mean value. For example$\max(tonnes)$will be just intercept term of your regression 113.92, but it is hardly reachable. You may try to fill the gap between mean(df$tonnes) and this value by some technically reasonable proportion. –  Dmitrij Celov Apr 28 '11 at 11:43
df$locationAdjustedTonnes = resid(lm(df$tonnes ~ df$dist_fr_middle))  A general way is tweaking this (The example below gives the same result as the one above): df$locationAdjustedTonnes = df$tonnes - predict(lm(df$tonnes ~ df$dist_fr_middle))  - Hi GaBorgulya, it seems that the way you've written those, df$locationAdjustedTonnes returns the difference between actual values and the model. Was that intentional? If so, no worries, I've changed it to df$locationAdjustedTonnes = df$tonnes - resid(lm(df$tonnes ~ df$dist_fr_middle)). –  TMOD Apr 28 '11 at 22:09
Apologies for not knowing what I'm doing, but I was hoping that a model wouldn't fit all values of df$tonnes exactly onto the regression line. I thought it would instead adjust all values of df$tonnes by adding the mean residual for a particular dist_fr_middle to each value at that dist_from_middle –  TMOD Apr 28 '11 at 22:16
Sorry, you're right, I made a typo. Please disregard my previous comment. qplot(dist, locationAdjustedTonnes, data = df) does show the data with the linear effect of dist_fr_middle removed. –  TMOD May 14 '11 at 8:10