Linear Regression - How to remove features behaves Heteroscedastic? I'm making a model with 5 features (Paths, PathSegs, Polygons, Scalar Instances, and Standard Vias), and the output is memory consumption.
Essentially using these 5 features, I can predict how much memory consumption I will use.
Here is a diagram of the features vs memory usage.

Some features seems to be Heteroscedastic, especially Scalar Instances and Standard Vias. 
If I trained a model (linear regression), and then plotted the output with the scalar instances, I would get something like this.

The problem is with the big jump when the Scalar instances are low. I imagine that when the features are small, then the memory consumption would also be small as well, then gradually increase.
I've tried a few different approaches with different models:

And all of them have the same issues. So I'm thinking if I threw away Standard Vias, and Pathsegs, which has a big jump at low numbers, then I would get a more stable model. However, is there are systematic way of doing this?
 A: (Summarizing from our conversation in the comments to your question).
Looking at the graphs, there is no apparent issue. You ran multiple regression and got an equation of the form: $y = \sum{m_i \times x_i} + c$. Afterwards, you are plotting $y$ against $x_i$ for a single $i$ so it isn't surprising that what you get (a) is not a line; and (b) is not even strictly increasing. The reason is that other $x$'s affect $y$. So, for example, when scaler instances is low, perhaps standard vias is high. As such, given with what know, there doesn't appear to be a problem with the graphs.
As for removing variables, there is no apparent reason to do at the moment. Its possible that some of  your features don't have much of an effect on memory usage but that's okay since the coefficients for this variables will be small.
By the way, if you do want to have a graph that shows just the effect of one feature (e.g. scaler instances) on memory usage, you can do this by holding the other features constant. Of course, the resulting line won't necessarily match up with the points in your data set, but that's okay because in those points, the other features are not being held constant.
