TLDR; I want to know the percentage % of explained variance of the dependent variable given a list of D independent variables with crazy different scales -- but I believe that given convexity of regression theoretically we don't need to normalize features. Is this correct?
I have a dependent variable ($y$)and I want to understand how much each of my independent variables ($x_i$) explain the $y$'s variance (ideally as a percentage). This sounds to me as a standard R^2. However, I have some $x_i$'s are bounded between $[0,1]$ and others can have a value of 10 Billion (100B, 10_000_000_000). Due to convexity the solution should be easy to find (though I'm realizing that perhaps if we have more data points that features some subtleties of ill-posedness/none-uniqueness might come into place, but regardless, even the psuedo norm gives the min norm solution in all I cases I think, so I think it's still easy to find due to convexity). Ideally I'd try to collect as many number of points as possible & hopefully the hand crafted/chosen features I am choosing for the scientific study explain the variance.
So is it correct that due to convexity normalization is not important in terms of the statistical results? (I infer numerically it might matter but just statistically does it matter?)
What about plotting y vs x?
Intuitively, what I am trying to see is how each $x_i$ explains $y$ -- or if it correlates with $x_i$. i.e. if $y$ is some function of $x_i$. One other way to do this is to plot the raw $y$'s vs $x_i$'s -- however, this doesn't scale beyond 2D point x's (perhaps 3D x's) -- but in my mind that would be nearly ideal way to plot the data because it wouldn't involve any numerical issues or statistical choices, we just see if $y$ really varies as $x_i$'s vary. It also, doesn't depend on the data because "nature" deals with the normalization implicitly/by itself when it computes $y = f(x)$ and we just try to plot the raw points. But of course this doesn't scale. Another way is to fix a subset of $x$'s and just plot 2 at a time. This idea come from trying to avoid the normalization of the features.
One case I realized makes sense to normalize is with synthetic data. If I generate synthetic data where the $x_i$'s are of such difference -- then I must normalize the features of choose the coefficients very carefully such that the 10 billion scale features doesn't dominate the output just randomly. So normalization when creating synthetic data seems important -- at least as a thought experiment.
So when do I need to normalize my features? Does my use case need it? Theoretically is it actually needed when the problem is convex?