The dataset comprises bandwidth usage for each customer. There is also a hybrid metric based on the distance covered by each traffic flow and aggregated to obtain 'Bit-Miles' for each customer (sum of traffic $\times$ miles for each flow).

Clearly, there is some (causal) dependence among the above features but owing to skewness I had to resort to transformations (log, square root, z-scores etc.)

After transforming both bandwidth and bit-miles, I see a strong linear relationship.

Does this imply that bit-miles is a redundant feature (which seems counter-intuitive IMHO had there been no transformation)?

Can I somehow prove/disprove that 'Bit-Miles' is redundant?

Here are the plots before and after the transformation. before


Here is a small view of the data as well.

Name            Traffic(bps)            Bit-Miles
Customer1       729797243234.54         416983889869721.00
Customer2       411886504711.92         43841920479614.30
Customer3       253240650503.96         269534485841579.00
Customer4       251982742984.49         158900272002478.00
  • $\begingroup$ Its worth pointing out that the observations are not independent since traffic from each customer is inherently dependent on the traffic from/to another customer. I'm not sure which statistics should make sense in this case. $\endgroup$ – tool.ish Mar 6 '14 at 8:57

I think "redundant" may be too strong. First, the relationship, while strong, isn't perfect (what is r on the transformed data?) Second, the variable on the y-axis has some features that the one on the x axis does not (a floor effect and a gap).

| cite | improve this answer | |
  • $\begingroup$ Ahh. I suspected that and wanted to get an expert's opinion. Thank you very much. Notice another feature that there is an almost perfectly linear top boundary on the transformed data? $\endgroup$ – tool.ish Mar 6 '14 at 6:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.