Comparing values to detect statistically significant difference

Question

I want to compare 2 values to see if there considerable significant different between them.

Basically these values comes from 2 credits score bureau providers (score 1 and score 2). The score 3 is just the avg between score 1 and 2. and we base our decision on this score 3.

+---------+-----------+----------+----------+
+       id+    score 1+   score 2+   score 3+
+---------+-----------+----------+----------+
+  006abXi+        100+        50+        75+
+---------+-----------+----------+----------+
+  006fsRe+        600+       550+       575+
+---------+-----------+----------+----------+
+  006csXm+        320+       350+       335+
+---------+-----------+----------+----------+

We are trying to identify if it is worth to keep using score 2, and just based our decision on the first credit bureau.

My current sample is a bit under 3500 records. And to have a very rough idea, I did an average of the score 1 and score 2 and see if there is much difference between them

+---------+-----------+----------+----------+
+      avg+    score 1+   score 2+   score 3+
+---------+-----------+----------+----------+
+         +     556.92+    584.14+    570.53+
+---------+-----------+----------+----------+

Doesn't look much but I do not think this will be the right way to base this decision on. I am wonder if there is any formula or sort of graph that I can use to visualize and be able to base my decision in some more scientific background?

Answer 1

You want to "see if there is considerable significant different between them".

With a sample size of 3500, your difference will very likely be significant. You could check using some appropriate paired difference test like the paired t-test or Wilcoxon signed rank test.

Whether the difference is considerable depends on what you want to do with it, what constraints exist in your task, and what you are willing to tolerate.

As far as visualization is concerned, I would suggest a histogram for the score differences.