Is there a way to measure the degree of similarity between two columns? I have a dataset of a list of genes with predicted scores (of likelihood to cause disease) from 2 different machine learning classifiers:
Gene            Score1      Score2
RP11-983P164    0.2678077   0.2119513
SLC25A20        0.2644568   0.2586816
GLS             0.2560175   0.2631010
IKZF4           0.2468294   0.2189585
NRIP3           0.2446390   0.2170968
SENP1           0.2372014   0.2724868
SLC27A6         0.2321821   0.2218227
SRFBP1          0.2293986   0.2688244
OBFC1           0.2279012   0.2187441
STEAP2          0.2239941   0.2001475

I want to measure if any of the two predicted scores per gene are significantly different from each other, or if the predictions are very similar. I have a biology background so I'm not sure what to start with searching for this, and so sorry if I've asked this question in the wrong place, any help would be appreciated.
Edit:
I now have 6 score columns in total (all look similar to Score1 and Score2) - are there any other statistical tests I can do? Would it be worth doing a t-test?
 A: One way to measure similarity is to estimate the correlation between Score1 and Score2. This will give you a number between -1 and 1 and the closer to 1 the higher the linear association between the scores.
If it is negative, then an increase in Score1 will give a decrease in Score2. This would mean that your models do not agree at all.
If the correlation is close to 0 then there is no linear relationship between Score1 and Score2.
A: It sounds like you just want a correlation matrix.
For x columns, this measures the correlation between each column's data.
Here, (Pearson's) correlation is a normalised version of the covariance of any two variables, so you don't need to worry about units.
In R, just read in your data frame with the 6 score columns. Since you want to check for significant differences, you can also do that with the Hmisc package, which gives significance levels (and yes, it uses the t-test for continuous scores).
# Just get correlation scores
cor_matrix <- cor(df)
cor_matrix

# Get correlations *and* p-values of correlations for each pair
# Install Hmisc package first
library("Hmisc")
cor_matrix2 <- rcorr(as.matrix(df))
cor_matrix2 # Gives a correlation matrix and a p-value matrix

Each element $x_{s1,s2}$ in the correlation matrix is $\in [-1,1]$, where 1 is perfectly correlated and -1 is perfectly inverse-correlated. Hence the diagonals will all be 1.
There are a number of assumptions made in calculating Pearson's correlation coefficient that you may or may not care about. E.g. if any of the data is ordinal, use Spearman's correlation coefficient instead; cor_matrix <- cor(df, method="spearman"). Check out the cor and rcorr help for more info on the R function and assumptions in general.
A: Cosine similarity is a thing you are looking for ;) By definition it is a measure of similarity.
https://www.r-bloggers.com/2021/08/how-to-calculate-cosine-similarity-in-r/
