# How to start an analysis of keywords from a bibliography and detect correlations?

(I'm a bit outside my comfort zone, so apologies if this is badly worded, or off-topic)

I have a bibliographic database, containign details of about 1200 different papers, books, web sites etc, all with various details, including keywords and an abstract. I want to somehow analyse this database and produce some graphics showing the correlations between different keywords. (like "drug" is often present with either "pharmacology" or "assay").

Ideally this would be in R, but general advise would also be welcome. (I've seen this question/answer which piqued my interest, and this heatmap graphic also seem related)

My database could be in bibtex, or could be converted to plain text.

-

I'm also outside my area of expertise, but assuming that you want to use R, here are a few thoughts.

• There is a bibtex package in R for importing bibtex files.
• Various character functions could be used to extract the key words.
• The data sounds a little like a two-mode network, which might mean packages like sna and igraph are useful.
• Plots of 2d multidimensional scaling can also also be useful in visualising similarities (e.g., based on co-occurrence or some other measure) between words (here's a tutorial).
-

so you have a document x keyword matrix which basically represents a bipartite graph (or two-mode network depending on your cultural background) with edges between documents and tags. If you're not interested in individual documents - as I understand you -, you can create a network of keywords by counting the number of cooccurrences between each keyword. Simply plotting this graph might already give you a neat idea of what this data looks like. You can further tweak the visualization if you, e.g., scale the size of the keywords by the number of total occurrences, or (in case you have a lot of keywords) introduce a minimum number of total occurrences for a keyword to appear in the first place.

As a tool, I can only recommend GraphViz which allows you to specify graphs like

keyword1 -- keyword2
keyword1 -- keyword3
keyword1[label="statistics", fontsize=...]


and "compile" them into pngs, pdfs, whatever, yielding very nice results (particularly if you play a bit with the font settings).

-

You may want to take a look at the phi coefficient which is a measure of association for nominal variables.

-

You could try to employ the theory and praxis of association analysis or market basket analysis to your problem (just read "items" as "keywords" / "cited reference" and "market basket" as "journal article").

Disclaimer - this is just an idea, I did not do anything like that myself. Just my 2Cents.

-