How to calculate centrality measures in a 4 million edge network using R? I have a CSV file with 4 million edges of a directed network representing people communicating with each other (e.g. John sends a message to Mary, Mary sends a message to Ann, John sends another message to Mary, etc.). I would like to do two things:


*

*Find degree, betweeness and (maybe) eigenvector centrality measures for each person.

*Get a visualization of the network.
I would like to do this on the command-line on a Linux server since my laptop does not have much power. I have R installed on that server and the statnet library. I found this 2009 post of someone more competent than me trying to do the same thing and having problems with it. So I was wondering if anyone else has any pointers on how to do this, preferably taking me step by step since I only know how to load the CSV file and nothing else. 
Just to give you an idea, this is how my CSV file looks like:
$ head comments.csv
    "src","dest"
    "6493","139"
    "406705","369798"
$ wc -l comments.csv 
4210369 comments.csv

 A: What you have is an edge list, which can be converted to a network object using the network library. Here is an example using fictitious data.
library(network)

src <- c("A", "B", "C", "D", "E", "B", "A", "F")
dst <- c("B", "E", "A", "B", "B", "A", "F", "A")

edges <- cbind(src, dst)
Net <- as.network(edges, matrix.type = "edgelist")

summary(Net)
plot(Net)

However, a warning is in order: you have a very large network and I am not sure a plot will be all that informative. It will probably look like a big ball of yarn. I am also not sure how well these libraries deal with such large datasets. I suggest you take a look at the documentation for the network, statnet, and ergm libraries. The Journal of Statistical Software (v24/3) offers several articles covering these libraries. The issue can be found here:
http://www.jstatsoft.org/v24
A: I don't think that R is a first choice here (maybe I'm wrong). You will need huge arrays here to index and prepare your networks files in the appropriate data format. First of all, I will try to use Jure's (Rob mention him in the post above) SNAP library; it's written in C++ and works very well on large networks.
A: Gephi ( http://gephi.org/ ) might be an easy way to explore the data. You can almost certainly visualize it, and perform some calculations (though I have not used it for some time so I can't remember all the functions).
A: From past experience with a network of 7 million nodes, I think visualizing your complete network will give you an uninterpretable image. I might suggest different visualizations using subsets of your data such as just using the top 10 nodes with the most inbound or outbound links. I second celenius's suggestion on using gephi.
A: If you're concerned with the size of the network, you could try the igraph package in R. And if that performs poorly inside R, it might do better as Python module.
Or even the networkx package for Python 
A: Do you suspect that the network has a small number of very large connected components?  If not, you can decompose it into distinct components which will make it much easier to compute measures of centrality.
A: There are several R software packages one could use, including "sna" and "network". One thing I wouldn't necessarily rely on if you're having performance issues with sna is NetworkX. I love NetworkX to death, and use it for most of my analysis, but NetworkX is quite proud of being a mostly purely Pythonic implementation. It doesn't particularly exploit speedy pre-compiled code well, and sna often outpaces NetworkX by a considerable margin.
