Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
165

The answers so far have focused on the data itself, which makes sense with the site this is on, and the flaws about it. But I'm a computational/mathematical epidemiologist by inclination, so I'm also going to talk about the model itself for a little bit, because it's also relevant to the discussion. In my mind, the biggest problem with the paper is not the ...


111

My primary concern with this paper is that it focuses primarily on Google search results. It is a well-established fact that smartphone use is on the rise (Pew Internet, Brandwatch), and traditional computer sales are declining (possibly just due to old computers still functioning) (Slate, ExtremeTech), as more people use smartphones to access the internet. ...


61

Well, this paper establishes the fact that the number of Google searches on Facebook fits a certain curve nicely. So at best it can predict that the searches on Facebook will decline by 80%. Which might be feasible, because Facebook might become so ubiquitous that nobody would need to search about it. The problem with such type of models is that they ...


16

That area is called microtargeting (if you would like to google for it). Campaigns are pretty secretive about their tools and procedures, so to my knowledge there is not that much published work except Hal Malchow's Political Targeting (2008) or Green & Gerber's (2008) Get out the Vote: How to Increase Voter Turnout (the latter deals more with social ...


15

Google Trend in my opinion can't produce a good data set for this case of study. Google trend shows how often a term is searched with Google so there are at least two reasons for raising some doubts about the prevision: We don't know if the user searches on Google Facebook to log in or if he searches information about Facebook Facebook is not only a site ...


13

A few basic issues stand out with this paper: It assumes correlation of search engine queries about a rising social network with the membership increases. This may have correlated in the past, but may not in the future. There are very few new large social networks. You can almost count them on one hand. Friendster, Myspace, Facebook, Google+. Also, Stack ...


13

A lot of techniques assume that data is sampled at regularly-spaced intervals. You might count how much litter is near each mile marker on the highway, or sample points in a forest on a regularly spaced grid (100, 200, 300, ... meters north and 100, 200, 300 meters east of some landmark). This also occurs in time--my EEG machine records a data point every ...


11

Pretty much any decent stats package will provide a log-gamma or log-factorial function. You mention R; it has: lgamma which is the log of the gamma function lfactorial which is the log of the factorial function lchoose which is the log of the binomial coefficient. using any of these, you can work out the log of the desired probability. If it's not going ...


9

All centrality measures are dependent on the shape of your data. Laplacian centrality is a convincing measure of centrality for weighted graphs. Define a matrix to store our weights. $ W_{ij} = \left\{ \begin{array}{lr} w_{ij} & : i \neq j\\ 0 & : i = j \end{array} \right. $ Define a matrix, where the diagonal is the sum ...


6

The question isn't "if" but "when". That it will end is already guaranteed. http://www.ted.com/talks/geoffrey_west_the_surprising_math_of_cities_and_corporations.html I take umbrage with the use of the SIR model. It comes with assumptions. One of the assumptions is that eventually everyone is "recovered". Infections are not perpetual, while technology ...


6

Good answers by Matt (+1) and others. Just to have a picture to drive the message (visually) home. In the following figure assuming that the squares represent sampling points the grey boxes follow an obvious regularly spaced design; the red box are just random samples that are irregularly spaced. Both designs have their pros and cons. Do not dismiss the ...


5

There is much of relevance at Graph for relationship between two ordinal variables The detail there of using ordinal variables does not bite with your problem where the workers are just different. You might need to expand on "appealing": there is often tension here between clever and unusual but difficult to decode and basic and simple but easy to ...


4

A short overview of community detection approaches is in this blog post. It bases on a longer overview, being the standard reference: S. Fortunato, Community detection in graphs, Physics Reports 486, 75 (2010), arXiv:0906.0612. Two nice algorithms working for big graphs are Louvain and Infomap, with the later (arguably) having stronger theoretical ...


4

This usually means that there is no clear underlying structure of the position of the points. I.e. it is not a rectangular grid or anything that can be represented compactly which has a clear structure. Imagine that you have weather stations around a country and you are monitoring temperature. These weather stations are most likely no on any specifically ...


4

I think you should have a look at the block-cut tree of our graph. The number of biconnected components a node is contained in is exactly the number of fragments the graph decomposes into when this node is removed. Or, put differently, your index is the degree of that node in the block-cut tree.


3

The closest concept I can think of in network science is betweenness centrality. Suppose you're interested in studying some node $u$. First, for every pair of nodes in your network (excluding $u$), you identify all possible shortest paths. Then, you count how many of them are passing through $u$. Some variants normalize this number in various ways, but, ...


3

Not sure if this amounts to an answer but here are a few remarks: The fact that variables are related in some way (e.g. that users who comment a lot tend to visit a lot pages) is actually a good thing. Otherwise, it wouldn't make sense to derive a single activity measure or indeed apply any dimensionality reduction technique. Beside dimensionality, the main ...


3

Last summer, Kaggle ran a competition to predict a users psychopathy, Machiavellianism, Narcissism etc. using only Twitter data. You can see the competition and results here: personality competition psychopathy I recall there was a published paper on the psychopathy prediction, here: http://www.onlineprivacyfoundation.org/research_/...


3

netlogit() and rmperm() (which is used for netlogit's QAP tests) represent networks with matrices. Representing relatively large networks with matrices will use considerable memory. Not related to memory usage, but the underlying code for netlogit() and rmperm() use for loops within R, which can be slow.


3

Response from Carter Butts: At present, those functions make heavy use of adjacency representations for the underlying data, and hence do not scale to graphs with more than ~40,000 or so nodes (give or take) - getting around this limit would require a very different implementation. (Generating sufficiently high quality random permutation vectors might ...


3

I think your question might be less of a method question and more of a theoretical question about what you are trying to achieve with your data. If you are interested in co-occurences only, does it really matters that some individuals have more time than other to form ties in order to find subgroups? Now, you could think of some ways to correct for the ...


3

You can fit an intercept only model, e.g. $y = a + \rho W y + e$ in lagsarlm. library(spdep) data(oldcol) Bin_W <- nb2listw(COL.nb, style="B") Empt <- lagsarlm(CRIME ~ 1, data=COL.OLD, listw=Bin_W) summary(Empt) I don't think there is anything wrong with using $\rho$ with the intercept to calculate your centrality measure, but I am not sure 100% sure....


3

As the logarithm is a monotone function, and because $\log(\prod_{i}a_i) = \sum_i \log(a_i)$ you can find the maximal product path by searching for the shortest path in the graph where you replaced the weights $a$ by $-\log(a)$ using the classic methods.


3

Check out the nodeicov() and nodeocov() terms. I'm assuming (can't comment yet!) that your edges represent conflict between ties, where $i \rightarrow j$ indicates that $i$ reports a conflict with $j$. The first term would tell you if individuals with greater extraversion scores are more likely to have in-directed conflict ties. The second would tell you if ...


3

Original Poster here, I found the answer to the question. The problem is "transitivity" wants to say "if $i$ is connected to $v$ and $v$ is connected to $w$, then what is the probability $i$ is connected to $w$?" So, where the denominator says "paths of length 2", it should say "paths of length 2 that are not loops". We are not interested in paths like $i ...


2

Find PNet here: http://sna.unimelb.edu.au/PNet This is Java based software for fitting exponential random graph models, now including a multilevel version. Incidentally, RSiena does not fit ERGM models. The old R-independent SIENA software (which is no longer maintained) did, however.


2

To model the global effects of co-variates such as exposure to advertising on joining/not-joining membership plans logistic regression is a useful approach. Modeling duration between exposures and joining/not-joining will require other more complicated forms of regression. After accounting for global effects, a tool from spatial statistics-- Moran's $I$-- ...


2

Since the two components that together explain almost 80% of the variance are orthogonal, most likely there is no single "activity" measure. If I were you, I would study the two components, trying to understand what types of activity they measure, on which variables they depend etc. I would also study the PCA plots to understand whether there are for example ...


2

The first graph you describe here is a tripartite graph, which means it has three types of nodes, and links only between nodes of different types. The second graph you describe, containing only user nodes, is the result of the so-called projection over the user dimension. However, performing such an operation results in a loss of information, because several ...


2

Questions #1 and #3 (and maybe #2) seem like they could be formulated as simple regression problems. Based on your question, it seems like you may have considered this and rejected it already. Here's a stab in the dark anyways... (Q1) Quantify the amount of agreement/disagreement in each group as the dependent variable ($y$). I assume you have a way of ...


Only top voted, non community-wiki answers of a minimum length are eligible