Newest questions tagged networks - Cross Validated most recent 30 from stats.stackexchange.com 2019-07-17T08:44:20Z https://stats.stackexchange.com/feeds/tag?tagnames=networks&sort=newest http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/413043 0 Summary statistics for bipartite networks Ezra https://stats.stackexchange.com/users/250944 2019-06-14T14:12:30Z 2019-06-14T14:12:30Z <p>I have a large bipartite network that I would like to summarise. So far, I have found the following summary statistics:</p> <ul> <li>Degree centrality</li> <li>Graph density</li> <li>Modularity</li> <li>Nestedness</li> </ul> <p>I have not found a textbook or review that presents commonly used summary statistics in the analysis of bipartite networks. Does such a summary exist? Could you recommend any other summary statistics that could be useful?</p> https://stats.stackexchange.com/q/412354 1 Reference request: Network/graph topology inference Rodrigo Zepeda https://stats.stackexchange.com/users/81181 2019-06-10T18:18:48Z 2019-06-11T21:35:02Z <p>I am a mathematician looking for a survey/book on methods for inference of graph/network topology (structure). Specifically, the kind of problem I am looking to study is as follows:</p> <blockquote> <p>Given a graph <span class="math-container">$G$</span> consider an unknown function <span class="math-container">$f$</span> such that <span class="math-container">$f(G)=y$</span> (for a real number <span class="math-container">$y$</span>). Assume we have a collection of graphs (<span class="math-container">$G_1, G_2, \dots, G_n$</span>) for which the values <span class="math-container">$f(G_k)=y_k$</span> are known. Given a proposed value <span class="math-container">$y_*$</span>:</p> <ul> <li><strong>How can one reconstruct a graph <span class="math-container">$\hat{G}_*$</span> such that <span class="math-container">$f(\hat{G}_*)\approx y_*$</span>?</strong> </li> <li><strong>How about confidence intervals or high-density regions in some space of graphs?</strong></li> </ul> </blockquote> <p>What I am not interested in is:</p> <ul> <li>A book such as Durrett's <em>Random Graph Dynamics</em> which presents specific probabilistic models of graph generation but no inference. </li> <li>Kolaczyk's <em>Statistical Analysis of Network Data</em> which focuses on inferring part of a network's topological descriptors but not the whole network.</li> </ul> https://stats.stackexchange.com/q/410716 0 Network analysis - Correlation is positive and significant, but coefficient of simple logistic regression is not significant? raffamaiden https://stats.stackexchange.com/users/154990 2019-05-29T18:49:36Z 2019-05-31T13:52:20Z <p>I have an adjacency matrix and another which represents whether the two nodes share an attribute. Consider it like an homophily test. We want to test if the likelihood to form a connect depends on the fact that the two nodes have an attribute in common. Now, using R and SNA package, I run a correlation and test is significance through a QAP test:</p> <pre><code>g &lt;- array(dim=c(2,nrow(x),nrow(x))) g[1,,] &lt;- x g[2,,] &lt;- y q.12 &lt;- qaptest(g, gcor, reps = 2000, g1=1, g2=2, diag=FALSE) </code></pre> <p>The correlation is 0.7479487, and the p-value is 0</p> <pre><code>QAP Test Results Estimated p-values: p(f(perm) &gt;= f(d)): 0 p(f(perm) &lt;= f(d)): 1 </code></pre> <p>Then I fit a logit to that data</p> <pre><code>nl &lt;- netlogit(y, x, mode="digraph", diag=FALSE, nullhyp="qap", reps=2000) </code></pre> <p>but its coefficient is not significant. How is that possible?</p> <pre><code>Network Logit Model Coefficients: Estimate Exp(b) Pr(&lt;=b) Pr(&gt;=b) Pr(&gt;=|b|) (intercept) -2.940634 5.283224e-02 0.000 1.000 0.00 x1 21.506702 2.188981e+09 0.519 0.481 0.97 Goodness of Fit Statistics: Null deviance: 17234.41 on 12432 degrees of freedom Residual deviance: 4617.118 on 12430 degrees of freedom Chi-Squared test of fit improvement: 12617.29 on 2 degrees of freedom, p-value 0 AIC: 4621.118 BIC: 4635.974 Pseudo-R^2 Measures: (Dn-Dr)/(Dn-Dr+dfn): 0.5036986 (Dn-Dr)/Dn: 0.7320989 </code></pre> https://stats.stackexchange.com/q/407661 0 Review paper on causal modelling of complex networks Aidan Rocke https://stats.stackexchange.com/users/44966 2019-05-10T12:47:23Z 2019-05-12T20:13:10Z <p>Although I have a growing interest in network neuroscience and complex neuroscience in general, I have quite a bit of trouble following Twitter discussions on causal modelling of brain network connectivity such as this <a href="https://twitter.com/KordingLab/status/1126476338670592000" rel="nofollow noreferrer">Twitter discussion between the neuroscientists Konrad Kording and Daniele Marinazzo</a>. This particular discussion appears to focus on the limitations of dynamic causal modelling. </p> <p>Might there be a series of papers or a single text which serves as an overview on causal modelling of complex networks? Ideally, this text would cover the state of the art as well. </p> <p><strong>Note:</strong> By complex network I mean a network whose behaviour isn't completely random and whose individual components can have a highly non-linear effect on the whole network. Examples include social networks, brain networks, protein interaction networks...almost any real-world network. </p> <h2>References:</h2> <ol> <li>S.J. Kiebel, M.I. Garrido, R.J. Moran , and K.J. Friston. Dynamic causal modelling for EEG and MEG. Cogn Neurodyn. 2008. </li> <li>G. Lohmann, K. Erfurth, K. Müller, and R. Turner. Critical comments on dynamic causal modelling. NeuroImage. 2012. </li> <li>K. Friston, J. Daunizeau, and K. Stephan. Model selection and gobbledygook: Response to Lohmann et al. NeuroImage. 2012. </li> <li>D. Mehler and K. Kording. The lure of causal statements: Rampant mis-inference of causality in estimated connectivity. arxiv. 2018. </li> <li>B. He, L. Astolfi, P. Valdés-Sosa, D. Marinazzo, S. Palva, Christian-George Bénar, C. Michel, and T. Koenig. Electrophysiological Brain Connectivity: Theory and Implementation. IEEE Transactions on Biomedical Engineering. 2019. </li> </ol> https://stats.stackexchange.com/q/401464 1 Missing values in a variable depending on the values of another variable Mark https://stats.stackexchange.com/users/243364 2019-04-06T05:39:11Z 2019-04-07T17:31:14Z <p>I'm working on a public procurement dataset where I have information on all the participants that presented offers in 358 tenders. I'm analysing relationships between all the companies of the dataset (1242). I'm running a logistic network regression that allow to predict a relationship knowing another. In my case, for example, I'm interested in understanding the extent to which companies that frequently submit the same offer in a tender are also part of the same cartel (I have information on 8 different colluding cartels active in these tenders).</p> <p>For each dyad (pair of companies) I'm calculating:</p> <p>a) the number of times they participated together, and b) the number of times they submitted exactly the same offer.</p> <p>I am unsure about the b) measure. When I calculate it, I obviously get missing values everytime two companies did not participate together in a tender and so did not have the "opportunity" to submit the same offer. This variable seems to create problems in regression because of the many missing values (85% of the dyads-observations are missing). Consider that the missing are not random and, as I said, I'm perfectly aware that they are missing "by default", because companies that did not participate together, did not bid on the same contract and by consequence could not bid the same offer! I thought that I could fill missing values with "0", thus without requiring this variable to depend to much co-bidding in the same tender. Do you think this approach makes sense or is it a way to force the data too much?</p> https://stats.stackexchange.com/q/401060 1 representative nodes in modular network saghi https://stats.stackexchange.com/users/64884 2019-04-03T21:35:56Z 2019-06-30T19:15:09Z <p>I want to find the most representative nodes in each module in a modular network. I have used the Louvain algorithm on my graph and found two main modules. Now I want to know what nodes are the most infuential in this structure. e.g. nodes that are connected other nodes in the same module rather than to nodes in the other module. </p> <p>Is there any node-level quantity based on the structure to represent this concept? </p> https://stats.stackexchange.com/q/399932 0 Inclusion of AIC statistic in QAP netlogit regression output NBK https://stats.stackexchange.com/users/192696 2019-03-28T15:17:06Z 2019-04-22T13:32:17Z <p>I am using the <code>netlogit</code> function of <code>sna</code> for a QAP regression. The output looks something like the table below. You will see that the AIC and BIC statistics are included, which I understand is not warranted in QAP regressions. But why are these statistics included in the default output of the <code>netlogit</code> function +<code>qapx</code> null hypothesis? Should I remove them manually, or are they there for a reason?</p> <pre><code>library(statnet) library(texreg) qapx &lt;- netlogit(yyy, list(xxx, yy, zzz), nullhyp=c("qapx"), reps = 1000) htmlreg(qapx, file = "output.doc") </code></pre> <p><a href="https://i.stack.imgur.com/jCJqP.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/jCJqP.png" alt="enter image description here"></a></p> https://stats.stackexchange.com/q/399575 1 "Multi-level" social network analysis? georgem https://stats.stackexchange.com/users/242538 2019-03-26T20:14:05Z 2019-03-26T20:43:19Z <p>I am very new to social network analysis and working on a project here that I am not sure what's possible to model. </p> <p>I don't want to quite call this multi-level because I am not exactly looking at variability over multiple levels (although maybe I am), but here is what I am interested in. The exact context here is community health workers who put on programs in their community with the help of local non-profits. </p> <p>I want to see what kinds of networks between <em>non-profits</em> are built by the community health workers. For example, maybe there has never been collaboration before between two non-profits and that has changed due to the effort of the health worker. </p> <p>It's a silly example but this is the type of data I would think I would want to collect -- individuals and the entities they worked with. </p> <p><a href="https://i.stack.imgur.com/5Yosz.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/5Yosz.png" alt="enter image description here"></a></p> <p>After that, I am not sure how I would model this as I'm interested in a network of organizations as "connected" by individuals.</p> <p>Like I said I am new to this area so if I'm overlooking something, not explaining this well or just all wet, let me know! Thank you. </p> https://stats.stackexchange.com/q/396630 1 Finding the effect of nodes on a density heatmap NeonBlueHair https://stats.stackexchange.com/users/96296 2019-03-10T03:54:28Z 2019-03-10T07:53:29Z <p>Let's say I have a <em>geo-tagged</em> dataset of all payment transactions for businesses in a city. I know whether each payment is made by cash or card, and have made a heatmap of where in the city the highest rate of cash payments occur. Now I have a hypothesis that businesses closest to ATM's have higher rates of cash payments since people leave ATM's with cash in their pockets. If I have this heatmap of the cash rate and the ATM as nodes on the map, how can I test my hypothesis? Is there a name for this type of problem or a typical approach to it? Note that I'm looking to solve this problem with Python, so programming-based solutions and package referrals would be appreciated. </p> https://stats.stackexchange.com/q/396216 0 Picking the two most distant classes user2295350 https://stats.stackexchange.com/users/12307 2019-03-07T19:37:26Z 2019-03-07T19:37:26Z <p>I have a big multi-label dataset which consist of thousands of classes. I would like to find the best way to choose the two most distant classes, i.e. classes that not only never co-exist but also are as little relevant to each other as possible. Ultimately, my goal is to covert this problem into a binary classification problem and use these results as a starting point. So, my question is, how to tackle this problem? Should I perform some sort of <code>clustering</code> with classical <code>machine learning</code> or I should see this as a big <code>graph</code> and use a <code>network analysis</code> package such as <code>networkx</code>?</p> https://stats.stackexchange.com/q/393141 1 How to obtain the statistical significance of a given community structure for a directed network? Andrew https://stats.stackexchange.com/users/238178 2019-02-18T21:15:05Z 2019-02-19T15:50:57Z <p>So, I have several directed (multi-edged) networks, and within them each node has been assigned to one of seven categories (based on some <em>a priori</em> circumstances). Each category <em>should</em> have a higher within-category interaction rate, but I want to test the statistical significance of this.</p> <p>Since "a set of nodes, densely connected internally" is pretty much the definition of a community, I want to manually impose my community assignments on the nodes in the network and then test whether this assignment is statistically more "community-like" than a random assignment. <strong>In essence, my question is: "Is this given community structure statistically significant?"</strong></p> <p>I found <a href="https://journals.aps.org/pre/abstract/10.1103/PhysRevE.81.046110" rel="nofollow noreferrer">this paper</a> which seems to have a way of measuring the statistical significance of a single community group in the network, but it doesn't seem to apply to a given, entire community structure. I also found <a href="http://snap.stanford.edu/networks2013/papers/netnips2013_submission_7.pdf" rel="nofollow noreferrer">this baby</a>, but it seems to only be focused on much smaller, local structures. </p> <p>There's gotta be a way to do this for directed, multi-edged networks, I just can't seem to find any! (Additionally, I'll have to do this analysis in R, so mega-triple-extra bonus points if you know of an R package that already does this.)</p> <p>Thanks in advance!</p> https://stats.stackexchange.com/q/392532 0 Visual representation of strong associations of two categorical variables pteridin https://stats.stackexchange.com/users/237716 2019-02-14T18:22:00Z 2019-02-14T18:22:00Z <p>I have a dataset of one categoric variable "supermarket" for each individual person and multiple "product" categories per person and supermarket.</p> <p>E.g.:</p> <p>Person 1 went shopping in supermarket X and bought product A,B,C</p> <p>Person 2 went shopping in supermarket Y and bought A,D,E</p> <p>My main question is how can I visualise a strong link between supermarket and specific products in a meaningful way - ideally in R? </p> <p>So to speak: I want to point out the "specialities" of a supermarket.</p> <p>The category "supermarket" is limited to about 12 factors, the products are quite diverse and contain over one hundred different products. The number of products is unknown and does not matter in the current context since this is only an example.</p> <p>I want to get rid of the "background noise" of products that are frequently bought by the overall sample and want to point out that a buy of product B is strongly associated with shopping in supermarket Y, but masking product A which is bought more or less equally frequent in all other supermarkets.</p> <p>I already tried to subtract the frequency of products bought in each supermarket by the frequency of products bought in the total sample and creating a barplot with the top-20 products with positive difference in frequency difference, but I wonder if there is a more "fancy" and overall better approach to this problem.</p> https://stats.stackexchange.com/q/392416 0 Which regression used for normalized count data 762 https://stats.stackexchange.com/users/217318 2019-02-14T07:15:32Z 2019-03-05T16:21:51Z <p>I am working with social network data. I have multiple networks of various sizes and I'm calculating indegree (the number of connections between people) in each of the networks. I've been told to normalize the count of the number of connections by dividing by the number of possible connections for that network. I'm unsure whether this is accurate, but if it is, can I just work with a linear regression on this transformed data? If it's not accurate should I just go with the Poisson or Negative Binomial route? If someone can provide citations or proofs for reasons that would be greatly appreciated </p> https://stats.stackexchange.com/q/390535 0 Why does a minimum value cutoff in edge values create this shape in the log-log plot of the edge values aggregated to start nodes? 3eyes https://stats.stackexchange.com/users/26029 2019-02-03T06:34:31Z 2019-02-03T06:34:31Z <p>I have a weighted network dataset that looks like this: <a href="https://i.stack.imgur.com/10gqo.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/10gqo.png" alt="enter image description here"></a></p> <p>The <strong>prev</strong> and <strong>curr</strong> values contain webpages, and <strong>n</strong> is the number of times users went from the <strong>prev</strong> webpage to <strong>curr</strong> webpage. So, each data row is a directed weighted edge, with the (<strong>prev</strong>, <strong>curr</strong>) pair describing the edge and the <strong>n</strong> being its weight. The dataset was cut off at <strong>n</strong>=10 (all edges with <strong>n</strong> &lt;10 were dropped). </p> <p>A log-log plot of the edge-level traffic, <strong>n</strong>, looks as expected: <a href="https://i.stack.imgur.com/XGZvk.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/XGZvk.png" alt="enter image description here"></a> It has a minimum edge traffic of 10, and follows a power-law-like straight line on the log-log scale. </p> <p>When I sum up the edge traffic <strong>n</strong> by <strong>prev</strong> webpages, to get outgoing traffic per webpage, I get the following log-log plot: <a href="https://i.stack.imgur.com/SEFt0.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/SEFt0.png" alt="enter image description here"></a> My question is about that little bump in the distribution at the top left. It occurs at outgoing traffic volume = 20, so my guess is that it's caused by cutting off the edge-level traffic <strong>n</strong> at 10. But if that's the case, why don't the frequency values immediately return to the power-law-like straight line followed by the rest of the data?<br> Here's a log(y) plot close-up of that bump in the distribution: <a href="https://i.stack.imgur.com/NDVa8.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/NDVa8.png" alt="enter image description here"></a> </p> <p>I've tried cutting off the data at edge-levels <strong>n</strong>=100 and <strong>n</strong>=500, and got similar bumps in the distribution plots. <a href="https://i.stack.imgur.com/weIBk.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/weIBk.png" alt="enter image description here"></a> <a href="https://i.stack.imgur.com/nOi4H.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/nOi4H.png" alt="enter image description here"></a> <a href="https://i.stack.imgur.com/fuAZO.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fuAZO.png" alt="enter image description here"></a> <a href="https://i.stack.imgur.com/kEXWf.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/kEXWf.png" alt="enter image description here"></a></p> <p>I'm guessing that smaller and smaller versions of these bumps repeat throughout these distributions at multiples of the <strong>n</strong> cutoff.</p> <hr> <p><strong>To summarize, my question is:</strong><br> Why is the bump at the top left of these frequency distributions shaped like that? Why is it gradually going up instead of there being two disjoint downward-sloping straight lines? </p> https://stats.stackexchange.com/q/388708 0 Visualising a networkx graph with vosviewer Daphna Keidar https://stats.stackexchange.com/users/215263 2019-01-23T08:57:22Z 2019-01-23T08:57:22Z <p>I am trying to export a networkx graph into a format that vosviewer can read. When I wrote my graph to gml and pajek via nx.write_gml(G, path) and nx.write_pajek(G, path), vosviewer did not accept the output files. </p> <p>With the pajek file, I got the folowing error : "error while reading Pajek network file (line 509) the index of a vertex must be between 1 and 709"</p> <p>And with the gml file, I got this error: "error whie reading GML file (line 77) for each [there must be a corresponding]" </p> https://stats.stackexchange.com/q/388335 0 Combining centrality into one based on same data type 762 https://stats.stackexchange.com/users/217318 2019-01-21T08:01:12Z 2019-01-21T08:01:12Z <p>I'm working on a project that involves count data (specifically number of interactions) from multiple different districts in a specific area. Our team has been talking about calculating a few different centrality measures (degree centrality, closeness centrality) of the networks in the different districts. Due to limitations in our design, the nodes from different districts do not overlap in their interactions; basically, someone from district A did not ever interact with someone from district B in our data. If we are trying to see how network metrics may be affected by demographics (let's say age for example), does it make sense for us to create one model of the entire area for how certain network characteristics affect an individuals network metrics? Or should we create models of each individual network? I've heard some of these network metrics are affected by size so I imagine there might be a problem with treating these metrics as the same thing when their networks(districts) vary greatly in size. </p> https://stats.stackexchange.com/q/386695 0 Determining the closeness of nodes by their neighbours James Allen-Robertson https://stats.stackexchange.com/users/217252 2019-01-11T11:30:17Z 2019-01-27T15:22:36Z <p><strong>Background</strong></p> <p>I have a directional network that is comprised of two types of nodes, <code>business_sector</code> nodes, and <code>event_type</code> nodes. Currently there are only edges between <code>business_sector</code> nodes and <code>event_type</code> nodes and vice versa. </p> <p>The edges represent a measure of co-occurence between a <code>business_sector</code> and an 'event_type' in a dataset I have. For every business and event, there are two edges that represent two related measures of co-occurence. The background to these measures is a little involved so I'll leave it out unless someone thinks it pertinent to the question.</p> <p><strong>My Question</strong></p> <p>Assuming these measures make sense and provide us a good representation of how related different <code>business_sectors</code> are to different <code>event_types</code>, is it possible to use this network model to also determine how similar different <code>business_sectors</code> are based on the relationship each <code>business_sector</code> has with each <code>event_type</code>? The business sectors have no edges between them, but share neighbour nodes as all <code>business_sector</code> nodes connect to all <code>event_type</code> nodes, with varying weighted edges. Is it possible to produce a measure that might allow us to say <code>business_sector</code> a is similar to <code>business_sector</code> b because they have a similar connectivity to all the <code>event_types</code>.</p> <p>I may not be using the correct vocabulary here but I hope my description is enough. Any input on possible measures or approaches (I'm using Networkx and Gephi but any input on the issue is very welcome.)</p> <p>Thank you!</p> <p><strong>Note:</strong></p> <p>I have attempted to measure this outside of graph theory by making every <code>business_sector</code> an observation, and the weight of each sector to each <code>event_type</code> a variable and then calculating pairwise cosine distance between each sector, but this doesn't appear to be producing valid results.</p> https://stats.stackexchange.com/q/386063 0 How does Kamada & Kawai Force Directed layout algorithm interpret negative edge weights? O.rka https://stats.stackexchange.com/users/92493 2019-01-07T22:29:13Z 2019-01-07T22:29:13Z <p>In Cytoscape, the Kamada &amp; Kawai Force Directed layout algorithm is able to take negative weights between edges. Does anyone know how it's actually using the sign of these? Is it forcing everything into a positive space or does it actually use the sign somehow? </p> <p><a href="https://en.wikipedia.org/wiki/Force-directed_graph_drawing" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Force-directed_graph_drawing</a></p> <p><a href="http://manual.cytoscape.org/en/stable/Navigation_and_Layout.html#edge-weighted-spring-embedded-layout" rel="nofollow noreferrer">http://manual.cytoscape.org/en/stable/Navigation_and_Layout.html#edge-weighted-spring-embedded-layout</a></p> https://stats.stackexchange.com/q/383962 -1 How to cluster a (directional) dissimilarity matrix with both positive and negative values? O.rka https://stats.stackexchange.com/users/92493 2018-12-20T19:01:54Z 2018-12-21T00:28:11Z <p>I may be thinking of this incorrectly but what would be the best way to cluster a dissimilarity measure that has direction?</p> <p>For example, if someone had <code>condition A</code> and <code>condition B</code> each represented by a <code>(n x n)</code> matrix with <code>diagonal=1</code>. A comparative measure could be <code>condition A</code> - <code>condition B</code> which would result in both positive and negative values [-1,1]. This would still be a dissimilarity measure of some sort because values that barely change between the 2 conditions will have smaller values while larger magnitude values would be a greater change. However, the directionality is also important because a positive value would indicate a higher value in <code>condition A</code> relative to <code>B</code> while a negative would be higher in <code>B</code> relative to <code>A</code>. </p> <p><strong>How could one work with this type of structure? Is it possible to cluster this type of data without ignoring the directionality (i.e. instead of absolute value)?</strong> </p> <p>Below is a simple example using the iris dataset. <code>X_iris</code> is a <code>(150,4)</code> dataframe with <code>3 classes</code> representing <code>species</code>. <code>y_iris</code> is a vector with species labels. </p> <pre><code># Get "conditions" # ================ # Condition A # ----------- df_setosa = X_iris.loc[y_iris.compress(lambda id_species:id_species == "setosa").index] # print(df_setosa.head()) # sepal_length sepal_width petal_length petal_width # iris_0 5.1 3.5 1.4 0.2 # iris_1 4.9 3.0 1.4 0.2 # iris_2 4.7 3.2 1.3 0.2 # iris_3 4.6 3.1 1.5 0.2 # iris_4 5.0 3.6 1.4 0.2 # ----------- # Condition B # ----------- df_versicolor = X_iris.loc[y_iris.compress(lambda id_species:id_species == "versicolor").index] # print(df_versicolor.head()) # sepal_length sepal_width petal_length petal_width # iris_50 7.0 3.2 4.7 1.4 # iris_51 6.4 3.2 4.5 1.5 # iris_52 6.9 3.1 4.9 1.5 # iris_53 5.5 2.3 4.0 1.3 # iris_54 6.5 2.8 4.6 1.5 # Get similarity measure # ====================== # Condition A # ----------- df_setosa_similarity = df_setosa.corr() # print(df_setosa_similarity) # sepal_length sepal_width petal_length petal_width # sepal_length 1.000000 0.742547 0.267176 0.278098 # sepal_width 0.742547 1.000000 0.177700 0.232752 # petal_length 0.267176 0.177700 1.000000 0.331630 # petal_width 0.278098 0.232752 0.331630 1.000000 # ----------- # Condition B # ----------- df_versicolor_similarity = df_versicolor.corr() # print(df_versicolor_similarity) # sepal_length sepal_width petal_length petal_width # sepal_length 1.000000 0.525911 0.754049 0.546461 # sepal_width 0.525911 1.000000 0.560522 0.663999 # petal_length 0.754049 0.560522 1.000000 0.786668 # petal_width 0.546461 0.663999 0.786668 1.000000 # ---------- # Comarative # ---------- df_comparative = df_setosa_similarity - df_versicolor_similarity # print(df_comparative) # sepal_length sepal_width petal_length petal_width # sepal_length 0.000000 0.216636 -0.486873 -0.268363 # sepal_width 0.216636 0.000000 -0.382822 -0.431247 # petal_length -0.486873 -0.382822 0.000000 -0.455038 # petal_width -0.268363 -0.431247 -0.455038 0.000000 </code></pre> https://stats.stackexchange.com/q/380858 0 Predict node attribute from network variables NBK https://stats.stackexchange.com/users/192696 2018-12-07T15:46:13Z 2018-12-10T23:26:47Z <p>ERGM models the probability of a tie forming in a network. Is there a way of using ERGM, or an equivalent model, where the response variable is an attribute of the node, not a tie? Basically, turning ERGM around?</p> https://stats.stackexchange.com/q/380136 0 How to interpret networks with multiple states (e.g. timeseries, conditions, etc.) that have same node set? O.rka https://stats.stackexchange.com/users/92493 2018-12-03T19:06:59Z 2018-12-03T19:06:59Z <p>Apologies if this is too general but I have been thinking about it all weekend and wasn't sure how to move forward with the idea. </p> <p>Here are the types of networks I am dealing with below:</p> <ul> <li><p>I have 2 networks: (<code>state_1</code>) from a control population; and (<code>state_2</code>) from a conditional population.</p></li> <li><p>Each network has the same nodes with the only difference being the edge weights between the nodes. </p></li> <li>Each network has self-loops for each node (e.g. <code>A &lt;-0.5-&gt; A</code>)</li> <li>Each network is fully-connected</li> <li>I have "transition" weights that are <code>+</code> and <code>-</code> which are essentially the difference between <code>state_2 (conditional) - state_1 (control)</code></li> </ul> <p>I am confused on how to approach this situation in terms of constructing the comparative/transitional network. I felt the closest analog would be networks from time series data but the biggest difference between time series networks and my situation would be that transitions between the time states would be directional (e.g. t_0 -> t_1 -> ... -> t_final) whereas my states are categorical.</p> <p>My question: <strong>What type of network can I implement to accurately visualize, use graph algorithms (e.g. pagerank), and naturally store the transition data between nodes within a state and between states?</strong> </p> <p>Here were my thoughts on implementation:</p> <p>(1) Duplicate the nodes and have the connections between nodes within the same network be undirected and then connections between states be directional depending on which network has the higher value. The problem with this is that I will now have 2 sets of redundant nodes when in reality they are actually the same node. Maybe this is the correct way to think about the graph but I wanted to ask the community if there is a better way to reduce the redundancy.</p> <p>(2) A Multigraph where I have multiple edges connecting the nodes. This seems like the most compact way but I'm not sure if many <a href="https://networkx.github.io/documentation/stable/reference/algorithms/index.html" rel="nofollow noreferrer">algorithms</a> can handle this type of architecture and I'm worried edges wouldn't be organized in the correct way with respect to the states. </p> <p>It seems like I'm making this overly complicated so I would appreciate it if someone could steer me down the right path. I feel that maybe someone familiar with timeseries has solved this problem?</p> <pre><code>import numpy as np import pandas as pd import networkx as nx import matplotlib.pyplot as plt # Create adjacency matrix def get_adjacency(number_of_nodes=3, random_state=0): A = np.random.RandomState(random_state).randint(0,100, size=number_of_nodes**2).reshape((number_of_nodes,number_of_nodes)) df_adj = pd.DataFrame((A + A.T)/2) df_adj.index = df_adj.columns = list("ABC") return df_adj # Adjacency to graph def adjacency_to_graph(name): df_adj = adjacencies[name] return nx.from_pandas_adjacency(df_adj, nx.Graph(name=name)) # Container for storing the adjacencies for the 2 graphs adjacencies = { "state_1":get_adjacency(random_state=1), "state_2":get_adjacency(random_state=2), } adjacencies["diff"] = adjacencies["state_2"] - adjacencies["state_1"] # print(adjacencies["state_1"]) # A B C # A 37.0 10.5 75.5 # B 10.5 75.0 34.5 # C 75.5 34.5 16.0 # print(adjacencies["state_2"]) # A B C # A 40.0 18.5 73.5 # B 18.5 43.0 44.5 # C 73.5 44.5 34.0 # print(adjacencies["diff"]) # A B C # A 3.0 8.0 -2.0 # B 8.0 -32.0 10.0 # C -2.0 10.0 18.0 # Showing graphs with plt.style.context("seaborn-white"): fig, axes = plt.subplots(figsize=(8,3), ncols=2) for i, name in enumerate(["state_1", "state_2"]): graph = adjacency_to_graph(name) pos = nx.spring_layout(graph, seed=0) nx.draw(graph, pos=pos, ax=axes[i], with_labels=True, node_color="teal", node_size=610) axes[i].set_title(name, fontsize=15, fontweight="bold") </code></pre> <p><a href="https://i.stack.imgur.com/ecGJN.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ecGJN.png" alt="enter image description here"></a></p> https://stats.stackexchange.com/q/379641 0 Compare the degree distribution of a real networks and multiple random networks The Last Word https://stats.stackexchange.com/users/156257 2018-11-30T15:44:06Z 2018-12-03T01:33:18Z <p>Before the paper <a href="https://arxiv.org/abs/1801.03400" rel="nofollow noreferrer">real world networks are rarely scale free</a>, studies used to (still do) check for the slope in degree distribution graphs and if the slope fell between 2 and 3, discern that the network is a real world network. The slope of my networks are not between 2 and 3, and so I decided to conduct a Kolmogorov-Smirnov test (KS test) to check if there is a significant change in the degree distributions of my network and random networks (my network is bipartite and hence I used the sample.bipartite function in R to create the random networks) I created with the same number of nodes and edges. </p> <p>Eventhough I created a set of 500 random networks, KS test being a two sample test, I could only test the degree distribution of my network against one random network at a time. The problem is that the KS test gives me different answers (significant and not significant) for the same network when I compare it with different random network degree distributions from the 500 that I generated.</p> <p>Is there a possibility to compare the degree distribution of my real world network with the degree distribution of all 500 random networks rather than get differing answers comparing one at a time? I read that Kruskal- Wallace test might be able to do this but I don't know if that's true.</p> <p>I use graphpad PRISM for my stats analysis and my R is rusty, so please give me a test I can use on PRISM along with an example if possible. Thank you for your help in advance. </p> <p>Or am I doing the whole thing wrong and KS test cannot be used to compare the distributions between two networks. Any ideas are gladly welcomed. </p> https://stats.stackexchange.com/q/374567 2 shortest path optimization for multiple edge attributes jchaykow https://stats.stackexchange.com/users/91326 2018-10-31T05:09:47Z 2018-10-31T06:00:24Z <p>Say I have a network. The edges each have two attributes: age and height. How might I run a shortest path algorithm that optimizes on both age and height? And could I weight it so that it optimizes 0.7 on age and 0.3 on height, or vice versa?</p> https://stats.stackexchange.com/q/367753 0 Compare treatment effects across Levels of aggregation samplesize1 https://stats.stackexchange.com/users/109539 2018-09-20T04:11:39Z 2018-09-20T04:11:39Z <p>Suppose I am running an experiment to see if a treatment changes the <em>mean weight</em> of a group of people. Note that I am specifically interested in the <em>mean weight</em>: if half the people get heavier, and half get lighter, that is considered a null result. There of course may be some random drift, so I employ a control group:</p> <p>SETUP: </p> <p>For the treatment, people are organized into k groups with m subjects each, and I have measurements only of the mean: k independent measurements.</p> <p>For the control, people are also organized into k groups with m subjects each, but I have measurements on every individual. So now I have k*m independent measurements of height. However, this still produces only k independent measurements of my outcome, since I'm interested in the change in the mean, not the change in individuals. </p> <p>QUESTION: </p> <p>How can I compare the difference in the change in the mean height between the control and treatment, while taking advantage of the fact that I have more independent measurements for the control groups?</p> <p>We <em>know</em> the variance on the control measurement is lower, but I can't figure out how to actually take advantage of that in a test, bootstrapped or otherwise. The only thing I can do is measure two separate confidence intervals and see if they overlap, but that's not a proper way to do science.</p> <p>MOTIVATION:</p> <p>I actually have individual data for the treatment groups, but the people embedded in a social network, so they are not independent. I can't use nested/hierarchical/multi-level models because that still assumes independence at the lowest level.))</p> https://stats.stackexchange.com/q/365189 1 Centrality of a directed network with edge weights - Gephi user10046100 https://stats.stackexchange.com/users/219497 2018-09-03T11:37:49Z 2019-06-25T22:01:25Z <p>I have the following network of court judgments in the form of a big network.(50k+ nodes and 100k+ edges with weight values). The size is the number of citations. The idea is that court judgments work on precedents (<em>stare decisis</em>), they cite previous judgments. So the most important cases can be either </p> <ul> <li>The ones with most citations (ie in-degree centrality)</li> <li>The ones with high eigenvector centrality values</li> </ul> <p>But they don't take into account the fact that edges have weight. HITS or Pagerank also do not take into account - weighted edges. What is the best way to calculate centrality (ie the most important nodes) in the case of a directed network with weighted edges? Does Gephi have any inbuilt tools to do so?</p> <p><a href="https://i.stack.imgur.com/CMyv2.png" rel="nofollow noreferrer" title="network"><img src="https://i.stack.imgur.com/CMyv2.png" alt="my-network" title="network"></a></p> https://stats.stackexchange.com/q/364502 1 Assortativity coefficient in igraph Daniel https://stats.stackexchange.com/users/121516 2018-08-29T10:29:42Z 2019-03-21T03:03:25Z <p>In the context of Social Network Analysis, I want to compute the homogeneity of different networks for specific attributes. For this case the igraph package contains the method <strong>assortativity()</strong> to compute the assortativity coefficient for similar vertices - <a href="http://igraph.org/r/doc/assortativity.html" rel="nofollow noreferrer">http://igraph.org/r/doc/assortativity.html</a>. </p> <pre><code>assortativity(myGraph, V(myGraph)$class == 0, V(myGraph)$class == 1, directed=T) </code></pre> <p>Is it legit to compute the assortativity by setting the first attribute value of one class equal zero and the second attribute of the second class equal one? In that case I am not interesting in the similarity of one specific class but rather in the homopholy of different classes. Or is it only possible to compare different vertices (other features) with each other?</p> https://stats.stackexchange.com/q/364434 0 what's the theory foundation of the giant component strategy? F.caren https://stats.stackexchange.com/users/154949 2018-08-28T23:50:16Z 2018-09-03T11:27:47Z <p>could anyone plz let me know what's the theoretical foundation of the giant component strategy?</p> <p>I have used this technique to get the giant components of a not fully connected graph but i need the theory foundation of this strategy.</p> <p>I appreciate any help!</p> https://stats.stackexchange.com/q/363374 2 Computation of Network Homophily / Heterogeneity Daniel https://stats.stackexchange.com/users/121516 2018-08-22T08:47:50Z 2018-09-20T22:58:35Z <p>I am quite confused about the calculation of network homophily in network analysis. Right now I am computing the homophily using the following function, which has been written and also described by the following URL: <a href="http://dappls.umasscreate.net/networks/calculating-network-homophily-part-1/" rel="nofollow noreferrer">http://dappls.umasscreate.net/networks/calculating-network-homophily-part-1/</a>. Well the definition of homophily in social science is "the tendency of individuals to associate and bond with similar others". In network analysis homophily is described as a process where similar nodes on a particular trait are more likely to form ties, which is quite the same as in social science right? My goal is to measure the homophily in a directed network to identify which group of actors are similar to each other.</p> <h3>Function</h3> <pre><code>homophily &lt;- function(graph,vertex.attr,attr.val=NULL,prop=T){ #Assign names as vertex attributes for edgelist output# V(graph)$name&lt;-vertex_attr(graph,vertex.attr) #Get the basic edgelist# ee&lt;-get.data.frame(graph) #If not specifying on particular attribute value, get percentage (prop=T)# #or count (prop=F) of all nodes tied with matching attribute# if(is.null(attr.val)){ ifelse(prop==T,sum(ee[,1]==ee[,2])/nrow(ee),sum(ee[,1]==ee[,2])) #If not null, get proportion (prop=T) or count (prop=F) of# #edges among nodes with that particular node attribute value# } else { ifelse(prop==T,sum(ee[,1]==attr.val &amp; ee[,2]==attr.val)/nrow(ee[ee[,1]==attr.val|ee[,2]==attr.val,]), sum(ee[,1]==attr.val &amp; ee[,2]==attr.val)) } } </code></pre> <h3>Sample Data</h3> <pre><code>set.seed(5165) #Random directed graph with 100 nodes and 30% chance of a tie# gg&lt;-random.graph.game(100,0.3,"gnp",directed=T) #Randomly assign the node attribute (group numbers 0:3)# V(gg)$group&lt;-sample(1:5,100,replace=T) </code></pre> <h3>Output</h3> By applying the function on sample data I receive the following output, which means that 20% of the ties in the network are between actors in the same group. It is also possible to compute the homophily for a specific group in percentage. <pre><code>homophily(graph = abc, vertex.attr = "group")  0.1971504 </code></pre> However I also noticed that the <strong>igraph</strong> package contains as well a homophily method called <code>assortativity()</code> <a href="http://igraph.org/r/doc/assortativity.html" rel="nofollow noreferrer">described here</a>. Executing this function gives completely different results, with the assortativity coefficient in a range(-1, 1). The assortativity coefficient is positive if similar vertices (based on some external property) tend to connect to each other, and negative otherwise. <pre><code>library(igraph) assortativity(abc, V(abc)$group, directed=T)  -0.02653782 </code></pre> <h3>Question</h3> <p>So right now I am quite confused, which of these methods is the right one to measure the homophily in a network, because both functions received different results. I also noticed that the igraph method does not support the calculation of particular groups. In my opinion I would rather go with the first one which is self-coded (not sure if there are some mistakes), because the interpretation makes more sense. So my question is, which of the following methods is the right one for measuring the homophily in a network? I mean if I want to know how heterogeneous or homogenous actors in a network communicate, I would rather choose the first technique. Both techniques measure homophily &amp; receive different results, but right know I can not see any difference (advantage) which one will be used for any reason. The goal of this technique is to measure the homophily in a network by their proportion of all edges in a network.</p> https://stats.stackexchange.com/q/362634 0 Investigating the change of a network / factorial structure over time Dominique Makowski https://stats.stackexchange.com/users/102345 2018-08-17T08:36:48Z 2018-08-17T18:08:53Z <p>I have several variables (questions from a questionnaire) that regroup into several factors (using factor analysis). However, I would be interested in knowing how this factorial structure changes depending on another numeric variable (in my case, time). </p> <p>From what I understood, SEM would allow me to estimate the influence of this "modulator" directly on the latent variables. But I am not sure I can estimate its influence on the link between variables itself. </p> <p>What statistical method could help me addressing my question? Thanks</p> https://stats.stackexchange.com/q/358221 2 How to measure "cyclicity" of a directed weighted graph? ampanmdagaba https://stats.stackexchange.com/users/125294 2018-07-20T17:47:09Z 2018-07-20T17:47:09Z <p>Say you have a weighted directed graph with (potentially) some cycles in it. You want to have some sort of a measure of how "cyclical" this graph is. The requirements are:</p> <ol> <li>This measure C=0 on an acyclical graph</li> <li>C=1 on a fully connected graph</li> <li>C monotonously decreases as you eliminate edges from a fully connected graph</li> <li>Should generalize well to unweighted directed graphs</li> <li>In my case, graph weights represent connections between nodes, not distances, so high weights make good cycles. It means that a low weight within a cycle should act as "bottleneck", lowering the "impact" of this cycle on C.</li> </ol> <p>Beyond that the measure is really flexible. </p> <p><strong>How to best build a cyclicity measure like that?</strong></p> <p>So far the best I could do is some sort of a free-association on spectral techniques, random walks, and stable flows, wherein I inject a flow of 1 into each node, make them propagate through the graph with some decay, and look for a a stable flow solution. Then I sum all flows that originate in each node and cycle back to this node. Here's an attempt to write it down mathematically, in case it's easier to read that way:</p> <p>I initialize$s_0 = E$, where$E$is a unitary matrix. I then run a stepwise equation$s_{i+1} = d\cdot A'\cdot\text{max}(s_i,E)$, where$A$is my adjacency matrix,$d$is a dampening coefficient of about 0.9 (pagerank-style), and$\text{max}$operates on both matrices elementwise. I do it until$s$reaches a stable solution; then I assume$C=\text{Tr}(s_{end})$.</p> <p>The problem here of course is that A needs to be normalized somehow, so that the solution would converge, yet I should not completely invalidate the values of weights, as I want it to work on weighted graphs. For now I solve it in the following way: I normalize A so that the highest in-degree (sum of in-weights) on the graph is equal to 1. Then I run this analysis on my graph, and also on a full graph of the same dimension. And then I normalize C achieved on my graph to C on a full graph. Essentially the measure I've built tells me the share of self-flow (cyclical flow) in my graph of interest, compared to a full graph.</p> <p>It sort of works, for my purposes, but I have three concerns. One is that it feels unnecessary complex. I am particularly concerned with the fact that I have to normalize things twice: first$A$, then$C$itself. Two, it feels suspiciously similar to spectral analysis, so I was wondering whether this problem is already solved long time ago, and I just don't know the solution, or fail to recognize it. Three, the normalization of$A$I perform, namely$A \rightarrow A/\text{max}(d_i) = A/\text{max}(\sum_i a_{ij})$, feels a bit random, and I'm concerned that it can cause "oscillations" in violation of my criterion 2 (monotonous change in C with graph reduction). I could not catch them by debugging so far, but it feels dangerous.</p> <p>The only alternative to this normalization I have is to make$A \rightarrow A/[\text{max}(a_{ij})\cdot n]$, where$n$is the dimension of$A\$. It also works, but values of C become astronomically small very quickly, which is uncomfortable as well.</p> <p>Sorry for a long-winded question, and I'll be extremely grateful for any suggestions or thoughts on this topic! Thanks!</p>