HigherOrderGraph / modified MultiOrderModel

docs/tutorial/advanced/01-07-message-passing-pyG.ipynb

@@ -445,7 +445,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Data(edge_index=[2, 12], num_nodes=5, node_sequence=[5, 1], x=[5, 1], y=[5, 2])\n",
+      "Data(edge_index=[2, 12], num_nodes=5, x=[5, 1], y=[5, 2])\n",
       "tensor([[1.],\n",
       "        [2.],\n",
       "        [3.],\n",

docs/tutorial/advanced/01-08-GCNs.ipynb

@@ -555,7 +555,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Data(edge_index=[2, 800], num_nodes=100, node_sequence=[100, 1], y=[100, 1], x=[100, 100], train_mask=[100], val_mask=[100], test_mask=[100])\n"
+      "Data(edge_index=[2, 800], num_nodes=100, y=[100, 1], x=[100, 100], train_mask=[100], val_mask=[100], test_mask=[100])\n"
      ]
     }
    ],
@@ -1392,7 +1392,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Data(edge_index=[2, 2390], num_nodes=200, node_sequence=[200, 1], y=[200, 1], x=[200, 200])\n"
+      "Data(edge_index=[2, 2390], num_nodes=200, y=[200, 1], x=[200, 200])\n"
      ]
     }
    ],

docs/tutorial/advanced/01-09-GCN-node-features.ipynb

@@ -191,7 +191,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Data(edge_index=[2, 800], num_nodes=100, node_sequence=[100, 1], x=[100, 101], cluster=[100], y=[100, 2], train_mask=[100], val_mask=[100], test_mask=[100])\n"
+      "Data(edge_index=[2, 800], num_nodes=100, x=[100, 101], cluster=[100], y=[100, 2], train_mask=[100], val_mask=[100], test_mask=[100])\n"
      ]
     }
    ],

docs/tutorial/advanced/02-02-trp-centralities-communities.ipynb

@@ -3840,7 +3840,7 @@
    "source": [
     "## Visualizing Temporal Communities\n",
     "\n",
-    "Time-respecting paths also let us uncover **temporal communities**: groups of nodes that are connected by many time-respecting paths, even in cases where this pattern is not visible in the topology of the time-aggregated graph. This is exactly the same phenomenon that motivates the higher-order De Bruijn graph models used in the previous notebooks, but here we take a complementary, simulation-based perspective.\n",
+    "Time-respecting paths also let us uncover **temporal communities**: groups of nodes that are connected by many time-respecting paths, even in cases where this pattern is not visible in the topology of the time-aggregated graph. This is exactly the same phenomenon that motivates the higher-order De Bruijn graph models that we introduce in the following notebooks, but here we take a complementary, simulation-based perspective.\n",
     "\n",
     "We load a synthetic temporal graph with 30 nodes and 60,000 time-stamped interactions, which was generated with a planted community structure (nodes 0-9, 10-19, and 20-29 each form a temporal community). Note that the CSV file uses the column names `source`, `target`, `time` instead of the `v`, `w`, `t` expected by `pp.io.read_csv_temporal_graph`, so we read it with `pandas` and rename the columns ourselves before converting it to a `TemporalGraph`:"
    ]

docs/tutorial/advanced/02-03-trp-event-graphs.ipynb

@@ -17,9 +17,9 @@
    "source": [
     "## Motivation\n",
     "\n",
-    "In the previous tutorial, we have seen how we can use higher-order models to model paths in complex networks. In this example, paths were directly given in terms of sequences of nodes traversed by some process (like a random walk). We have further seen that higher-order De Bruijn graph models can be used to capture patterns that influence the **causal topology** of a complex network, i.e. which nodes can possibly influence each other via paths. The same is true for time-respecting paths in a temporal graph. Due to the fact that time-stamped edges need to occur in the correct temporal ordering (and within a given time interval based on the maximum time difference $\\delta$), the causal topology given by time-respecting paths can be very different from what we would expect from the (static) topology of links.\n",
+    "In the tutorial on paths and higher-order models, we have seen how we can use higher-order models to model paths in complex networks. In this example, paths were directly given in terms of sequences of nodes traversed by some process (like a random walk). We have further seen that higher-order De Bruijn graph models can be used to capture patterns that influence the **causal topology** of a complex network, i.e. which nodes can possibly influence each other via paths. The same is true for time-respecting paths in a temporal graph. Due to the fact that time-stamped edges need to occur in the correct temporal ordering (and within a given time interval based on the maximum time difference $\\delta$), the causal topology given by time-respecting paths can be very different from what we would expect from the (static) topology of links.\n",
     "\n",
-    "In the following, we will show how we can easiy and efficiently construct higher-order models for time-respecting paths in a temporal graph. To illustrate this, we use the same toy example as before:"
+    "In the following, we will show how we can easily and efficiently construct higher-order models for time-respecting paths in a temporal graph. To illustrate this, we use the same toy example as before:"
    ]
   },
   {
@@ -400,7 +400,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "As you can see, these time-respecting paths are actually very similar to the paths data that we have previously represented using the `PathData` object. In fact, we could - in theory - first extract all time-respecting paths of all lengths, add them to a `PathData` object and then use the `MultiOderModel` class to generate higher-order De Bruijn graph models of all orders. In the example above, since we have paths of length one to four, we could create higher-order models with orders from one to four. \n",
+    "As you can see, these time-respecting paths are actually very similar to the paths data that we have previously represented using the `PathData` object. In fact, we could - in theory - first extract all time-respecting paths of all lengths, add them to a `PathData` object and then use the `MultiOrderModel` class to generate higher-order De Bruijn graph models of all orders. In the example above, since we have paths of length one to four, we could create higher-order models with orders from one to four. \n",
     "\n",
     "However, this approach would not be efficient for large temporal graphs, as it is computationally expensive to calculate all possible time-respecting paths as well as subpaths of length $k$, especially for larger values of $\\delta$. To avoid this bottleneck, `pathpyG` uses a smarter, GPU-based algorithm to calculate time-respecting paths of length $k$ that are needed for a given order $k$.\n",
     "\n",
@@ -423,11 +423,13 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Directed graph with 6 nodes and 4 edges\n",
+      "Higher-order graph of order 3 with 6 nodes and 4 edges\n",
+      "(over 4 first-order nodes)\n",
       "{   'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([4])\"},\n",
       "    'Graph Attributes': {'inverse_idx': \"<class 'torch.Tensor'> -> torch.Size([7])\", 'num_nodes': \"<class 'int'>\"},\n",
       "    'Node Attributes': {}}\n",
-      "Directed graph with 4 nodes and 2 edges\n",
+      "Higher-order graph of order 4 with 4 nodes and 2 edges\n",
+      "(over 4 first-order nodes)\n",
       "{'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([2])\"}, 'Graph Attributes': {'num_nodes': \"<class 'int'>\"}, 'Node Attributes': {}}\n"
      ]
     },
@@ -464,7 +466,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Directed graph with 4 nodes and 6 edges\n",
+      "Higher-order graph of order 1 with 4 nodes and 6 edges\n",
+      "(over 4 first-order nodes)\n",
       "{'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([6])\"}, 'Graph Attributes': {'num_nodes': \"<class 'int'>\"}, 'Node Attributes': {}}\n"
      ]
     },
@@ -769,7 +772,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Directed graph with 6 nodes and 6 edges\n",
+      "Higher-order graph of order 2 with 6 nodes and 6 edges\n",
+      "(over 4 first-order nodes)\n",
       "{   'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([6])\"},\n",
       "    'Graph Attributes': {'inverse_idx': \"<class 'torch.Tensor'> -> torch.Size([10])\", 'num_nodes': \"<class 'int'>\"},\n",
       "    'Node Attributes': {}}\n",
@@ -1078,7 +1082,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Directed graph with 6 nodes and 4 edges\n",
+      "Higher-order graph of order 3 with 6 nodes and 4 edges\n",
+      "(over 4 first-order nodes)\n",
       "{   'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([4])\"},\n",
       "    'Graph Attributes': {'inverse_idx': \"<class 'torch.Tensor'> -> torch.Size([7])\", 'num_nodes': \"<class 'int'>\"},\n",
       "    'Node Attributes': {}}\n"
@@ -1385,7 +1390,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Directed graph with 4 nodes and 2 edges\n",
+      "Higher-order graph of order 4 with 4 nodes and 2 edges\n",
+      "(over 4 first-order nodes)\n",
       "{'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([2])\"}, 'Graph Attributes': {'num_nodes': \"<class 'int'>\"}, 'Node Attributes': {}}\n"
      ]
     },
@@ -1678,7 +1684,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Intuitively, since in our example there are no time-respecting paths longer than four, if we were to generate a multi-order model with De Bruijn graphs with orders larger than four, those graphs cannot contain any edges. We see this in the following example. The first-order graph is simply the time-aggregated weighted graph, i.e. the number of nodes is equal to the number of nodes in the temporal graph and the number of edges is equal to the number of different time-stamped edges. In each graph of order $k>1$, the number of nodes corresponds to the number of edges in the graph with order $k-1$, since each of those nodes corresponds to a time-respecting path of length $k-1$, which are represented by edges in a $k-1$-th order gaph. This implies that the graph with order five has two nodes, which are the two time-respecting paths of length four. Those nodes are not connected since there is no time-respecting path with length five."
+    "Intuitively, since in our example there are no time-respecting paths longer than four, if we were to generate a multi-order model with De Bruijn graphs with orders larger than four, those graphs cannot contain any edges. We see this in the following example. The first-order graph is simply the time-aggregated weighted graph, i.e. the number of nodes is equal to the number of nodes in the temporal graph and the number of edges is equal to the number of different time-stamped edges. In each graph of order $k>1$, the number of nodes corresponds to the number of edges in the graph with order $k-1$, since each of those nodes corresponds to a time-respecting path of length $k-1$, which are represented by edges in a $k-1$-th order graph. This implies that the graph with order five has two nodes, which are the two time-respecting paths of length four. Those nodes are not connected since there is no time-respecting path with length five."
    ]
   },
   {
@@ -1697,9 +1703,11 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Directed graph with 4 nodes and 2 edges\n",
+      "Higher-order graph of order 4 with 4 nodes and 2 edges\n",
+      "(over 4 first-order nodes)\n",
       "{'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([2])\"}, 'Graph Attributes': {'num_nodes': \"<class 'int'>\"}, 'Node Attributes': {}}\n",
-      "Directed graph with 2 nodes and 0 edges\n",
+      "Higher-order graph of order 5 with 2 nodes and 0 edges\n",
+      "(over 4 first-order nodes)\n",
       "{'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([0])\"}, 'Graph Attributes': {'num_nodes': \"<class 'int'>\"}, 'Node Attributes': {}}\n"
      ]
     },
@@ -2004,7 +2012,7 @@
    "source": [
     "## Constructing Higher-Order De Bruijn Graph Models for Empirical Temporal Networks\n",
     "\n",
-    "Let us now use `pathpyG` to construcct higher-order De Bruijn graph models for time-respecting paths in empirical temporal network. For this, we first read a number of temporal graphs using `TemporalGraph.from_csv`. In the following, we use the following three publicly available data sets:\n",
+    "Let us now use `pathpyG` to construct higher-order De Bruijn graph models for time-respecting paths in empirical temporal networks. For this, we first read a number of temporal graphs using `TemporalGraph.from_csv`. In the following, we use the following three publicly available data sets:\n",
     "\n",
     "- Antenna interactions between ants in a colony [(Blonder and Dornhaus, 2011)](https://pubmed.ncbi.nlm.nih.gov/21625450/)  \n",
     "- E-Mail exchanges in a manufacturing company [(Nurek and Michalski, 2020)](https://www.ii.pwr.edu.pl/~michalski/index.php?content=datasets)  \n",
@@ -2073,17 +2081,21 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Directed graph with 89 nodes and 947 edges\n",
+      "Higher-order graph of order 1 with 89 nodes and 947 edges\n",
+      "(over 89 first-order nodes)\n",
       "{'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([947])\"}, 'Graph Attributes': {'num_nodes': \"<class 'int'>\"}, 'Node Attributes': {}}\n",
-      "Directed graph with 947 nodes and 1780 edges\n",
+      "Higher-order graph of order 2 with 947 nodes and 1780 edges\n",
+      "(over 89 first-order nodes)\n",
       "{   'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([1780])\"},\n",
       "    'Graph Attributes': {'inverse_idx': \"<class 'torch.Tensor'> -> torch.Size([1911])\", 'num_nodes': \"<class 'int'>\"},\n",
       "    'Node Attributes': {}}\n",
-      "Directed graph with 1780 nodes and 2410 edges\n",
+      "Higher-order graph of order 3 with 1780 nodes and 2410 edges\n",
+      "(over 89 first-order nodes)\n",
       "{   'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([2410])\"},\n",
       "    'Graph Attributes': {'inverse_idx': \"<class 'torch.Tensor'> -> torch.Size([2518])\", 'num_nodes': \"<class 'int'>\"},\n",
       "    'Node Attributes': {}}\n",
-      "Directed graph with 2410 nodes and 3292 edges\n",
+      "Higher-order graph of order 4 with 2410 nodes and 3292 edges\n",
+      "(over 89 first-order nodes)\n",
       "{   'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([3292])\"},\n",
       "    'Graph Attributes': {'inverse_idx': \"<class 'torch.Tensor'> -> torch.Size([3572])\", 'num_nodes': \"<class 'int'>\"},\n",
       "    'Node Attributes': {}}\n"
@@ -2126,28 +2138,14 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Directed graph with 327 nodes and 5818 edges\n",
-      "\n",
-      "Node attributes\n",
-      "\tnode_sequence\t\t<class 'torch.Tensor'> -> torch.Size([327, 1])\n",
-      "\n",
-      "Edge attributes\n",
-      "\tedge_weight\t\t<class 'torch.Tensor'> -> torch.Size([5818])\n",
-      "\n",
-      "Graph attributes\n",
-      "\tnum_nodes\t\t<class 'int'>\n",
-      "\n",
-      "Directed graph with 16307 nodes and 8712 edges\n",
-      "\n",
-      "Node attributes\n",
-      "\tnode_sequence\t\t<class 'torch.Tensor'> -> torch.Size([16307, 5])\n",
-      "\n",
-      "Edge attributes\n",
-      "\tedge_weight\t\t<class 'torch.Tensor'> -> torch.Size([8712])\n",
-      "\n",
-      "Graph attributes\n",
-      "\tnum_nodes\t\t<class 'int'>\n",
-      "\n"
+      "Higher-order graph of order 1 with 327 nodes and 5818 edges\n",
+      "(over 327 first-order nodes)\n",
+      "{'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([5818])\"}, 'Graph Attributes': {'num_nodes': \"<class 'int'>\"}, 'Node Attributes': {}}\n",
+      "Higher-order graph of order 5 with 24550 nodes and 17433 edges\n",
+      "(over 327 first-order nodes)\n",
+      "{   'Edge Attributes': {'edge_weight': \"<class 'torch.Tensor'> -> torch.Size([17433])\"},\n",
+      "    'Graph Attributes': {'inverse_idx': \"<class 'torch.Tensor'> -> torch.Size([12314285])\", 'num_nodes': \"<class 'int'>\"},\n",
+      "    'Node Attributes': {}}\n"
      ]
     }
    ],

docs/tutorial/advanced/02-05-tgn-baseline.ipynb

@@ -373,9 +373,9 @@
    "source": [
     "## Conclusion\n",
     "\n",
-    "Both checks tell the same story: the mean within-community and cross-community distances are essentially identical, the Adjusted Rand Index is close to zero (no better than a random clustering), and the visualization shows the three communities scattered on top of each other rather than forming separated clusters. Unlike the causality-aware DBGNN model from the previous notebook, TGN **fails to recover the planted temporal communities** in this data set, despite being trained on the exact same interactions.\n",
+    "Both checks tell the same story: the mean within-community and cross-community distances are essentially identical, the Adjusted Rand Index is close to zero (no better than a random clustering), and the visualization shows the three communities scattered on top of each other rather than forming separated clusters. TGN **fails to recover the planted temporal communities** in this data set, despite being trained on the exact same interactions.\n",
     "\n",
-    "The reason is architectural: TGN's memory update and its graph attention embedding only ever look at a node's own history and its most recent, *direct* neighbors. It has no mechanism to explicitly aggregate information along longer chains of time-respecting interactions that connect nodes several hops apart in time. Since the community signal in this data set is encoded purely in such longer time-respecting paths (recall that the static, time-aggregated topology is statistically indistinguishable from a random graph), a model that only reasons locally and pairwise, however well it models the timing of interactions, simply has no access to the relevant signal. This supports the conclusion that DBGNN's success in the previous notebook is not just a matter of being *temporally aware* in some general sense, but specifically relies on its explicit, higher-order modelling of time-respecting paths via De Bruijn graphs."
+    "The reason is architectural: TGN's memory update and its graph attention embedding only ever look at a node's own history and its most recent, *direct* neighbors. It has no mechanism to explicitly aggregate information along longer chains of time-respecting interactions that connect nodes several hops apart in time. Since the community signal in this data set is encoded purely in such longer time-respecting paths (recall that the static, time-aggregated topology is statistically indistinguishable from a random graph), a model that only reasons locally and pairwise, however well it models the timing of interactions, simply has no access to the relevant signal. In the next notebook, we will see that the causality-aware DBGNN model does recover the communities from the same interactions. This supports the conclusion that DBGNN's success is not just a matter of being *temporally aware* in some general sense, but specifically relies on its explicit, higher-order modelling of time-respecting paths via De Bruijn graphs."
    ]
   }
  ],