Skip to content

Add PPR relation token features to graph transformer - #700

Draft
mkolodner-sc wants to merge 3 commits into
mainfrom
mkolodner/gt-ppr-relation-features
Draft

Add PPR relation token features to graph transformer#700
mkolodner-sc wants to merge 3 commits into
mainfrom
mkolodner/gt-ppr-relation-features

Conversation

@mkolodner-sc

@mkolodner-sc mkolodner-sc commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds Graph Transformer support for typed-PPR edge attributes.

Typed PPR can emit multi-column edge_attr tensors where column 0 is the scalar PPR weight and the remaining columns describe relation/channel metadata. This PR lets Graph Transformer consume those extra columns as token-input features while preserving the existing scalar PPR path.

What This Enables

A PPR edge can now carry:

  • ppr_weight: the first edge_attr column, used as the scalar PPR score.
  • ppr_relation_features: any remaining edge_attr columns, projected into the token embedding as continuous token features.

This allows typed-PPR metadata, such as channel scores or channel presence bits, to flow into Graph Transformer tokens without changing how scalar PPR batches behave.

Changes

  • Adds reserved feature name ppr_relation_features.
  • Supports scalar and multi-column PPR edge_attr.
  • Keeps ppr_weight mapped to edge_attr[..., 0].
  • Maps ppr_relation_features to edge_attr[..., 1:].
  • Preserves multi-column PPR edge order as emitted by the sampler, so feature rows stay aligned with sampler-ranked tokens.
  • Keeps scalar PPR behavior unchanged, including descending PPR-weight sorting.
  • Validates that ppr_relation_features is used only as token input, not attention bias.
  • Updates token-input handling to support multi-column continuous features.
  • Adds transform and encoder tests for relation-feature extraction, ordering, and forward-pass support.

Toy Example

Suppose typed PPR emits PPR edges from a user anchor to item tokens.

Each PPR edge represents one selected (anchor, token) pair in the sampled PPR sequence. The edge_attr row for that edge describes how typed PPR reached/scored that token:

  • Column 0 is the scalar PPR weight used for ranking/weight-style features.
  • Columns 1..num_channels are per-channel calibrated PPR scores.
  • Columns 1 + num_channels.. are per-channel presence bits indicating which typed channels reached the token.

For example, with two typed PPR channels:

data["user", "ppr", "item"].edge_index = torch.tensor([
    [0, 0, 0],
    [7, 3, 9],
])

data["user", "ppr", "item"].edge_attr = torch.tensor([
    # best_ppr, channel_0_score, channel_1_score, channel_0_present, channel_1_present
    [0.90,     0.90,            0.00,            1.0,               0.0],
    [0.75,     0.10,            0.75,            1.0,               1.0],
    [0.40,     0.00,            0.40,            0.0,               1.0],
])

Graph Transformer interprets this as:

ppr_weight = edge_attr[:, 0:1]
ppr_relation_features = edge_attr[:, 1:]

A model can consume both as token-input features:

encoder = GraphTransformerEncoder(
    ...,
    sequence_construction_method="ppr",
    anchor_based_input_attr_names=[
        "ppr_weight",
        "ppr_relation_features",
    ],
)

For the example above, each token receives:

ppr_weight = [
    [0.90],
    [0.75],
    [0.40],
]

ppr_relation_features = [
    [0.90, 0.00, 1.0, 0.0],
    [0.10, 0.75, 1.0, 1.0],
    [0.00, 0.40, 0.0, 1.0],
]

These continuous features are concatenated, passed through the learned token-input projection, and added to the corresponding token embeddings before the Transformer layers run.

Validation

  • Added focused unit coverage for multi-column PPR edge attrs and Graph Transformer token-input consumption.
  • Ran focused ruff check on the changed Graph Transformer files and tests.

@mkolodner-sc
mkolodner-sc force-pushed the mkolodner/gt-ppr-relation-features branch from 7fb45ce to 8ccc1a0 Compare July 28, 2026 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant