We are excited to announce the release of PyG 2.6 🎉🎉🎉
- Highlights
- Breaking Changes
- Features
- Bugfixes
- Changes
- Full Changelog
PyG 2.6 is the culmination of work from 59 contributors who have worked on features and bug-fixes for a total of over 238 commits since torch-geometric==2.5.0
.
Highlights
PyTorch 2.4 Support
PyG 2.6 is fully compatible with PyTorch 2.4, and supports the following combinations:
PyTorch 2.2 | cpu
| cu118
| cu121
| cu124
|
---|---|---|---|---|
Linux | ✅ | ✅ | ✅ | ✅ |
macOS | ✅ | |||
Windows | ✅ | ✅ | ✅ | ✅ |
You can still install PyG 2.6 with an older PyTorch release up to PyTorch 1.13 in case you are not eager to update your PyTorch version.
GNNs+LLMs
In order to facilitate further research on combining GNNs with LLMs, PyG 2.6 introduces
- a new sub-package
torch_geometric.nn.nlp
with fast access toSentenceTransformer
models and LLMs - a new model
GRetriever
that is able to co-trainLLAMA2
withGAT
for answering questions based on knowledge graph information - a new example folder
examples/llm
that shows how to utilize these models in practice
Index
Tensor Representation
Similar to the EdgeIndex
class introduced in PyG 2.5, torch-geometric==2.6.0
introduces the Index
class for efficient storage of 1D indices. While Index
sub-classes a general torch.Tensor
, it can hold additional (meta)data, i.e.:
dim_size
: The size of the underlying sparse vector, i.e. the size of a dimension that can be indexed viaIndex
. By default, it is inferred asdim_size=index.max() + 1
is_sorted
: Whether indices are sorted in ascending order.
Additionally, Index
caches data via indptr
for fast CSR conversion in case its representation is sorted. Caches are filled based on demand (e.g., when calling Index.get_indptr()
or when explicitly requested via Index.fill_cache_()
, and are maintained and adjusted over its lifespan.
from torch_geometric import Index
index = Index([0, 1, 1, 2], dim_size=3, is_sorted=True)
assert index.dim_size == 3
assert index.is_sorted
# Flipping order:
index.flip(0)
assert not index.is_sorted
# Filtering:
mask = torch.tensor([True, True, True, False])
index[:, mask]
assert index.is_sorted
EdgeIndex
and Index
will interact seamlessly together, e.g., edge_index[0]
will now return a Index
instance.
This ensures optimal computation in GNN message passing schemes, while preserving the ease-of-use of regular COO-based PyG workflows. EdgeIndex
and Index
will fully deprecate the usage of SparseTensor
from torch-sparse
in later releases, leaving us with just a single source of truth for representing graph structure information in PyG.
Breaking Changes
- Allow
None
outputs inFeatureStore.get_tensor()
-KeyError
should now be raised based on the implementation inFeatureStore._get_tensor()
(#9102) cugraph
-based GNN layers such asCuGraphSAGEConv
now expectEdgeIndex
-based inputs (#8938)
Features
Examples
- Added a multi-GPU example for training GNNs on the PCQM4M graph-level regression task (#9070)
- Added a multi-GPU
ogbn-mag240m
example (#8249) - Added support for
cugraph
data loading capabilities in thepapers100m
examples (#8173) - Improved the hyper-parameters of the [single-node](ogbn-papers100m example
](https://github.com/pyg-team/pytorch_geometric/blob/master/examples/ogbn_papers_100m.py) and [multi-node](https://github.com/pyg-team/pytorch_geometric/blob/master/examples/multi_gpu/papers100m_gcn_cugraph_multinode.py)
ogbn-papers100m examples, and added evaluation on all ranks (#8823, #9386, #9445)
EdgeIndex
and Index
- Added
torch_geometric.Index
(#9276, #9277, #9278, #9279, #9280, #9281, #9284, #9285, #9286, #9287, #9288, #9289, #9296, #9297) - Added support for
EdgeIndex
inMessagePassing
(#9007, #9026, #9131) - Added support for
torch.compile
in combination withEdgeIndex
(#9007) -
- Added support for
EdgeIndex.unbind()
(#9298)
- Added support for
- Added support for
EdgeIndex.sparse_narrow()
(#9291) - Added support for
EdgeIndex.sparse_resize_()
(#8983)
torch_geometric.nn
- Added the
GRetriever
model (#9480) - Added the
ClusterPooling
layer (#9627) - Added the
PatchTransformerAggregation
layer (#9487) - Added a
residual
option inGATConv
andGATv2Conv
(#9515) - Added a
nlp.LLM
model wrapper (#9462) - Added a
nlp.SentenceTransformer
model wrapper (#9350) - Added the heterogeneous
HeteroJumpingKnowledge
module for applying jumping knowledge in heterogeneous graphs (#9380) - Added the
VariancePreservingAggregation
layer (#9075) - Added approximate
faiss
-based KNN-search capabilities viaApproxKNN
(#8952, #9046)
torch_geometric.metrics
- Added the
LinkPredMRR
metric (#9632)
torch_geometric.transforms
- Added the
RemoveSelfLoops
transformation (#9562)
torch_geometric.utils
- Added
normalize_edge_index()
for symmetric/asymmetric normalization of graph edges (#9554) - Added
from_rdmol
/to_rdmol
functionality (#9452) - Added ONNX export for
scatter
with min/max reductions (#9587)
torch_geometric.datasets
- Added the
WebQSPDataset
(#9481) - Added the
OPFDataset
(#9379) - Added
CornellTemporalHyperGraphDataset
hypergraph dataset (#9090) - Added option to pass custom
from_smiles
functionality toPCQM4Mv2
andMoleculeNet
(#9073)
torch_geometric.loader
- Added support for negative sampling in
LinkLoader
acccording to source and destination node weights (#9316)
Bugfixes
- Fixed
VirtualNode
transform for empty edge indices (#9605) - Fixed an issue where the import order in the multi-GPU
cugraph
example could cause anrmm
error (#9577) - Fixed
load_state_dict
behavior with lazy parameters inHeteroDictLinear
(#9493) Sequential
modules can now be properly pickled (#9369)- Fixed
pickle.load
for jittableMessagePassing
modules (#9368) - Fixed batching of sparse tensors saved via
data.edge_index
(#9317) - Fixed arbitrary keyword ordering in
MessagePassing.propagate()
(#9245) - Fixed the node mapping in the
RCDD
dataset (#9234) - Fixed incorrect treatment of
edge_label
andedge_label_index
inToSparseTensor
transform (#9199) - Fixed
EgoData
processing inSnapDataset
in case filenames are unsorted (#9195) - Fixed empty graph and isolated node handling in
to_dgl()
function (#9188) - Fixed bug in
to_scipy_sparse_matrix()
when CUDA is set as default torch device (#9146) - Fixed the
MetaPath2Vec
model in case the last node is isolated (#9145) - Ensured backward compatibility in
MessagePassing
viatorch.load()
(#9105) - Prevented model compilation on custom
MessagePassing.propagate()
functions (#9079) - Ignore
self.propagate
appearances in comments when parsingMessagePassing
implementation (#9044) - Fixed
OSError
on read-only file systems withinMessagePassing
(#9032) - Fixed metaclass conflict in
Dataset
(#8999) - Fixed import errors on
MessagePassing
modules with nested inheritance (#8973) - Fixed TorchScript compilation error for
MessagePassing._check_input()
on older torch versions (#9564)
Changes
- Use
torch.load(weights_only=True)
by default (#9618) - Allow optional but untyped tensors in
MessagePassing
(#9494) - Added support for modifying
filename
of the stored partitioned file inClusterLoader
(#9448) - Support other than two-dimensional inputs in
AttentionalAggregation
(#9433) - Added the
fmt
argument toDataset.print_summary()
(#9408) - Skip zero atom molecules in
MoleculeNet
(#9318) - Ensure proper parallelism in
OnDiskDataset
for multi-threadedget
calls (#9140) - Allow mini-batching of uncoalesced sparse matrices (#9099)
- Default to
scatter()
operations inMessagePassing
in casetorch.use_deterministic_algorithms
is not set (#9009) - Added XPU support to basic GNN examples (#9421, #9439)
New Contributors
- @arthurdjn made their first contribution in #8918
- @project-delphi made their first contribution in #8946
- @arlesniak made their first contribution in #8978
- @simon-forb made their first contribution in #9057
- @rf523281 made their first contribution in #9075
- @BenediktAlkin made their first contribution in #9076
- @luckynozomi made their first contribution in #9113
- @MatthieuMelennec made their first contribution in #9116
- @brenting made their first contribution in #9146
- @drivanov made their first contribution in #9178
- @Akkete made their first contribution in #9204
- @1taroh made their first contribution in #9230
- @nelsonaloysio made their first contribution in #9334
- @chaojun-zhang made their first contribution in #9396
- @zhouyu5 made their first contribution in #9407
- @mzgubic made their first contribution in #9379
- @guanxingithub made their first contribution in #9386
- @devanshamin made their first contribution in #9445
- @ihkao made their first contribution in #9436
- @oiao made their first contribution in #9452
- @kano5266 made their first contribution in #9473
- @andyhuang-kumo made their first contribution in #9487
- @bryceForrest made their first contribution in #9533
- @ECMGit made their first contribution in #9543
- @nilserranestle made their first contribution in #9499
- @hnsgrvr made their first contribution in #9498
- @Linnore made their first contribution in #9515
- @alexbarghi-nv made their first contribution in #9541
- @MFairley made their first contribution in #9564
- @viktor-ktorvi made their first contribution in #9569
- @lee-clement-oxb made their first contribution in #9587
- @Danial-sb made their first contribution in #9554
- @DonBoscoBlaiseA made their first contribution in #9611
- @kaarthiksundar made their first contribution in #9621
- @rhjohnstone made their first contribution in #9632
Full Changelog: 2.5.0...2.6.0