We have received many requests from our community for more GNN layers, models and examples. This is the time to respond. In this minor release, we enriched DGL with a ton of common GNN modules. We have also verified their correctness on some popular datasets so feel free to try them out. Another direction we are working on is to build more domain friendly packages based on DGL. As a first step, we released several pretrained GNN models for molecular property prediction and molecule generation (currently grouped under dgl.model_zoo
namespace). We will continue explore this idea and release more domain specific models and packages.
New APIs
New NN Modules
GATConv
from “Graph Attention Network”RelGraphConv
from “Modeling Relational Data with Graph Convolutional Networks”TAGConv
from “Topology Adaptive Graph Convolutional Networks”EdgeConv
from “Dynamic Graph CNN for Learning on Point Clouds”SAGEConv
from “Inductive Representation Learning on Large Graphs”GatedGraphConv
from “Gated Graph Sequence Neural Networks”GMMConv
from “Geometric Deep Learning on Graphs and Manifolds using Mixture Model CNNs”GINConv
from “How Powerful are Graph Neural Networks?”ChebConv
from “Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering”SGConv
from “Simplifying Graph Convolutional Networks”NNConv
from “Neural Message Passing for Quantum Chemistry”APPNPConv
from “Predict then Propagate: Graph Neural Networks meet Personalized PageRank”AGNNConv
from “Attention-based Graph Neural Network for Semi-Supervised Learning
”DenseGraphConv
(Dense implementation ofGraphConv
)DenseSAGEConv
(Dense implementation ofSAGEConv
)DenseChebConv
(Dense implementation ofChebConv
)
New global pooling module
Sum/Avg/MaxPooling
SortPooling
GlobalAttentionPooling
from GGNN modelSet2Set
from “Order Matters: Sequence to sequence for sets”SetTransformerEncoder
andSetTransformerDecoder
from “Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks”
Please refer to the API document for more details.
New graph transformation routines
dgl.transform.khop_adj
dgl.transform.khop_graph
dgl.transform.laplacian_lambda_max
dgl.transform.knn_graph
dgl.transform.segmented_knn_graph
Please refer to the API document for more details.
Model zoo for chemistry and molecule applications
To make it easy for domain scientists, we are now releasing a model zoo for chemistry, with training scripts and pre-trained models, and focuses on two particular tasks: property prediction and targeted molecular generation/optimization.
Credit: Shout out to @geekinglcq from Tencent Quantum Lab for contributing three models (MGCN, SchNet and MPNN). We also thank WuXi AppTec CADD team for their critical feedback on usability.
Property prediction
In practice, the determination of molecular properties is mostly achieved via wet lab experiments. We can cast the problem as a regression or classification problem.
Featurization is the beginning of prediction. Traditionally, chemists develop pre-defined rules to convert molecular graphs into binary strings where each bit indicates the presence or absence of a particular substructure.
Graph neural networks enable a data-driven representation of molecules out of the atoms, bonds and molecular graph topology, which may be viewed as a learned fingerprint. The message passing mechanism allows the model to learn the interactions between atoms in a molecule.
The following code script is self-explanatory.
from dgl.data import Tox21
from dgl import model_zoo
dataset = Tox21()
model = model_zoo.chem.load_pretrained('GCN_Tox21') # Pretrained model loaded
model.eval()
smiles, g, label, mask = dataset[0]
feats = g.ndata.pop('h')
label_pred = model(g, feats)
print(smiles) # CCOc1ccc2nc(S(N)(=O)=O)sc2c1
print(label_pred[:, mask != 0]) # Mask non-existing labels
# tensor([[-0.7956, 0.4054, 0.4288, -0.5565, -0.0911,
# 0.9981, -0.1663, 0.2311, -0.2376, 0.9196]])
Supported Models
- Graph Convolution
- Graph Attention Networks
- SchNet
- Multilevel Graph Convolutional neural Network
- Message Passing Neural Networks
Generative Models
Targeted molecular generation refers to finding new molecules with desired properties. This gives rise to the need for generative models for two purposes:
- Distribution Learning: Given a collection of molecules, we want to model their distribution and generate new molecules consistent with the distribution.
- Goal-directed Optimization: Find molecules with desired properties.
For this model zoo, we provide only graph-based generative models. There are other generative models working with alternative representations like SMILES.
Example with Pre-trained Models
# We recommend running the code below with Jupyter notebooks
from IPython.display import SVG
from rdkit import Chem
from rdkit.Chem import Draw
from dgl import model_zoo
model = model_zoo.chem.load_pretrained('DGMG_ZINC_canonical')
model.eval()
mols = []
for i in range(4):
SMILES = model(rdkit_mol=True)
mols.append(Chem.MolFromSmiles(SMILES))
# Generating 4 molecules takes less than a second.
SVG(Draw.MolsToGridImage(mols, molsPerRow=4, subImgSize=(180, 150), useSVG=True))
Supported Models
- Learning Deep Generative Models of Graphs
- Junction Tree Variational Autoencoder for Molecular Graph Generation
API break
We refactor the nn
package to make all APIs more consistent. Thus, there are following changes to the API that breaks the previous behavior:
- Change the argument order of
dgl.nn.pytorch.GraphConv
anddgl.nn.mxnet.GraphConv
. The argument order is now firstgraph
and thenfeat
, which follows the convention of all the other new modules.
New model example
Recurrent Relational Networks in PyTorch (credit: @HuXiangkun )
There are also many bug fixes and minor changes. We will list them in the next 0.4 major release.