This is the first major release of the Adversarial Robustness 360 Toolbox (ART v1.0)!

This release generalises ART to support all possible classifier models, in addition to its existing support for neural networks. Furthermore, it generalises the label format, to accept index labels as well as one-hot encoded labels, and the input shape, to accept, for example, tabular data as input features. This release also adds new model-specific white-box and poisoning attacks and provides new methods to certify and verify the adversarial robustness of neural networks and decision tree ensembles.

Added

• Add support for all classifiers and pipelines of scikit-learn including but not limited to LogisticRegression, SVC, LinearSVC, DecisionTreeClassifier, AdaBoostClassifier, BaggingClassifier, ExtraTreesClassifier, GradientBoostingClassifier, RandomForestClassifier, and Pipeline. (#47)

• Add support for gradient boosted tree classifier models of XGBoost, LightGBM and CatBoost.

• Add support for TensorFlow v2 (rc0) by introducing a new classifier TensorFlowV2Classifier providing support for eager execution and accepting callable models. KerasClassifier has been extended to provide support for TensorFlow v2 tensorflow.keras Models without eager execution. (#66)

• Add support for models of the Gaussian Process framework GPy. (#116)

• Add the High-Confidence-Low-Uncertainty (HCLU) adversarial example formulation as an attack on Gaussian Processes. (#116)

• Add the Decision Tree attack as a white-box attack for decision tree classifiers (#115)

• Add support for white-box attacks on scikit-learn’s LogisticRegression, SVC, LinerSVC, and DecisionTreeClassifier, as well as GPy and black-box attacks on all scikit-learn classifiers and XGBoost, LightGBM and CatBoost models.

• Add Randomized Smoothing as wrapper class for neural network classifiers to provide certified adversarial robustness under the L2 norm. (#114)

• Add the Clique Method Robustness Verification method for decision-tree-ensemble classifiers and extend it for models of XGBoost, LightGBM, and scikit-learn's ExtraTreesClassifier, GradientBoostingClassifier, RandomForestClassifier. (#124)

• Add BlackBoxClassifier expecting only a single Python function as interface to the classifier predictions. This is the most general and versatile classifier of ART. New tutorial notebooks demonstrate BlackBoxClassifier testing the adversarial robustness of remote, deployed classifier models and of the Optical Character Recognition (OCR) engine Tesseract. (#123, #152)

• Add the Poisoning Attack for Support Vector Machines with linear, polynomial or radial basis function kernels. (#155)

Changed

• Introduce a new flexible API for all classifiers with an abstract base class for basic classifiers (minimal functionality to support black-box attacks), and mixins for neural networks, gradient-providing classifiers (to support white-box attacks), and decision-tree-based classifiers.

• Update, extend and introduce new get started examples and notebook tutorials for all supported frameworks. (#47, #140)

• Extend label format to accept index labels in addition to the already supported one-hot-encoded labels. Internally ART continues to treat labels as one-hot-encoded. This feature allows users of ART to use the label format preferred by their machine learning framework and datasets. (#126)

• Change the order of the preprocessing steps of applying defences and standardisation/normalisation in classifiers. So far the classifiers first applied standardisation followed by defences. With this release the defences will be applied first followed by standardisation to enable comparable defence parameters across classifiers with different standardisation/normalisation parameters. (#84)

• Use the batch_size of an attack as argument to the method predict of its classifier to reduce out-of-memory errors for large models. (#105 )

• Generalize the classifiers of TensorFlow, Keras, PyTorch, and MXNet by removing assumptions on their output (logits or probabilities). The Boolean parameter logits has been removed from Classifier API in methods predict and class_gradient. The predictions and gradients are now computed at the output of the model without any modifications. (#50, #75, #106, #150)

• Rename TFClassifier to TensorFlowClassifier and keep TFClassifier for backward compatibility.

Removed

• Sunset support for Python 2 in preparation for its retirement on Jan 1, 2020. We have stopped running unittests with Python 2 and do not require new contributions to run with Python 2. We keep existing compatibility code for Python 2 and 3 where possible. (#83)

Fixed

• Improve VirtualAdversarialMethod by making the computation of the L2 data normalisation more reliable and raising an exception if it is used with a model providing logits as output. Currently, VirtualAdversarialMethod is expecting probabilities as output. (#120, #157)