Introducing MeshTensorFlow - this enables training really big models O(Billions) of parameters.
Models/Layers:
- Layers Added: NAC and NALU from https://arxiv.org/abs/1808.00508 Thanks @lukaszkaiser !
- Added a sparse graph neural net message passing layer to tensor2tensor.
- Targeted dropout added to ResNet. Thanks @aidangomez !
- Added VQA models in
models/research/vqa_* - Added
Weight Normalizationlayer from https://arxiv.org/abs/1602.07868.
Datasets/Problems:
- MSCoCo paraphrase problem added by @tlatkowski - many thanks!
VideoBairRobotPushingWithActionsby @mbz !
Usability:
- Code cleaup in autoencoder, works both on image and text. Thanks @lukaszkaiser
- Set the default value of Text2TextProblem.max_subtoken_length to 200, this prevents very long vocabulary generation times. Thanks @afrozenator
- Add examples to distributed_training.md, update support for async training, and simplify run_std_server codepath. Thanks @rsepassi !
- Store variable scopes in T2TModel; add T2TModel.initialize_from_ckpt. Thanks @rsepassi !
- Undeprecate exporting the model from the trainer Thanks @gcampax !
- Doc fixes, thanks to @stefan-it :)
- Added t2t_prune: simple magnitude-based pruning script for T2T Thanks @aidangomez !
- Added task sampling support for more than two tasks. Thanks @urvashik !
Bug Fixes:
- Override serving_input_fn for video problems.
StackWrappereliminates problem with repeating actions. Thanks @blazejosinski !- Calculated lengths of sequences using _raw in lstm.py
- Update universal_transformer_util.py to fix TypeError Thanks @zxqchat !
Testing:
- Serving tests re-enabled on Travis using Docker. Thanks @rsepassi !
Many more fixes, tests and work on RL, Glow, SAVP, Video and other models and problems.