TensorFlow Serving using TensorFlow 2.4.0-rc4.
Major Features and Improvements
- Update TF Text to v2.3.0.
- Upgrade to CUDA Version 11.0.
- Update CUDNN_VERSION to 8.0.4.30.
- Adds user guide for Remote Predict Op
- Upgrade to PY3 for tests.
Breaking Changes
Bug Fixes and Other Changes
- Migrate the remaining references of tf.app to compat.v1. (commit: 06fbf87)
- Enable a way to 'forget' unloaded models in the ServableStateMonitor. (commit: 53c5a65)d91cd4a00)
- Added abstract layer for remote predict op over different RPC protocols with template. (commit: c54ca7e)
- Adds user guide for Remote Predict Op (commit: fc82463)
- Add support for serving regress/classify (native) TF2 models. (commit: b724ced)
- For batching session in TF serving model server, introduce options to enable large batch splitting. (commit: f84187e)
- Add multi-inference support for TF2 models that use (commit: abb8d3b)
- Change the SASS & PTX we ship with TF (commit: 0869292)
- Adding custom op support. (commit: 892ea42)
- Broaden net_http client visibility and use HTTPStatusCode in ClientResponse status (commit: 7292714)
- In model warm-up path, re-write error code out-of-range (intended when reading EOF in a file) to ok. (commit: d9bde73)
- Support multiple SignatureDefs by key in TFLite models (commit: 2e14cd9)
- Add dedicated aliases field to ModelServerConfig. (commit: 718152d)
- Add
portpicker
module required to run modelserver e2e tests. (commit: 82f8cc0) - Minimal commandline client to trigger profiler on the modelserver. (commit: c0a5619)
- Add signature name to RPOp. (commit: 84dfc8b)
Thanks to our Contributors
This release contains contributions from many people at Google.