Major Features and Improvements
- Update TF Text to v2.3.0.
- Upgrade to CUDA Version 11.0.
- Update CUDNN_VERSION to 8.0.4.30.
- Adds user guide for Remote Predict Op.
- Add support for serving regress/classify (native keras) TF2 models.
Breaking Changes
Bug Fixes and Other Changes
- Adding /usr/local/cuda/extras/CUPTI/lib64 to LD_LIBRARY_PATH in order to unblock profiling (commit: 1270b8c)
- Improve error message when version directory is not found (commit: d687d3e)
- Migrate the remaining references of tf.app to compat.v1. (commit: 06fbf87)
- Cleanup TraceMe idioms (commit: f22f802)
- Adds LICENSE file to tensorflow-serving-api python package. (commit: 41188d4)
- Enable a way to 'forget' unloaded models in the ServableStateMonitor. (commit: 53c5a65)
- Added abstract layer for remote predict op over different RPC protocols with template. (commit: c54ca7e)
- Add an example which call the Remote Predict Op directly. (commit: d5b980f)
- For batching session in TF serving model server, introduce options to enable large batch splitting. (commit: f84187e)
- Add multi-inference support for TF2 models that use (commit: abb8d3b)
- Use absl::optional instead of tensorflow::serving::optional. (commit: c809305)
- Use absl::optional instead of tensorflow::serving::optional. (commit: cf1cf93)
- Remove tensorflow::serving::MakeCleanup and use tensorflow::gtl::MakeCleanup. (commit: 6ccb003)
- Use absl::optional and remove tensorflow::serving::optional. (commit: e8e5222)
- Deprecate tensorflow::CreateProfilerService() and update serving client. (commit: 98a5503)
- Change the SASS & PTX we ship with TF (commit: 0869292)
- Adding custom op support. (commit: 892ea42)
- Upgrade to PY3 for tests. (commit: 02624a8)
- Makes clear how to make a default config file for serving multiple models. (commit: 084eaeb)
- Use TraceMeEncode in BatchingSession's TraceMe. (commit: 78ff058)
- Export metrics for runtime latency for predict/classify/regress. (commit: c317582)
- Refactor net_http/client to expose request/response functionality as a public API (not yet finalized) for usage testing ServerRequestInterface and HttpServerInterface instances. (commit: 0b951c8)
- In model warm-up path, re-write error code out-of-range (intended when reading EOF in a file) to ok. (commit: d9bde73)
- fix Client Rest API endpoint (commit: b847bac)
- Support multiple SignatureDefs by key in TFLite models (commit: 2e14cd9)
- Add dedicated aliases field to ModelServerConfig. (commit: 718152d)
- Remove deprecated flag fail_if_no_model_versions_found from tensorflow serving binary (commit: 4b62462)
- Fix TraceMe instrumentation for the padding size. (commit: 0cb94cd)
- Add vlog to dump updated model label map (for debugging) each time the map is updated. (commit: ac10e74)
- Add python wrapper for remote predict op and clean the build and include files. (commit: d0daa10)
- Add
portpickermodule required to run modelserver e2e tests. (commit: 82f8cc0) - changing "infintiy" to "really high value" (commit: c96474c)
- Minimal commandline client to trigger profiler on the modelserver. (commit: c0a5619)
- Add signature name to RPOp. (commit: 84dfc8b)
- When RPC error occurs, the output tensors should still get allocated. (commit: 9113de2)
- Fix BM_MobileNet benchmark (commit: af66562)
- Add JSPB BUILD targets for inference and example proto files. (commit: f1009eb)
- Fall back to legacy TFLite tensor naming when parsing signature defs in TFLiteSession. (commit: 3884187)
Thanks to our Contributors
This release contains contributions from many people at Google, as well as:
Adarshreddy Adelli, Lescurel