TensorFlow Serving using TensorFlow 2.5.0-rc1
Major Features and Improvements
- Upgrade to CUDA 11.2 and and cuDNN 8.1.0 (commit: 1975e3e)
- Update TF Text to v2.4.3. (commit: ccfb606)
- Experimental support for serving JAX and XLA/CPU models. (commit: 3c1b2b3)
- Add latency and availability metrics to the Prometheus API (#1623) (commit: dfb41f1)
- Support URL reserved characters for REST API (#1726) (commit: dd9c467)
- Add Cross-Origin Resource Sharing (CORS) headers to REST API (#1817)
Breaking Changes
- No breaking changes
Bug Fixes and Other Changes
- Fix typo in REQUIRED_PACKAGES for grpcio (commit: b9ed0f8)
- Update resnet_k8s.yaml file (commit: e7b7b33)
- Fix a compile warning thrown by gcc-9 (commit: 38a017d)
- Fix typo (commit: dbcd54f)
- Update json_tensor.cc (commit: a0a9d14)
- Add TfLiteInterpreterPool to make concurrent use of TfliteSession better (commit: d9efa43)
- Enable download of TF Serving sources at arbitrary commit for CPU docker image. (commit: de1ab9e)
- Updated tests to newer API (commit: 30dd2fe)
- Control number of grpc threads for request handling to avoid OOM (Fixes #1785). (commit: ac0eb73)
- Add dedicated aliases field to ModelServerConfig. (commit: 358f7d1)
- Update docker command line to work with GPUs (Fixes #1768). (commit: b41a28b)
- Option to disable grpc over http (Fixes #1764) (commit: f087290)
- Remove an unused experimental config option "experimental_fixed_input_tensors_filepath". (commit: 3234fca)
- Removing CurriedSession, since it is no longer used. (commit: 87793ad)
- Improve error message for file not exists. (commit: 78d47f7)
- Fix inference request delay when model is switched (Fixes #1796). (commit: 803dd42)
- Transition TensorFlow Serving to TensorFlow's new WORKSPACE protocol. (commit: 50a7ef3)
- Clarifying object values in REST requests to include B64 encoding and similar key:value pair objects. (commit: 0536678)
- Remove experimental comment on TfLiteSession (commit: ab7f9a5)
- Register custom TfLite ParseExample and add benchmark (commit: 20fe3ca)
- Use respectful terms. (commit: b73bd7b)
- Pre-allocate memory for certain vectors where the size is known. (commit: e208b6e)
- Updating serving_basic for adjusting serving_basic.md file and making it up2date with TF2.x - including: (commit: cea306a)
- Use NullSafeStringView for potentially null pointer returned from libevent (commit: a46fdb2)
- Replace nullptr constructor for string_view with empty strings (commit: a98d164)
- Fixing MKL builds due to missing 'build_with_openmp' option (commit: 0ed23df)
- Implement batch parallelism for tflite sessions (commit: fec1d5d)
- Fix GPU docker image massive increase in size (#1813) (commit: 5a0dfd9)
- Fix TensorFlow Serving build with MKL+OpenMP (commit: ddad074)
- Remove hashtable custom op dependencies (commit: bb51722)
- Enable aspired version which failed to load to attempt reload. (commit: 2530a33)
- Fixed a compilation error in aspired_versions_manager.cc (commit: 4ca9a4b)
- Add "_r" root event annotation to ProcessBatch events. (commit: e5c3aec)
- Bump minimum bazel version 3.7.2. (commit: 5edcd13)
- Dont hardcode path to python3. (commit: 63b2d1c)
- Fix package build due to config move in: (commit: 18dd766)
- Add model_service_cc_grpc_proto (commit: a670ff5)
- Fix memory leak from allocating input tensors (commit: 2f9b6a0)
- Allowing lossy floating point conversions for JSON inputs (commit: 57dac6c)
- Adding enable_profiler command line flag. (commit: 7e8720d)
- Add logging in ServerCore. (commit: 623da67)
- Removes mention of ASCII (commit: 8e97b59)
Thanks to our Contributors
This release contains contributions from many people at Google, as well as:
Abhinav Pundir, Abolfazl Shahbazi, Aurélien Geron, Bairen Yi, gbaned, handong, Hao Ziyu, Junqin Zhang, kiddos, Oliver Sampson, OniB, Runzhen Wang, skawasak, zou000