2.0.2 Release Note

重要更新

本版本主要是对2.0.1中一些功能和性能问题的修复，并对部分功能点做了增强，重点如下：

• paddle.nn.functional.cross_entropy 新增了 use_softmax 参数，控制是否在计算交叉熵前先进行softmax运算；并给paddle.nn.functional.softmax_with_cross_entropy 添加了 deprecated 标志，该API将在未来的版本中废弃。
• 修复了分布式训练中参数服务器模式下的多个问题。
• 升级Paddle的oneDNN版本至2.2版本，提升了多个模型的预测性能。

训练框架

功能优化

API

• 新增 paddle.io.random_split 与 paddle.io.Subset。(#32090)

问题修复

API

• 修复 paddle.nn.MaxPool3D和 paddle.nn.AvgPool3D 的stride 和 padding 没有默认值的问题。(#32014)
• 修复支持cudnn的 RNN 创建参数时报告重复创建的问题。(#31916)
• 修复 paddle.nn.functional.cross_entropy 的 soft_label 为 True，并指定 weight 参数时报错的问题；新增参数 use_softmax，用于控制是否在计算交叉熵前先进行softmax运算；同时，给 paddle.nn.functional.softmax_with_cross_entropy 添加 deprecated 说明，该API将会在未来的版本中废弃。(#31953、#32105、#32035)
• 修复paddle.nn.ClipByNorm在梯度全部为零时产生NaN数值的问题，该问题会导致使用混合精度训练时不收敛。(#32038)
• 修复 paddle.stack 内存越界访问的问题。(#32005)

分布式

• 修复参数服务器模式下计算图切分支持GradClip策略的问题。(#31945)
• 修复参数服务器模式下截断高斯分布初始化的问题。(#31945)
• 修复参数服务器模式下Profiler多线程信息打印不准确的问题。(#31945)
• 修复在Python3环境下使用Dataset读取数据时，使用zip输出数据时的兼容性问题。(#31945)
• 清理多余日志信息， 优化exe.train_from_dataset输出格式。(#32009)

推理部署

Paddle Inference

功能升级

• Paddle-TRT适配由Paddle 2.0 训练保存的ERNIE/BERT模型。(#31959)

性能优化

• 升级Paddle的oneDNN版本到oneDNN 2.2,多个模型预测性能有提升。(#31270)
  • Upgrade onednn to onednn 2.2 which improved many models inference performance. (#31270)
• 添加hard_swish oneDNN算子支持，增加 conv + hard_swish 算子融合, 使得ocr_det模型性能在SkyLake上提升18%。(#31870)
  • Add hard_swish oneDNN support and conv + hard_swish fusion, which improved ocr_det model inference performance by 18%. (#31870)

问题修复

• 修复rnn模型动态图转静态图导出保存后，运行时崩溃问题。(#31846)
• 修复了开启oneDNN预测连续多个图像时报错的问题。(#31837)
  • Fix continuous images inference failure when oneDNN is ON. (#31837)
• 修复了部署在CPU上的部分oneDNN int8 模型与原量化模型存在精度差的问题。(#31810)
  • Fix the accuracy difference between fake quantized models and deployed oneDNN int8 models. (#31810)
• 去除了SkipLayerNorm融合的多余限制条件。 (#32082、#32119)

Important Updates

This version fixed some function and performance issues of PaddlePaddle 2.0.1, and optimized some function. The important updates are as following:

• Add the use_softmax parameter to paddle.nn.functional.cross_entropy, which controls whether to perform softmax operation before calculating the cross entropy; add the deprecated mark to paddle.nn.functional.softmax_with_cross_entropy, for this API will be deprecated in the future version.
• Fix multiple issues of distributed training in parameter server mode。
• Upgrade Paddle's oneDNN version to 2.2, which improves the inference performance of multiple models.

Training Framework

Function Optimization

API

• Add paddle.io.random_split and paddle.io.Subset. (#32090)

Bug Fixes

API

• Fix the issue that the stride and padding of paddle.nn.MaxPool3D and paddle.nn.AvgPool3D do not have default values. (#32014)
• Fix the issue that when RNN supporting cudnn creates parameters, repeated creations are reported. (#31916)
• Fix the issue that when the soft_label of paddle.nn.functional.cross_entropy is True, and the weight parameter is specified, an error will be reported; add the use_softmax parameter to paddle.nn.functional.cross_entropy, which controls whether to perform softmax operation before calculating the cross entropy; add the deprecated mark to paddle.nn.functional.softmax_with_cross_entropy, for this API will be deprecated in the future version. (#31953, #32105, #32035)
• Fix the issue of paddle.nn.ClipByNorm generating NaN values as the gradients are all zero, which will lead to non-convergence when using mixed precision training. (#32038)
• Fix the issue of accessing array out of bounds in paddle.stack. (#32005)

Distributed Training

• Fix the issue that in parameter server mode the calculation graph segmentation supports GradClip strategy.(#31945)
• Fix the initialization of truncated gaussian distribution in parameter server mode.(#31945)
• Fix the issue of incorrectly printing the Profiler's multi-threaded information in parameter server mode.(#31945)
• Fix the Python3 incompatibility issue when data are read by Dataset and output by zip.(#31945)
• Clean up redundant log information and optimize the output format of exe.train_from_dataset.(#32009)

Inference Deployment

Paddle Inference

Function Upgrades

• Paddle-TRT adapts to the ERNIE/BERT model trained and saved by PaddlePaddle 2.0.(#31959)

Performance Optimization

• Upgrade onednn to version 2.2, which has improved many models inference performance. (#31270)
• Add hard_swish oneDNN support and conv + hard_swish fusion, which has improved ocr_det model inference performance by 18% on SkyLake. (#31870)

Bug Fixes

• Fix the issue that a run of the rnn model, which is saved after the dynamic graph to static graph, will crash.(#31846)
• Fix the error of inferring continuous images when oneDNN is ON. (#31837)
• Fix the accuracy difference between fake quantized models and deployed oneDNN int8 models. (#31810)
• Remove the redundant constraints of SkipLayerNorm fusion. (#32082、#32119)