New Features
- Arm NN TOSA Backend
- Added Concatenation support to TOSA Reference Backend.
- Added Constant layer support to TOSA Reference Backend.
- Added Convolution 2D support to TOSA Reference Backend.
- Added Pooling2d support to TOSA Reference Backend.
- Added Reshape support to TOSA Reference Backend.
- Added RSqrt support to TOSA Reference Backend.
- Added Slice support to TOSA Reference Backend.
- Added Transpose Convolution 2D support to TOSA Reference Backend.
- Added Subtraction and Multiplication support to TOSA Reference Backend.
- Added support for GpuAcc BatchMatMul with FP32.
- Extend BatchMatMul support for 4D tensors in GpuAcc.
ONNX Parser
- Provide a CreateNetworkFromBinary method for the ONNX parser.
TfLite Parser:
- Fixed issue in ParseReshape where the targetShape wasn't always calculated correctly.
- Fixed issue in ParseFullyConnected where the wrong name was used for the ReshapeLayer.
- Added an ExpandDims to the FullyConnected to ensure that we reshape the output correctly.
Bug Fixes
- Bug fixed on ExecuteNetwork when input names where not given, input files were not used.
- Bug Fixed on delegate Profiling in ExecuteNetwork with multiple iterations.
- Bug Fixed for CpuAcc and GpuAcc. BuildArmComputePermutationVector() function needed to be rewritten to account for all possible permutation vectors.
- Fixed an ExecuteNetwork unhandled exception when using option --import-inputs-if-aligned.
- Fixed Arm NNAPI Support Library to fail gracefully if device is unavailable.
- Fixed edge cases where some permute vectors for Arm Compute were not converted correctly.
- Fixed bug where GPU backend options were not being correctly passed by our delegate.
- Fixed bug when converting Constants with Per-Axis Quantization.
- Fixed bug where call on SubstituteSubgraph on working copy of subgraph in Optimize fails.
- Fixed segfault in ExecuteNetwork when no operator is supported by Arm NN.
- Fixed bug for slot replacement during UpdateSubgraphViewSlotPointers.
- Fixed bug for ExecuteNetwork using delegate when output is boolean from comparison layer.
Other Changes
- Disabled BF16-Turbo-Mode and remove conversion layers.
- Added Arm NN include directory into build-tool output.
- Code improvement through removal of unused includes.
- Optimization of IsLayerSupported to reduce calls to it.
- Removed deprecated code due to be removed in 23.02.
- Changed Arm NN Support LIbrary to use static libraries instead of object libraries.
- Added option of static build of Execute Network.
- Improved error handling when ExecuteNetwork creates a directory when -F option used.
- Changed ArmNNExecutors to now share a single IRuntime, which allows ExecuteNetwork to create and run multiple Executors instead of one.
- Added documentation relating to multithreading.
ABI/API Changes
The following front-end API changes have occurred during the implementation of 23.02 that users should be aware of before upgrading.
.
| Feature | SHA | Gerrit Review | Resultant ABI/API changes |
|---|---|---|---|
| Optimize the calling of IsLayerSupported(). | 5383767 | https://review.mlplatform.org/c/ml/armnn/+/8742 | In class IConnectableLayer: Pure virtual method SetBackendId (BackendId const&) has been added to this class. Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method. The layout of v-table has been changed. Call of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications. |
| When creating multiple Executors only the last one works fine | 5446a4d | https://review.mlplatform.org/c/ml/armnn/+/8997 | In class Delegate: Size of this type has been changed from 688 bytes to 680 bytes. The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications. Type of field m_Runtime has been changed from armnn::IRuntimePtr (16 bytes) to armnn::IRuntime* (8 bytes). Size of the inclusive type has been changed |
| Fix incorrect last layer in Types.hpp | 6701daf | https://review.mlplatform.org/c/ml/armnn/+/8944 | In enum LayerType: Value of member LastLayer has been changed from 66 to 71. Applications may execute a wrong branch of code in the library and therefore change the behavior. |
| Change to MemorySource to keep it usable as a bit mask | 1cebf49 | https://review.mlplatform.org/c/ml/armnn/+/9053 | In enum MemorySource: Value of member Gralloc has been changed from 5 to 8. Applications may execute a wrong branch of code in the library and therefore change the behavior. |
The following back-end API changes have occurred during the implementation of 23.02 that users should be aware of before upgrading.
| Feature | SHA | Gerrit Review | Resultant ABI/API changes |
|---|---|---|---|
| Remove deprecated code due to be removed in 23.02 | ec67a0f | https://review.mlplatform.org/c/ml/armnn/+/8319 | In struct Convolution2dQueueDescriptor, DepthwiseConvolution2dQueueDescriptor and FullyConnectedQueueDescriptor: Field m_Bias has been removed from this type. Field m_Weight has been removed from this type. 1) Applications will access incorrect memory when attempting to access this field. 2) Size of the inclusive type has been changed. 3) The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications. In class BaseWorkload , BaseWorkload and BaseWorkload: 1) Size of the inclusive type has been changed. 2) Previous accesses of applications and library functions to this field and fields at higher positions of the structure definition may be broken. 3) The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications. |
| Return INetwork* not INetworkPtr& from OptimizationViews::GetINetwork() | 5b2145c | https://review.mlplatform.org/c/ml/armnn/+/8828 | In OptimizationViews::GetNetwork(): Base type of return value has been changed from std::unique_ptr<INetwork, void()(INetwork)> to INetwork. Recompilation of a client program may be broken. |
| Allow working copy SubgraphView to get Original Slots | 01f7269 | https://review.mlplatform.org/c/ml/armnn/+/8918 | In class SubgraphView: Base class std::enable_shared_from_this has been added. 1) Size of the class has been changed from 160 bytes to 176 bytes. 2) The memory layout in this class has been shifted by 16 bytes. 3) The class has only inline or auto-generated constructors which will be copied to applications at compile time and will allocate an older memory layout. Call of any exported method of this class may access a memory outside the allocated objects or inside the older memory structure and result in crash or incorrect behavior of applications. 4) The memory layout and size of subclasses will be changed. |
TfLite Delegate
New features
- Added support for Slice operator.
- Made change to allow constant tensors as inputs for input data in the delegate.
Bug Fixes
- Fixed delegate fallback during VisitNode so that an ArmNN exception is now caught and the process is handed over to TFLite.
- Added an ExpandDims to the FullyConnected to ensure that we reshape the output correctly.
- Fixed delegate fallback when fused activation is unsupported.
- Fixed uncaught warnings treated as errors in delegate release build.
PyArm NN
- Add installation instructions for prebuilt binaries.
Build Dependencies
| Tools | Supported Version |
|---|---|
| Git | 2.17.1 or later |
| SCons | 2.4.1 (Ubuntu) 2.5.1 (Debian) |
| Cmake | 3.5.1 (Ubuntu) and 3.7.2 (Debian) |
| Tensorflow | 2.10.0 |
| Onnx | 1.6.0 |
| Flatbuffer | 2.0.6 |
| Protobuf | 3.12.0 |
| Android NDK | r25 |
| mapbox/variant | 1.2.0 |
| cxxopts | SHA 12e496da3d486b87fa9df43edea65232ed852510 |
| doctest | 2.4.6 |
| fmt | 7.0.1 |
| ghc | 1.3.2 |
| half | 1.12.0 |
| stb | 2.16 |
| xxd | 1.10 |