github ARM-software/armnn v23.08
Release 23.08
on GitHub

latest releases: v24.02, v23.11
8 months ago

Summary

New Features

  • Added support for tile operator in CpuRef, CpuAcc, GpuAcc.
  • Added support for reverse_v2 operator in CpuRef.
  • Added pow and squared_difference as ElementWiseBinary layers in CpuRef, CpuAcc, and GpuAcc.
  • Added squared_difference, power and ceil to TypeUtils.hpp.
  • Enabled dynamic / non-constant bias for:
    • Fully-Connected layers in CpuAcc and GpuAcc
    • 3-D Convolutional layers in CpuAcc and GpuAcc
    • Depthwise Convolutional layers in GpuAcc
  • Added DataType to .dot files for constant layers.
  • Added BinaryElementwiseOperation to .dot files.
  • Added a FileComparisonExecutor to ExecuteNetwork.
  • Added an optional TensorInfo to InputSlot.
  • Added 3D tensors to batch_to_space and space_to_batch for CpuAcc and GpuAcc.
  • Added check for half-precision floating-point infinity values and backend support (FP16).
  • Added backend optimisations to remove reshape layers where possible.
  • Added data layout to tensors in NeonStridedSliceWorkload.
  • Added names to workloads.
  • Enabled slice end-to-end tests in all backends and Signed32 in CpuRef.
  • Added axis to ViewsDescriptor.
  • Refactored ElementBinaryOps to use ElementBinaryLayer.

TfLite Parser

  • Added reverse_v2 support to TFLite Parser.
  • Added tile to TFLite Parser.
  • Added square as mul in the TFLite Parser.
  • Check for options != null before adding fused activation in TFLite Parser.
  • Fixed segfault with some models in the TFLite Parser.

Arm NN Serializer/Deserializer:

  • Added tile to Serialiser/Deserialiser.
  • Added reverse_v2 to Serialiser/Deserialiser.

Support library

  • Added reverse_v2 to Support Library.
  • Added tile to Support Library.
  • Added cache-size check to Support Library.

Bug Fixes

  • Fixed incorrect validation of unidirectional_sequence_lstm on CL and Neon.
  • Fixed issue with ExecuteNetwork when running with TFLite Executor.
  • Replaced asserts with exceptions in Gather reference workload.
  • Introduced fix to explicitly state the correct header to be included (following prior deprecation warning).
  • Fixed XML parsing error in Arm NN Doxygen.
  • Fixed -Werror=unused-result error.
  • Introduced fix for ExecuteNetwork where --output-network-details-only was not working with -T delegate flag.
  • Introduced fix for duplicate definitions in cross-compilation build.
  • Fixed incorrect Concat permutation parameters in Support Library.
  • Removed unnecessary warnings for certain models.
  • Introduced fix to allow SplitterLayer to use overridden TensorInfos correctly.
  • Introduced fix for some cases where the use of sub-tensors was causing an error.
  • Fixed read memory access caused by missing printf arguments.
  • Introduced fix for failing dynamic backend build.
  • Fixed issue where the dimension's specificity didn't match the number of dimensions.
  • Fixed ambiguous method name in BackendHelper.
  • Introduced fix for segmentation fault when an input was directly connected to an output.
  • Fixed uninitialised variable error found during static analysis.
  • Fixed fault in ExecuteNetwork when a model file was passed without an extension.
  • Fixed GitHub issue where search bar was not working in Doxygen documentation.

Other Changes

  • Replaced use of std::filesystem with ghc::filesystem.
  • Refactored ConnectedToSplitterWithMoreThan4Dims function to a more generally useful ConnectedToLayerType function.
  • Customised Doxygen output.
  • Removal of deprecated code due to be removed in 23.08 or earlier:
    • INetworkProperties
    • SubgraphView
    • ILayerSupport
    • WorkloadFactory
  • Updated documentation with new operators in 23.08.
  • Audited the use of armnn_assert.

Known Issues

  • Intermittent issue on Dma Buf memory import on GPU. This is fix in Mali Driver r30p0.
  • There might be performance regressions against v20.08 in Inception v3 using int8 data types on Arm Mali-G77 GPUs. Currently under investigation.

ABI/API Changes

The following front-end API changes have occurred during the implementation of 23.08 that users should be aware of before upgrading. Due to these changes we have bumped our ARMNN_VERSION to 33.0.0, following Semantic Versioning guidelines.

Feature SHA Gerrit Review Resultant ABI/API changes
Removal of Reshape 4cc341c https://review.mlplatform.org/c/ml/armnn/+/9885 4 additional virtual methods added to class IInputSlot:
SetTensorInfo ( TensorInfo ),
GetTensorInfo ( ) const,
IsTensorInfoSet ( ) const,
IsTensorInfoOverridden ( ) const
Front end and reference implementation for TILE 79a06a5 https://review.mlplatform.org/c/ml/armnn/+/9920 LayerType enum has had the LastLayer member value changed from 72 to 74 The member Tile with value 74 has been added
Remove deprecated code 09e4d05 https://review.mlplatform.org/c/ml/armnn/+/9266 2 functions have been removed from the BatchMatMulDescriptor class:
  • BatchMatMulDescriptor::GetAxesNotMul ( struct BatchMatMulDescriptor const& desc, TensorShape const& inputXShape, TensorShape const& inputYShape ) [static]
  • BatchMatMulDescriptor::GetAxesToMul ( struct BatchMatMulDescriptor const& desc, TensorShape const& tensorXShape, TensorShape const& tensorYShape ) [static]
Remove deprecated code (INetworkProperties) b179382 https://review.mlplatform.org/c/ml/armnn/+/10001 INetworkProperties structure has had the following fields removed: m_ExportEnabled m_ImportEnabled
Remove deprecated code (ILayerSupport) 6627703 https://review.mlplatform.org/c/ml/armnn/+/10005 In ArmNN individual virtual IsXXXSupported() functions in the ILayerSupport class have been removed. This functionality has been replaced by a more ABI compliant model whereby an IsLayerSupported() function now accepts a LayerType argument.
In ArmNNTestUtils, removal of 4 virtual methods from class MockLayerSupport:
  • IsAdditionSupported ( TensorInfo const&, TensorInfo const&, TensorInfo const&, Optional<std::__cxx11::basic_string&> ) const
  • IsConvolution2dSupported ( TensorInfo const&, TensorInfo const&, struct Convolution2dDescriptor const&, TensorInfo const&, Optionalconst&, Optional<std::__cxx11::basic_string&> ) const
  • IsInputSupported ( TensorInfo const&, Optional<std::__cxx11::basic_string&> ) const
  • IsOutputSupported ( TensorInfo const&, Optional<std::__cxx11::basic_string&> ) const
Remove deprecated code (WorkloadFactory) 7894ef9 https://review.mlplatform.org/c/ml/armnn/+/10006 In ArmNNTestUtils the MockWorkloadFactory class has had the following virtual method removed: CreateInput ( InputQueueDescriptor const&, struct WorkloadInfo const& ) const
Added Axis to ViewsDescriptor fca5916 https://review.mlplatform.org/c/ml/armnn/+/10073 The size of thte ViewsDescriptor structure has changed from 48 bytes to 56 bytes Field m_IsAxisSet has been added

The following back-end API changes have occurred during the implementation of 23.08 that users should be aware of before upgrading.

Feature SHA Gerrit Review Resultant ABI/API changes
Add names to workloads 7cbe781 https://review.mlplatform.org/c/ml/armnn/+/9983 Pure virtual method GetName ( ) const has been added to IWorkload class. Size of CopyMemGenericWorkload class has been changed from 152 bytes to 184 bytes.
Remove deprecated code (SubgraphView) 0f3e9a0 https://review.mlplatform.org/c/ml/armnn/+/10009 Removed the following methods from SubgraphView:
GetInputSlot, GetIputSlots, GetLayers, GetOutputSlot, GetOutputtSlots.
Resulted in a change to return types for the following methods in SubgraphView as follows:
begin() returns SubgraphView::IConnectableLayerIterator,
begin() const returns SubgraphView::ConstIConnectableIterator,
cbegin() const returns, SubgraphView::ConstIConnectableIterator,
cend() const returns SubgraphView::ConstIConnectableIterator ,
end () returns SubgraphView::IConnectableLayerIterator,
end() const returns SubgraphView::ConstIConnectableIterator
Remove deprecated code (ILayerSupport) a504834 https://review.mlplatform.org/c/ml/armnn/+/10071 Made IWorkloadFactory::CreateWorkload a pure virtual function to force client to write its own implementation.
Fix coverity error on variable initialize b9b9792 https://review.mlplatform.org/c/ml/armnn/+/10075 Adjusted sequence of variable initialization in struct WorkloadInfo:
m_Name
m_WeightsTensorInfo
m_BiasTensorInfo
m_ConvolutionMethod

TfLite Delegate

  • Extended support for 3D tensors (batch_to_space and space_to_batch) in CpuRef.
  • Added opaque delegate Options subsection to Doxygen.
  • Added layerNames to classic and opaque delegate.
  • Added reverse_v2 to classic and opaque delegates.
  • Added tile to delegate and opaque delegate.
  • Added leaky_relu to delegate.

Bug Fixes

  • Fixed versions in delegate "Quick Start" guide.
  • Opaque delegate cleanup.
  • Fixed failure on unidirectional_sequence_lstm operator.
  • Fixed instance where ExpandDims would not work where batch != 1.

PyArmNN

  • Updated PyArm NN to include new features added in Arm NN.
  • Added relevant deprecation message when building PyArm NN.

Build Dependencies

Tools Supported Version
Git 2.17.1 or later
SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
Cmake 3.19.0 (Ubuntu) and 3.19.0 (Debian)
Tensorflow 2.12.0
Onnx 1.6.0
Flatbuffer 2.0.6
Protobuf 3.12.0
Android NDK r25
mapbox/variant 1.2.0
cxxopts 3.1.1
doctest 2.4.6
fmt 8.3.0
ghc 1.3.2
half 1.12.0
mapbox/variant 1.2.0
stb 2.16
xxd 1.10
Check out latest releases or
releases around ARM-software/armnn v23.08

Don't miss a new armnn release

NewReleases is sending notifications on new releases.

Get notifications