github ARM-software/armnn v21.11
Release 21.11
on GitHub

latest releases: v24.08, v24.05, v24.02...
2 years ago

Arm NN 21.11 was focused on providing new capabilities and improve performance:

New Features

  • Added support for Reduce Prod.
  • Added support for Channel Shuffle.
  • Added support for Conv3d.
  • Added support for Symmetric and Reflect Padding on CpuRef backend.
  • Added support for statically linking ArmNN TfLite Delegate against Tensorflow Lite.
  • Added Import Input/Output functions to async API, allowing for imported I/O buffers to be used by multiple network executions.
  • Added external memory manager that allows for customization of network memory management ( Note: currently only fully supported on the CpuRef Backend ).

TfLite Parser

  • Added support for Reduce Prod.
  • Added support for Conv3d.
  • Added support for MirrorPad.
  • Added support for size of -1 for Slice.

ONNX Parser

  • Add support for Concat
  • Add support for Gather
  • Add support for Gemm
    • The parser supports constant bias or non-constant bias where bias dimension = 1.
  • Add support for Shape
  • Add support for Unsqueeze
  • Add support of min/max as attribute for Clip

ArmNN Serializer/Deserializer

  • Add support for Reduce Prod.
  • Add support for Channel Shuffle.
  • Add support for Conv3d.
  • Add support for Symmetric and Reflect Padding.

ExecuteNetwork App Changes

  • Added 'do-not-print-output' option to ExecuteNetwork.

Bug Fixes

  • Using output-network-details or output-network-details-only during ExecuteNetwork profiling created an invalid JSON format. This has since been fixed.
  • Fixed undefined reinterpret_cast in BFloat16.hpp. It fixes gcc builds with version 8 or above.
  • Fixed format of the delegate JSON output.
  • Fixed bug related with constant tensor flag.
  • Fixed pyarmnn py35 unit tests.

Other Changes

  • Added sample app for asynchronous execution.
  • Printed new Optimize and LoadedNetwork profiling points.
  • Added new serialized model supported on Netron.
  • Made it possible for backends to add include paths in Android.
  • Changed order of the Doxygen tree.

ABI/API Changes

The following front-end API changes have occurred during the implementation of 21.11 that users should be aware of before upgrading. Due to these changes we have bumped our ARMNN_VERSION to 27.0.0, the Delegate to 25.0.0 and also bumping our Parsers to 24.3.0 following Semantic Versioning guidelines.

.

Feature SHA Gerrit Review Resultant ABI/API changes
Remove deprecated code 1b2654f https://review.mlplatform.org/c/ml/armnn/+/6254 Removed Symbols:
  • INetwork::AddAbsLayer ( char const* name )
  • INetwork::AddDepthwiseConvolution2dLayer ( struct DepthwiseConvolution2dDescriptor const& convolution2dDescriptor, ConstTensor const& weights, ConstTensor const& biases, char const* name )
  • INetwork::AddDepthwiseConvolution2dLayer ( struct DepthwiseConvolution2dDescriptor const& convolution2dDescriptor, ConstTensor const& weights, char const* name )
  • INetwork::AddEqualLayer ( char const* name )
  • INetwork::AddGatherLayer ( char const* name )
  • INetwork::AddGreaterLayer ( char const* name )
  • INetwork::AddMergerLayer ( MergerDescriptor const& mergerDescriptor, char const* name )
  • INetwork::AddResizeBilinearLayer ( struct ResizeBilinearDescriptor const& descriptor, char const* name )
  • INetwork::AddRsqrtLayer ( char const* name )
  • LayerSupport::IsMergerSupported ( BackendId const& backend, std::vector<TensorInfo const*> inputs, TensorInfo const& output, struct OriginsDescriptor const& descriptor, char* reasonIfUnsupported, size_t reasonIfUnsupportedMaxLength )
  • LayerSupport::IsResizeBilinearSupported ( BackendId const& backend, TensorInfo const& input, TensorInfo const& output, char* reasonIfUnsupported, size_t reasonIfUnsupportedMaxLength )
  • LayerSupport::IsRsqrtSupported ( BackendId const& backend, TensorInfo const& input, TensorInfo const& output, char* reasonIfUnsupported, size_t reasonIfUnsupportedMaxLength )
  • LayerSupport::IsSplitterSupported ( BackendId const& backend, TensorInfo const& input, struct ViewsDescriptor const& descriptor, char* reasonIfUnsupported, size_t reasonIfUnsupportedMaxLength )
    Removed pure virtual methods, resulting in change to v-table layout:
  • ILayerVisitor::VisitAbsLayer
  • ILayerVisitor::VisitEqualLayer
  • ILayerVisitor::VisitGatherLayer
  • ILayerVisitor::VisitGreaterLayer
  • ILayerVisitor::VisitMergerLayer
  • ILayerVisitor::VisitResizeBilinearLayer
  • ILayerVisitor::VisitRsqrtLayer
    Removed DataTypes:
  • DataType::QuantisedAsymm8
  • DateType::QuantisedSymm16
  • DataType::QuantizedSymm8PerAxis
    • 'IMemoryOptimizerStrategy Add strategy library and add support in BackendRegistry' b8a26d8 https://review.mlplatform.org/c/ml/armnn/+/6297 struct IRuntime::CreationOptions:
  • Member variable m_MemoryOptimizerStrategyMap has been added, changing the size of the type. class BackendRegistry:
  • Member variable m_MemoryOptimizerStrategyMap has been added, changing the size of the type.
    • Add missing runtime parameters to TfLite delegate. 3e32a87 https://review.mlplatform.org/c/ml/armnn/+/6388 class Delegate:
  • Size of field m_Options has been changed from 136 bytes to 352 bytes.
    class DelegateOptions had the following fields added and so the size of the inclusive type has been changed.
  • Field m_DynamicBackendsPath has been added to this type.
  • Field m_EnableGpuProfiling has been added to this type.
  • Field m_InternalProfilingDetail has been added to this type.
  • Field m_InternalProfilingEnabled has been added to this type.
  • Field m_ProfilingOptions has been added to this type.
  • Field m_SerializeToDot has been added to this type.
    • Profiling instrumentation throughout the Optimizer f1e0ad3 https://review.mlplatform.org/c/ml/armnn/+/6432 struct OptimizerOptions:
  • Field m_ProfilingEnabled has been added to this type.
    class Delegate:
  • Size of this class has been increased from 416 bytes to 424 bytes.
    class DelegateOptions:
  • Size of this class has been increased from 352 bytes to 360 bytes.
    Objects of these classes can be allocated by the applications and old size will be hardcoded at the compile time. Call of any exported constructor will break the memory of neighboring objects on the stack or heap.
    This is due to addition of m_ProfilingEnabled to the OptimizerOptions used in constructors of both Delegate classes.
    • Fix armnn_external_delegate option parsing b1c62f1 https://review.mlplatform.org/c/ml/armnn/+/6519 class Delegate:
  • Size of field m_Options has been changed from 360 bytes to 616 bytes.
    class DelegateOptions:
  • Field m_RuntimeOptions has been added to this type.
  • Field m_BackendOptions has been removed from this type.
  • Field m_DynamicBackendsPath has been removed from this type.
  • Field m_EnableGpuProfiling has been removed from this type.
    Objects of these classes can be allocated by the applications and old size will be hardcoded at the compile time. Call of any exported constructor will break the memory of neighboring objects on the stack or heap.
    • Support the new memory API in loaded network b1aad42 https://review.mlplatform.org/c/ml/armnn/+/6552 class INetworkProperties:
  • Field m_ExternalMemoryManagementEnabled has been added to this type.
    The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications.

    • The following back-end API changes have occurred during the implementation of 21.11 that users should be aware of before upgrading.

    Feature SHA Gerrit Review Resultant ABI/API changes
    Remove deprecated code 1b2654f https://review.mlplatform.org/c/ml/armnn/+/6254 IBackendInternal.hpp
    Removed Symbols:
  • virtual ISubGraphConverterPtr CreateSubGraphConverter(const std::shared_ptr& subGraph) const;
  • virtual Optimizations GetOptimizations() const;
  • virtual SubGraphUniquePtr OptimizeSubGraph(const SubGraph& subGraph, bool& optimizationAttempted) const;
    Removed Aliases:
  • GraphUniquePtr, SubgraphViewUniquePtr, ISubGraphConverterPtr, SubGraphUniquePtr
    ILayerSupport.hpp
    Removed Symbols:
  • IsEqualSupported
  • IsGatherSupported
  • IsGreaterSupported
  • IsMergerSupported
  • IsResizeBilinearSupported
  • IsRsqrtSupported
  • IsSplitterSupported
    • Add Channel Shuffle Front end and Ref Implementation 51f6777 https://review.mlplatform.org/c/ml/armnn/+/6211 ILayerSupport.hpp
  • Pure virtual function IsChannelShuffleSupported added requiring implementation by backend developers.
    • Add Conv3d FrontEnd and Ref Implementation b63a311 https://review.mlplatform.org/c/ml/armnn/+/6338 ILayerSupport.hpp
  • Pure virtual function IsConvolution3dSupported added requiring implementation by backend developers.

    • TfLite Delegate

    New features

    • Added support for Reduce Prod.
    • Added support for Conv3d.
      • Conv3d is only currently supported in the TfLite Delegate when compiling with TensorFlow 2.6 and above.
    • Added support for Floor Div.
    • Added support for MirrorPad.

    Build Dependencies

    Tools Supported Version
    Git 2.17.1 or later
    SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
    Cmake 3.5.1 (Ubuntu) and 3.7.2 (Debian)
    Tensorflow 2.5.0
    Onnx 1.6.0
    Flatbuffer 1.12.0
    Protobuf 3.12.0
    Android NDK r20b
    mapbox/variant 1.2.0
    cxxopts SHA 12e496da3d486b87fa9df43edea65232ed852510
    doctest 2.4.6
    fmt 7.0.1
    ghc 1.3.2
    half 1.12.0
    stb 2.16

    Note: We have also added an Arm NN Android Library as a new experimental feature. It allows you to easily integrate Arm NN into an Android app. Please find the .aar file in the Asset section.

    Android 12 Compatibility Testing was performed using the following:

    Android Tag Android Build ID Mali Driver Android Compatibility Test Suite Android Vendor Test Suite
    android-12 SP1A.210812.003 r34p0_01eac0 12_r1 (eng.upr473.20210901.005349) 12_r1 (eng.upr473.20210901.024841)
    android-12 SP1A.210812.003 r32p1_01eac0 12_r1 (eng.upr473.20210901.005349)1 12_r1 (eng.upr473.20210901.024841)

    1: CtsNNAPITestCases with Mali Driver r32p1_01eac0. The following test is known to be failing: AddTwoWithHardwareBufferInputWithGPUUsage. Investigations indicate this failure is due to Android NN HAL utilizing Gralloc functionality not required by the Gralloc API. This issue has been raised with Google Android team, and is tracked as https://partnerissuetracker.corp.google.com/issues/202025253. Please quote Arm reference MIDCET-3783 when discussing this issue.

    Android 11 Compatibility Testing was performed using the following:

    Android Tag Android Build ID Mali Driver Android Compatibility Test Suite Android Vendor Test Suite
    android-11.0.0_r6 RPM1.210413.002 r33p0_01eac0 11_r3 (7127450) 11_r3 (7137996)

    Android 10 Compatibility Testing was performed using the following:

    Androidtag Android Build ID Mali Driver
    android-10.0.0_r39 QQ3A.200605.002.A1 R23P0_01REL0
    Check out latest releases or
    releases around ARM-software/armnn v21.11

    Don't miss a new armnn release

    NewReleases is sending notifications on new releases.

    Get notifications