google-ai-edge/mediapipe v0.10.2

MediaPipe v0.10.2

on GitHub

Build changes

• Added gesture_recognizer.task to vision tasks test data

Framework and core calculator improvements

• Log the Bazel build
• Fix tests to work with arch haswell/sandybridge.
• Update base audio/vision tasks api to suuport proto3 graph options.
• Add an option to disable explicit CPU sync for ExternalTextureRenderer
• Add support for int64 constant side package value.
• Deprecate GraphStatus()
• Modify the TensorToImageFrameCalculator to support normalized outputs.
• Add metadata for all PREFIX/image... prefixes.
• Update Tensorflow dependency in MediaPipe
• update face drawing function.
• Speed up TimeSeriesFramerCalculator.
• Add MatrixData as a packet option for ConstantSidePacketCalculatorOptions.
• Fix timestamp computation when copying within first block.
• Fix -Wsign-compare warning in api2/builder.h
• Shows the recently added warning when WaitUntilIdle is called with source nodes only once. Otherwise, it is very spammy as it's shown every frame. Moreover, display the names of the sources, so the warning is more actionable.
• Exposes OpenCV photo lib.
• Add keys for the context that better match the featurelist for text.
• Do not send PreviousLoopback output packets to closed streams
• Add gpu to cpu fallback for tensors_to_detections_calculator.
• Revert Add location info in registry (debug mode only)
• Fix bounds calculation in RefineLandmarksFromHeatMapCalculator
• Add concatenate image vector calculator

MediaPipe Tasks update

This section should highlight the changes that are done specifically for any platform and don't propagate to other platforms.

Android

• Add delegate options to base options for java API and add unit tests for BaseOptions.

iOS

• Added iOS Gesture Recognizer Protobuf utils
• Added iOS Gesture Recognizer ObjC Test for simple recognition
• Added more recognize tests to iOS Gesture Recognizer Objective C tests
• Added convenience method for creating results for tests in MPPGestureRecognizerResult Helpers
• Added MPPHandLandmarkerResult Helpers
• Added MPPConnection
• Added MPPHandLandmarker
• Added MPPHandLandmark
• Renamed iOS gesture recognizer protobuf utils
• Fixed import in iOS gesture recognizer test utils
• Updated MPPHandLandmarker.h to return the hand connections via class mathods
• Add FaceLandmarker iOS API
• Add FaceLandmarker iOS Live Stream API
• Update iOS Gesture Recognizer error assertion
• Updated method names in MPPGestureRecognizer
• Rename MPPFaceLandmarker.m to MPPFaceLandmarker.mm
• Added hand landmarker implementation file and hand landmarker connections
• Updated constant names in MPPHandLandmarkConnections
• Add FaceLandmarker constants for iOS
• Added hand landmarker protobuf utils
• Added iOS Objective C hand landmarker tests
• Added iOS segmentation mask
• Updated documentation of MPPMask
• Added live stream mode tests for iOS Hand Landmarker
• Updated protobuf helper method name in iOS Gesture Recognizer Helpers
• Updated iOS hand landmarker tests
• Updated hand connections in iOS hand landmarker to class properties.
• Updated signature of initializer in MPPMask
• Fixed float calculations in MPPMask
• Removed generic methods for alloc and memcpy from MPPMask
• Updated init method implementations in MPPMask
• Fixed implementation of init methods in MPPMask
• Added MPPMask Tests
• Added iOS Image Segmenter Result
• Added iOS Image Segmenter Options
• Updated image segmenter delegate method to be required
• Added copying of running mode in NSCopying implementation in iOS tasks
• Added iOS Image Segmenter Options Helpers
• Added Image Segmenter Result Helpers

Javascript

• Add CommonJS bundle for MediaPipe Tasks
• Use .mjs for ESM Modules and use .cjs for CommonJS
• Add "exports" field definitions to package.json

Model Maker changes

• Use GFile for internal file systems.
• 1. Model maker core classifier change _metric_function field to _metric_functions in order to support having multiple metrics.
• Add a face alignment preprocessor to face stylizer.
• Support ExBert training and option to select between AdamW and LAMB optimizers for BertClassifier
• Add MobileNetV2_I320 and MobileNetMultiHWAVG_I384 to support larger input image sizes.

MediaPipe Dependencies

• Update WASM files for 0.10.2 release