Build changes
- Allow split_vector_calculator to be build with iOS and MEDIAPIPE_DISABLE_GPU
- Update mediapipe_aar.bzl to put more mediapipe framework java proto classes into AARs.
Bazel changes
Update Bazel dependencies for Apple
Framework and core calculator improvements
- Add HandLandmarkerGraph which connect HandDetectorGraph and HandLandmarkerSubgraph with landmarks tracking.
- Updated image classifier to use a region of interest parameter
- Add support for input image rotation in ImageClassifier and ObjectDetector C++ API
- Adding BypassCalculator for use with SwitchContainer.
- Add MergeDetectionsToVectorCalculator, CombinedPredictionCalculator, EndLoopMatrixCalculator, ConcatenateClassificationListCalculator, RegexPreprocessingCalculator and BERTPreprocessorCalculator, TextToTensorCalculator and UniversalSentenceEncoderPreprocessorCalculator
- Added the TextClassifier C++ API, the TextPreprocessingSubgraph.
- Rename "Bound" struct to "Rect" and remove unused "Landmark" struct.
- Add tensor_index and tensor_name fields to ClassificationList
- Replace numpy.float with the builtin float type as numpy removes its own float type in v1.24.
- Add BGR -> RGB color conversion to ColorConvertCalculator.
- Add SQRT_HANN window type to both SpectrogramCalculator and InverseSpectrogramCalculator.
- Allow conversion of GlTextureBuffer to CVPixelBufferRef. This means that, if an iOS application sends in a GlTextureBuffer but expects a CVPixelBufferRef as output, everything will work even if the graph just forwards the same input. Also, access by Metal calculators will also work transparently.
- Allowing BypassCalculator to accept InputSidePackets.
- Enable unsigned quantized infererence using XNNPACK.
- Adds a preprocessor for Universal Sentence Encoder models.
MediaPipe solutions update
Android
- Enable creating MediaPipe Image c++ packet directly from an Android media image object when its format is RGBA_8888.
- Add Java ImageEmbedder API and TextEmbedder API.
- Fix aar breakage caused by missing "//mediapipe/tasks/java/com/google/mediapipe/tasks/components/containers:normalized_landmark".
- Fix aar breakage caused by missing "//mediapipe/tasks/cc/vision/image_segmenter/proto:segmenter_options_java_proto_lite".
Web
- Hand Landmarker Web API
- Allow Web developers to opt into CPU or GPU processing
- Add support for browsers without SIMD
- Add pre-compiled WASM files to NPM packages
Bug fixes
- Fix RGBA vs RGB selection when creating GLTexture.
- Fix accidental suppressions of GLSL linker error reporting
- Fix for CHECK failure due to pointer description sometimes being larger than allocated string space
- ClassificationAggregationCalculator and EmbeddingAggregationCalculator now fill in the
timestamp_msfield of the classification results in the stream mode. - Fix ObjectDetector C++ flow limiter and improve documentation.
- Better handling of empty packets in vector calculators.
MediaPipe Dependencies
- Bump up the dependency library pybind11's version to 2.10.1.