Bazel changes
- Bazel version upgrade to v6.1.1
- Update Halide build rules for MediaPipe to use Halide v15.0.1
- Use "x86_32" instead of "i386" for Bazel CPU ID
Framework and core calculator improvements
- Added MPPImageClassifierOptionsHelpers, TensorsToSegmentationCalculatorOptionsProto.java into tasks core's maven package, MPPObjectDetectorOptions, MPPObjectDetectorOptionsHelpers, MPPClassifierOptions, MPPGestureRecognizerOptions, MPPGestureRecognizerOptions.m, support for more standard scaling options in GlSurfaceViewRenderer
- Updated cosine similarity utility
- Added method to send packet map to C++ task runner
- Added methods to MPPVisionTaskRunner
- Added methods to MPPVisionPacketCreator
- Updated build targets of vision packet creator and task runner
- Added MPPImageClassifierResultHelpers, MPPImageClassifierOptionsHelpers
- Added MPPImageClassifier
- Updated method signature in MPPTaskRunner
- Added Face Detector implementation and tests
- Added the AudioRecord API
- Update audio_record_test.py
- Add FaceLandmarker C++ API
- Updated models
- Add the dataset module for face stylizer in model maker
- Update Node version to 16.19.0
- Add metadata writer for image segmentation
- Add Interactive Segmenter MediaPipe Task
- Add label_map filtering into filter_detection drishti calculator
- Add the source code TensorsToSegmentationCalculatorOptionsProto.java into tasks core's maven package
- Add ImageData output to GraphRunner
- Added MPPImage Utils for tests
- Added stream info for some modes in MPPImageClassifier
- Added flow limiting for live stream mode in MPPImageClassifier
- Add WebGLTexture output for ImageSegmenter
- Add face_landmarker to vision types
- Add a function to convert CoreAudio buffers into a MediaPipe time series matrix
- Add the model configuration and training hyperparameters for BlazeFaceStylizer
- Add landmarks smoothing filter when requested face num is 1
- Added MPPDetection
- Added MPPObjectDetectionResult
- Added MPPObjectDetectorOptions
- Added MPPObjectDetectorOptionsHelpers, MPPObjectDetectionResultHelpers, MPPDetectionHelpers
- Added MPPObjectDetector
- Add FrameBuffer view on ImageFrame
- Add EDGETPU_NNAPI delegate option in MediaPipe tasks API
- Added MPPLandmark
- Added MPPLandmarkHelpers
- Added MPPGestureRecognizerResult
- Added MPPGestureRecognizerOptions, MPPClassifierOptions
- Added EndLoopImageCalculator and FaceToRectCalculator
- Updated FaceStylizer API to align with the new Base Vision Task API changes
- Added some face landmarks constants
- Added pose landmarker C++ API
- Update TF version to 2023-04-12
- Added CoreAudio and MediaToolbox to BUILD file
- Update Flatbuffers to 23.1.21
- Updated error with info about unsupported mirrored orientations in MPPVisionTaskRunner
- Add VEC32F4 support to ImageFrame
- Add shaders that support better landscape rendering with GlSurfaceViewRenderer
- Update TensorsToFaceLandmarksGraph to support face mesh v2 model
- Add support for more standard scaling options in GlSurfaceViewRenderer
MediaPipe solutions update
This section should highlight the changes that are done specifically for any platform and don't propagate to
other platforms.
Android
- Add FaceDetector, Pose Landmarker, FaceLandmarker and FaceStylizer Java API
- Add getLabels to ImageSegmeter Java API
- Fix the vision tasks aar build rule to solve the "cannot find symbol" error:
- Add LabelMapProto.java source code to MediaPipe AAR
- Add interactive segmenter java API
- Add face landmarker and face geometry java lite proto source code into mediapipe tasks AAR
- Switch to use the
isPresent()API since theisEmpty()is only available since java 11: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Optional.html#isEmpty() - Update java image segmenter to always output confidence masks and optionally output category mask
- Adds a LanguageDetector Java API
- Update Java interactive segmenter to output both confidence masks and category mask optionally
iOS
- Changed method Updated method calls to process packet map in iOS text tasks
- Solve iOS build error for gpu_buffer.cc
- Fixed iOS running mode display strings
- Linked in Opencv iOS framework with vision tasks
- Added flow limiter calculator to iOS vision tasks
Javascript
- Add FaceLandmarker Web API
- Add the FaceStylizer Web API
- Add InteractiveSegmenter Web API
Python
- Python support for M1
- Added Interactive Segmenter Python API and some tests
- Expose face detector, face landmarker, face stylizer and interactive segmenter as MediaPipe Tasks Python API
- Enable TextClassifier and TextEmbedder on Windows Python
- Gracefully fail resource path lookup for Python on Windows
- Expose as mediapipe python API
- Make AudioTools compile when build from python:framework_bindings
Bug fixes
- Upgrades and fixes for image segmentation category mask on GPU
MediaPipe Dependencies
- Added dependency for image format
- Disable OpenCL dependency for OpenCV
- Add missing dependency library targets to mediapipe_task_aar