google-ai-edge/mediapipe v0.10.20

MediaPipe v0.10.20

on GitHub

Build changes

• Add comments to explain how to configure OpenCV in the opencv_macos.BUILD file.
• Add libc++_shared.so to MediaPipe Android examples.
• Add linkstatic to OpenCV prebuilts

Framework and core calculator improvements

• Fix ParseFromString() compilation issue in OSS
• All the dead links fixed
• Add troubleshooting tip for unsupported XNNPACK flags during build
• Add UniqueId::Dup.
• Updating the XNNPACK latest commit hash
• Format Workspace file
• Add EglSync wrapper.
• Update Bazel version to 6.5.0
• Update sync_wait to support UniqueFd.
• Fix GlContext includes
• Add EglSyncPoint/CreateEglSyncPoint.
• Bump MediaPipe version to 0.10.19.
• More perfetto tracking for EglSync.
• Patch for supporting WebGPU .deviceInfo during API migration.
• Log the Tensor multi-write error message only once.
• Enable GpuBufferStorageAhwb ASYNC usage for use case: AhwbView write -> GlTextureView read
• Add IsSignaled function (the previous SyncWait for checking status triggers unnecessary StrFormat)
• Add type information to error message when accessing an empty packet.
• Add SharedFD type
• Update SyncWait/IsSignaled to work with SharedFd.
• Enable SharedFd usage in EglSync
• Adding VLOG overrides - MediaPipe utilizes VLOG heavily, but it's not straightforward for how to enable this when running an Android app. VLOG overrides allow to relatively quickly enable VLOGs for various modules within MediaPipe.
• Updating Troubleshooting with VLOG info.
• Slice only the tokens which are needed for the next stage of the LLM pipeline.
• Adds DebugInputStreamHandler.
• Delete YUVImage copy and move operations
• Adds GetGraphRuntimeInfo methods which generates runtime debugging information about the state of InputStreams.
• Add a sample script to run LLM inference on Android via the MediaPipe LLM inference engine.
• Update bot_config.yml
• Add option to set max sequence size in PackMediaSequenceCalculator instead of having it hard coded.
• Update run_llm_inference.sh with recommended models.
• Allow to read the input frame rate from the header in the input side stream and to limit the frame rate.
• Add stream operator<< for TypeId
• Extract native-to-UTF8 path string conversion; add FormatLastError()
• Update comment in yuv_image.h
• Introduce shadow_copy parameter to PathToResourceAsFile
• Fix header includes after refactoring
• Migrate away from status builders
• Avoids the creation of two "default" GpuExecutor instances
• Adds and integrates GraphRuntimeInfoLogger into CalculatorGraph.
• nit: don't overwrite InitializeDefaultExecutor argument "use_application_thread"
• Add Name() access to the source names in api2.
• Add memory mapping and locking to file helpers
• Fix Windows build
• Fix Windows build, part 2
• Support memory mapping in resources.
• Bump MediaPipe version to 0.10.20.
• Enable log message output for messages larger than 4096 bytes.
• Add vision modality to the C API

MediaPipe Tasks update

This section should highlight the changes that are done specifically for any platform and don't propagate to
other platforms.

Android

• Adds the canonical toBuilder method to the LlmInferenceOptions object.
• Add Vision Modality to the MediaPipe LLM JNI Layer
• Add vision modality to the Java LLM API
• Remove unused Proto dependency

iOS

• Fixed empty pose world landmarks in iOS holistic landmarker

Javascript

• Improve logging to allow users to understand 1) which InferenceCalculator backend is used (without extra VLOG flags) and 2) when a model is loaded (including its size).
• nits: Remove linter warnings, fix unused includes.

Python

• Update the expected accuracy for text embedder test.
• Remove the check for start and stop tokens in the LLM bundler.

Model Maker changes

• Move tensorflow lite python calls to ai-edge-litert.

MediaPipe Dependencies

• Update WASM files