google/mediapipe v0.10.11

MediaPipe v0.10.11

on GitHub

Build changes

• Updated genai C package visibility

Framework and core calculator improvements

• Prevent UnpackMediaSequenceCalculator from segfaulting on a type of malformed input.
• Updated import statements of llm_inference_engine.h to support C
• Refactor OpenGL 3.1 path out of TensorsToSegmentationCalculator main file.
• Add 'addRawDataSpanToInputSidePacket' and addRawDataSpanToInputStream binding functions.
• Update DotAttention interface to take SelfAttentionWeights
• Remove customized DotAttention
• Update Tensorflow dependency to latest release
• Update InferenceCalculator documentation on DELEGATE side input.

MediaPipe Tasks update

This section should highlight the changes that are done specifically for any platform and don't propagate to other platforms.

Android

• Add LlmInference stats logging
• Changed max_sequence_length naming to max_tokens

iOS

• Added iOS LlmInferenceError, LlmInference, LlmTaskRunner
• Removed init with default params in LlmInferenceOptions
• Updated iOS task runner to delete C LLM Session on deallocation
• Updated variable names in iOS LlmTaskRunner
• Updated access of C LLMSession to fileprivate
• Updated access of some constants in iOS LlmInference
• Removed unwanted iOS BUILD targets
• Added iOS Gen AI build scripts
• Added iOS Gen AI files
• Updated parameter names in LlmSessionConfig create
• Added asynchronous predict and generate function to iOS LLM Task Runner
• Removed decoded response from iOS LlmTaskRunner
• Updated iOS error enum cases
• Updated response generation state logic in iOS LlmInference
• Fixed error handling in iOS LlmInference
• Updated error message in iOS GenAiInferenceError
• Fixed unitialized response array in iOS LlmTaskRunner
• Added podspec templates of the iOS Gen AI framework

Javascript

• Add visibility field in landmark.d.ts
• Update landmark_result.ts with visibility support

Python

• Register model_ckpt_util in Python framework
• Expose HolisticLandmarker module as other modules
• Create empty module if ENABLE_ODML_CONVERTER is not set
• Optimized memory usage for conversion script

Model Maker changes

• Remove jax and torch from model maker requirements.txt
• Use tf model optimization < 0.8.0 due to tf.keras and tf_keras compat issues

MediaPipe Dependencies

• Update WASM files for 0.10.11 release