1.4.0 Release Notes
This is a major update. Please read through the release notes and breaking changes especially. See the updated docs if you have any questions.
A huge thank you to everyone who helped test this version and shared their feedback!
Warning
The blueprint has been significantly restructured, which means you will have to set up some parts again. Due to many issues with the Frigate mode of this blueprint it has been removed. However, the blueprint is still compatible with Frigate: You will have to use the camera entities exposed by the Frigate integration as well as one of the binary_sensors for each camera.
Tip
The blueprint requires a separate update. You can update it by reimporting.
⚠️ Breaking Changes
input_mode(Blueprint) has been deprecated. The blueprint now only supports what was previously the camera mode. See above for how you can still use Frigate with this blueprint.detailhas been deprecated. To control the quality of images use thetarget_widthparameter which downscales images that are larger than the set width.expose_images_persisthas been deprecated.expose_imagesnow saves the image to/www/llmvision/<uid>-<key_frame>.jpg. This path is returned with the response variable in thekey_framekey. Use the newcleanupaction to remove unneeded images from time to time.
Features
Integration
- Timeline replaces Event Calendar: LLM Vision now has a frontend card for your dashboard to display events along with an image and a summary of the event. Get the card here: https://github.com/valentinfrlch/llmvision-card
- Added Memory: You can now store reference images along with a description in memory. This provides additional context to the model and can be used e.g. to provide the model with pictures of members of your household to identify them. Content in memory syncs across all providers (except Groq as it can only process one image per call). With this there is now also a
use_memorytoggle in all actions to control whether content stored in memory should be used or not.
Disclaimer: This is not a facial recognition model. AI can make mistakes. Do not use for unlocking doors or similarly critical actions! - Added customizable system prompt: In the memory settings, a system prompt has been added which can be customized as well. The default system prompt greatly improves accuracy compared to before. The system prompt applies to all providers and is sent even when
use_memoryis off. - Added customizable title prompt: The prompt used to generate titles for events can no be customized in the Memory settings.
- Added a new action
remember: Use this action to store custom events to the timeline (and the LLM Vision Card). - Added a new action
cleanup: Remove images in/www/llmvision/that are not associated with an event in the timeline.
Blueprint
Warning
See the breaking changes above before updating!
- Added a
run_condition: You can now add custom conditions to the blueprint. All conditions have to be true in order for the blueprint to run. Use this to only run the automation when no one is home or when the alarm is enabled. - Added
notification_delivery: Choose between ‘Dynamic’ and ‘Consolidated’ delivery modes. Dynamic will immediately send a notification with a live camera preview and update the notification silently when the summary is available. Consolidated will only send the notification once the summary is available. Use this if you got two separate notifications before. - Added
use_memory: Enables access to information stored in memory. Memory needs to be set up for this.