github invoke-ai/InvokeAI v5.6.0
on GitHub

one day ago

This release brings major improvements to Invoke's memory management, new Blur and Noise Canvas filters, and expanded batch capabilities in Workflows.

Memory Management Improvements (aka Low-VRAM mode)

The goal of these changes is to allow users with low-VRAM GPUs to run even the beefiest models, like the 24GB unquantised FLUX dev model.

Despite the focus on low-VRAM GPUs and the colloquial name "Low-VRAM mode", most users benefit from these improvements to Invoke's memory management.

Low-VRAM mode works on systems with dedicated GPUs (Nvidia GPUs on Windows/Linux and AMD GPUs on Linux). It allows you to generate even if your GPU doesn't have enough VRAM to hold full models.

Low-VRAM mode involves 4 features, each of which can be configured or fine-tuned:

  • Partial model loading
  • Dynamic RAM and VRAM cache sizes
  • Working memory
  • Keeping a copy of models in RAM

Most users should only need to enable partial loading by adding this line to their invokeai.yaml: 

enable_partial_loading: true

🚨 Windows users should also disable the Nvidia sysmem fallback.

For more details and instructions for fine-tuning, see the Low-VRAM mode docs.

Thanks to @RyanJDick for designing and implementing these improvements!

Workflow Batches

We've expanded the capabilities for Batches in Workflows:

  • Float, integer and string batch data types
  • Batch collection generators
  • Grouped (aka zipped) batches

Float, integer and string batch data types

There's a new batch node for each of the new data types. They work the same as the existing image batch node.

image

You can add a list of values directly in the node, but you'll probably find generators to be a nicer way to set up your batch.

Batch collection generators

These are essentially nodes that run in the frontend and generate a list of values to use in a batch node. Included in the release are these generators:

  • Arithmetic Sequence (float, integer): Generate a sequence of count numbers, starting from start, that increase or decrease by step.
  • Linear Distribution (float, integer): Generate a distribution of count numbers, starting with start and ending with end.
  • Uniform Random Distribution (float, integer): Generation a random distribution of count numbers from min to max. You can set a seed for reproducible sequences.
  • Parse String (float, integer, string): Split the input on the specified string, parsing each value as a float, integer or string. You can load the input from a .txt file. Use \n as the split string to split on new lines.

Screen.Recording.2025-01-21.at.9.27.05.pm.mov

You'll notice the different handle icon for batch generators. These nodes cannot connect to non-batch nodes, which run in the backend.

Grouped (aka zipped) batches

When you use multiple batches, we run the graph once for every possible combination of values in the batch collections. In mathematical terms, we "take the Cartesian product" of all batch collections.

Consider this simple workflow that joins two strings:
image

We have two batch collections, each with two strings. This results in 2 * 2 = 4 runs, one for each possible combination of the strings. We get these outputs:

  • "a cute cat"
  • "a cute dog"
  • "a ferocious cat"
  • "a ferocious dog"

But what if we wanted to group or "zip" up the two string collections into a single collection, executing the graph once for each pair of strings? This is now possible - we can set both nodes to the same batch group:

image

This results in 2 runs, one for each "pair" of strings. We get these outputs:

  • "a cute cat"
  • "a ferocious dog"

You can use grouped and ungrouped batches arbitrarily - go wild! The Invoke button tooltip lets you know how many executions you'll end up with for the given batch nodes.

Keep in mind that grouped batch collections must have the same size, else we cannot zip them up into one collection. The Invoke button grey out and let you know there is a mismatch.

Details and technical explanation

On the backend, we first zip each group's batch collections into a single collection. Ungrouped batch collections remain as-is.

Then, we take the product of all batch collections. If there is only a single collection (i.e. a single ungrouped batch node, or multiple batch nodes all with the same group), the product operation outputs the single collection as-is.

There are 5 slots for groups, plus a 6th ungrouped option:

  • None: Batch nodes will always be used as separate collections for the Cartesian product operation.
  • Groups 1 - 5: Batch nodes within a given group will first be zipped into a single collection, before the the Cartesian product operation.

All Changes

Fixes

  • Fix issue where excessively long board names could cause performance issues.
  • Fix error when using DPM++ schedulers with certain models. Thanks @Vargol!
  • Fix (maybe, hopefully) the app scrolling off screen when run via launcher.
  • Fix link to Scale setting's support docs.
  • Fix image quality degradation when inpainting an image repeatedly.
  • Fix issue with transparent Canvas filter previews blend with unfiltered parent layer.

Enhancements

  • Support float, integer and string batch data types.
  • Add batch data generators.
  • Support grouped (aka zipped) batches.
  • Reduce peak memory during FLUX model load.
  • Add Noise and Blur filters to Canvas. Adding noise or blurring before generation can add a lot detail, especially when generating from a rough sketch. Thanks @dunkeroni!
  • Reworked error handling when installing models from a URL.
  • Updated first run screen and OOM error toast with links to Low-VRAM mode docs.
  • Add a small handful of nodes designed to support inpainting in workflows. See #7583 for more details and an example workflow.

Internal

  • Tidied some unused variables. Thanks @rikublock!
  • Added typegen check to CI pipeline. Thanks @rikublock!

Docs

  • Added stereogram nodes to Community Nodes docs. Thanks @simonfuhrmann!
  • Updated installation-related docs (quick start, manual install, dev install).
  • Add Low-VRAM mode docs.

Installing and Updating

The new Invoke Launcher is the recommended way to install, update and run Invoke. It takes care of a lot of details for you - like installing the right version of python - and runs Invoke as a desktop application.

Follow the Quick Start guide to get started with the launcher.

If you already have the launcher, you can use it to update your existing install.

We've just updated the launcher to v1.3.2. Review the launcher releases for a changelog. To update the launcher itself, download the latest version from the quick start guide - the download links there are kept up to date.

Legacy Scripts (not recommended!)

We recommend using the launcher, as described in the previous section!

To install or update with the outdated legacy scripts 😱, download the latest legacy scripts and follow the legacy scripts instructions.

What's Changed

New Contributors

Full Changelog: v5.5.0...v5.6.0

Check out latest releases or
releases around invoke-ai/InvokeAI v5.6.0

Don't miss a new InvokeAI release

NewReleases is sending notifications on new releases.

Get notifications