github mostlygeek/llama-swap v151

latest releases: v157, v156, v155...
18 days ago

Changes:

Pre-loading of models with hooks

  • Using hooks.on_startup.preload a set of models can be automatically started on startup.
# hooks: a dictionary of event triggers and actions
# - optional, default: empty dictionary
# - the only supported hook is on_startup
hooks:
  # on_startup: a dictionary of actions to perform on startup
  # - optional, default: empty dictionar
  # - the only supported action is preload
  on_startup:
        # preload: a list of model ids to load on startup
        # - optional, default: empty list
        # - model names must match keys in the models sections
        # - when preloading multiple models at once, define a group
        #   otherwise models will be loaded and swapped out
    preload:
      - "llama"

Prompt Processing Metrics added to Activities page in UI

image

Changelog

  • 5dc6b3e Add barebones but working implementation of model preload (#209, #235)
  • 74c69f3 Add prompt processing metrics (#250)
  • a186318 Update Readme, Add screenshot for Activities page [skip ci]
  • c4e4d5e Update Readme UI Screenshot [skip ci]

Don't miss a new llama-swap release

NewReleases is sending notifications on new releases.