letta-ai/letta 0.5.0

v0.5.0

on GitHub

This release introduces major changes to how model providers are configured with Letta, as well as many bugfixes.

🧰 Dynamic model listing and multiple providers (#1814)

Model providers (e.g. OpenAI, Ollama, vLLM, etc.) are now enabled using environment variables, where multiple providers can be enabled at a time. When a provider is enabled, all supported LLM and embedding models will be listed as options to be selected in the CLI and ADE in a dropdown.

For example for OpenAI, you can simply get started with:

> export OPENAI_API_KEY=...
> letta run 
   ? Select LLM model: (Use arrow keys)
   » letta-free [type=openai] [ip=https://inference.memgpt.ai]
      gpt-4o-mini-2024-07-18 [type=openai] [ip=https://api.openai.com/v1]
      gpt-4o-mini [type=openai] [ip=https://api.openai.com/v1]
      gpt-4o-2024-08-06 [type=openai] [ip=https://api.openai.com/v1]
      gpt-4o-2024-05-13 [type=openai] [ip=https://api.openai.com/v1]
      gpt-4o [type=openai] [ip=https://api.openai.com/v1]
      gpt-4-turbo-preview [type=openai] [ip=https://api.openai.com/v1]
      gpt-4-turbo-2024-04-09 [type=openai] [ip=https://api.openai.com/v1]
      gpt-4-turbo [type=openai] [ip=https://api.openai.com/v1]
      gpt-4-1106-preview [type=openai] [ip=https://api.openai.com/v1]
      gpt-4-0613 [type=openai] [ip=https://api.openai.com/v1]
     ... 

Similarly, if you are using the ADE with letta server, you can select the model to use from the model dropdown.

# include models from OpenAI 
> export OPENAI_API_KEY=...

# include models from Anthropic 
> export ANTHROPIC_API_KEY=... 

# include models served by Ollama 
> export OLLAMA_BASE_URL=...

> letta server

We are deprecating the letta configure and letta quickstart commands, and the the use of ~/.letta/config for specifying the default LLMConfig and EmbeddingConfig, as it prevents a single letta server from being able to run agents with different model configurations concurrently, or to change the model configuration of an agent without re-starting the server. This workflow also required users to specify the model name, provider, and context window size manually via letta configure.

🧠 Integration testing for model providers

We added integration tests (including testing of MemGPT memory management tool-use) for the following model providers, and fixed many bugs in the process:

• OpenAI #1827
• Anthropic #1837
• Azure OpenAI #1871 #1842
• Groq #1845
• Google AI (Gemini) #1850

📊 Database migrations

We now support automated database migrations via alembic, implemented in #1867. You can expect future release to support automated migrations even if there are schema changes.

What's Changed

• feat: add back support for using AssistantMessage subtype of LettaMessage by @cpacker in #1812
• feat: Add Groq as provider option by @mattzh72 in #1815
• chore: allow app.letta.com access to local if user grants permission by @4shub in #1830
• feat: Set up code scaffolding for complex e2e tests and write tests for OpenAI GPT4 endpoint by @mattzh72 in #1827
• feat: require LLMConfig and EmbeddingConfig to be specified for agent creation + allow multiple simultaneous provider configs for server by @sarahwooders in #1814
• test: Add complex e2e tests for anthropic opus-3 model by @mattzh72 in #1837
• refactor: remove get_current_user and replace with direct header read by @cpacker in #1834
• Docker compose vllm by @hitpoint6 in #1821
• fix: Fix Azure provider and add complex e2e testing by @mattzh72 in #1842
• fix: patch user_id in header by @cpacker in #1843
• feat: add agent types by @vivek3141 in #1831
• feat: list out embedding models for Google AI provider by @sarahwooders in #1839
• test: add complex testing for Groq Llama 3.1 70b by @mattzh72 in #1845
• feat: persist tools to db when saving agent by @vivek3141 in #1847
• feat: list available embedding/LLM models for ollama by @sarahwooders in #1840
• feat: Add listing llm models and embedding models for Azure endpoint by @mattzh72 in #1846
• fix: remove testing print by @mattzh72 in #1849
• fix: calling link_tools doesnt update agent.tools by @vivek3141 in #1848
• fix: refactor Google AI Provider / helper functions and add endpoint test by @mattzh72 in #1850
• fix: factor out repeat POST request logic by @mattzh72 in #1851
• fix: CLI patches - patch runtime error on main loop + duplicate internal monologue by @cpacker in #1852
• test: add complex gemini tests by @mattzh72 in #1853
• chore: deprecate letta configure and remove config defaults by @sarahwooders in #1841
• chore: add CLI CI test by @mattzh72 in #1858
• fix: insert_many checks exists_ok by @vivek3141 in #1861
• fix: delete agent-source mapping on detachment and add test by @sarahwooders in #1862
• feat: cleanup display of free endpoint by @sarahwooders in #1860
• fix: add missing hardcodings for popular OpenAI models by @cpacker in #1863
• chore: fix branch by @sarahwooders in #1865
• chore: add e2e tests for Groq to CI by @mattzh72 in #1868
• test: Fix Azure tests and write CI tests by @mattzh72 in #1871
• chore: support alembic by @4shub in #1867
• fix: fix typo by @kl2806 in #1870
• feat: add VLLMProvider by @sarahwooders in #1866
• fix: Fix config bug in alembic by @mattzh72 in #1873
• fix: patch errors with OllamaProvider by @cpacker in #1875
• refactor: simplify Agent.step inputs to Message or List[Message] only by @cpacker in #1879
• feat: Enable adding files by @mattzh72 in #1864
• feat: refactor the POST agent/messages API to take multiple messages by @cpacker in #1882
• feat: Add MistralProvider by @mattzh72 in #1883

New Contributors

• @hitpoint6 made their first contribution in #1821
• @vivek3141 made their first contribution in #1831
• @kl2806 made their first contribution in #1870

Full Changelog: 0.4.1...0.5.0