RubyLLM 1.1.0: Cloud Expansion, Smarter Conversations, Resilient Tooling, and More 🚀

We're thrilled to announce RubyLLM 1.1.0, a major update expanding cloud provider support, enhancing conversation management, and introducing a resilient error handling paradigm for tools.

🌩️ AWS Bedrock Support

Access Claude models through your existing AWS infrastructure:

RubyLLM.configure do |config|
  config.bedrock_api_key = ENV.fetch('AWS_ACCESS_KEY_ID')
  config.bedrock_secret_key = ENV.fetch('AWS_SECRET_ACCESS_KEY')
  config.bedrock_region = ENV.fetch('AWS_REGION')
  config.bedrock_session_token = ENV.fetch('AWS_SESSION_TOKEN') # optional
end

# Using Claude through Bedrock
chat = RubyLLM.chat(model: 'claude-3-5-sonnet', provider: 'bedrock')

Great for teams who want to keep everything within their existing AWS environment.

🧠 Simpler System Instructions

Introducing with_instructions for a cleaner way to set system prompts:

chat = RubyLLM.chat
  .with_instructions("You are a helpful coding assistant that specializes in Ruby")
  .ask("How would I implement a binary search tree?")

You can also explicitly replace previous instructions instead of accumulating them:

chat.with_instructions("New instructions", replace: true)

This works across all providers, with smart handling of differences between OpenAI, Anthropic, and others.

🛠️ Smarter Tool Error Handling

A completely new approach to handling errors in tools that lets LLMs handle recoverable issues while bubbling up critical errors to your application:

class Weather < RubyLLM::Tool
  description "Gets current weather for a location"
  param :latitude, desc: "Latitude (e.g., 52.5200)"
  param :longitude, desc: "Longitude (e.g., 13.4050)"
  
  def execute(latitude:, longitude:)
    validate_coordinates!(latitude, longitude)
    response = Faraday.get(weather_api_url(latitude, longitude))
    
    case response.status
    when 429
      # Return errors the LLM should know about and can retry
      { error: "Rate limit exceeded. Please try again in 60 seconds." }
    when 200
      JSON.parse(response.body)
    else
      # Let serious problems bubble up
      raise "Weather API error: #{response.status}"
    end
  end
end

When to Return vs Raise Errors

• Return errors to the LLM when:
  • Input validation fails
  • The operation can be retried (rate limits, temporary failures)
  • Alternative approaches might work
• Let errors bubble up when:
  • The tool encounters unexpected states
  • System resources are unavailable
  • Authentication or authorization fails
  • Data integrity is compromised

🔄 Configurable Retries

New configurable retry settings for API requests, making your applications more resilient:

RubyLLM.configure do |config|
  # Configure retry settings
  config.max_retries = 5                  # Default: 3
  config.retry_interval = 0.5             # Default: 0.1 (seconds)
  config.retry_backoff_factor = 3         # Default: 2
  config.retry_interval_randomness = 0.3  # Default: 0.5
end

The retry mechanism now includes:

• retry_interval: Initial wait time between retries
• retry_backoff_factor: Multiplier for subsequent retry intervals
• retry_interval_randomness: Adds jitter to prevent thundering herd problems

We've also expanded the retry status codes to handle service overload errors.

🔀 Smarter Model Resolution

Model identification is now much smarter:

• Model Aliases: Use simple names like claude-3-5-sonnet instead of full version strings
• Provider-Specific Matching: Target specific providers with the same base model
• Exact Match Priority: Exact matches are prioritized over aliases

# These all work now
chat = RubyLLM.chat(model: 'claude-3-5-sonnet')
chat = RubyLLM.chat(model: 'claude-3-5-sonnet', provider: 'anthropic')
chat = RubyLLM.chat(model: 'claude-3-5-sonnet', provider: 'bedrock')

🚂 Rails Enhancements

ActiveRecord integration improvements make Rails usage even better:

# Now all these methods return self for chainability
chat = Chat.create!(model_id: 'gpt-4o-mini')
  .with_instructions("You are a helpful assistant")
  .with_tool(Calculator)
  .ask("What's 123 * 456?")

🚿 Streaming Improvements

Fixed several streaming issues:

• Improved error parsing across all providers during streaming
• Fixed JSON parsing errors when using Gemini with tools and no arguments
• Enhanced multi-turn streaming conversations

🔧 Technical Improvements

• Fixed multimodal inputs for Bedrock Anthropic
• Improved system prompt handling across all providers
• Better error parsing for AWS Bedrock responses
• Fixed model refreshing for read-only filesystems in production
• Improvements to DeepSeek and Anthropic providers
• Support for parameterless tools

📖 Documentation Updates

• New comprehensive model information guide listing 100+ models with capabilities
• Updated error handling guidelines
• Working with multiple providers
• Using the model registry effectively
• Deploying in production environments

Install or update RubyLLM today:

gem 'ruby_llm', '~> 1.1.0'

Merged Pull Requests

• Support tools without params by @redox in #62
• Switched from git ls-files to Ruby's Dir.glob for defining spec.file by @seuros in #84
• 16: Add support for Claude through Bedrock by @tpaulshippy in #50
• Fix refresh of Bedrock models and remove some expired credentials from cassettes by @tpaulshippy in #89
• fix acts_as delegation to return self instead of RubyLLM by @kieranklaassen in #82
• Add a rake task to generate an 'available model information' markdown guide. by @keithrbennett in #71
• Parse errors from Bedrock properly by @tpaulshippy in #94

New Contributors

• @redox made their first contribution in #62
• @seuros made their first contribution in #84
• @tpaulshippy made their first contribution in #50
• @kieranklaassen made their first contribution in #82
• @keithrbennett made their first contribution in #71

Full Changelog: 1.0.1...1.1.0