Bacalhau v1.7.0 Release Notes

This release introduces enterprise support options, enhanced partitioning capabilities, improved networking, comprehensive authentication, and significant WebAssembly enhancements.

Enterprise Support through Expanso

• Enterprise support now available through Expanso while maintaining Bacalhau's open source core
• Free tier for everyone supporting up to 5 nodes with all core features
• Enterprise licensing option for organizations requiring professional support

Expanso Cloud Enhancements

• Comprehensive job submission capabilities through the Expanso Cloud web interface
• Library of pre-defined job templates for common use cases:
  • Log processing workflows
  • Data analysis with DuckDB
  • Apache Iceberg operations
  • And more
• Custom template creation capabilities with parameterization
• Simplified complex distributed computing tasks while maintaining flexibility

Enhanced Job Execution with Partitioned Workloads

• Distribution of work across compute nodes with built-in partition management
• Horizontal scaling and parallel processing for improved performance
• Granular failure recovery with automatic retry mechanisms for failed partitions
• Independent partition execution with context through environment variables

S3 Storage Input Partitioning

• Automatic data distribution across multiple executions with graceful failure handling
• Multiple partitioning strategies:
  • Object-Based Distribution: Even distribution without specific grouping
  • Date-Based Partitioning: Parallel processing for time-series analysis
  • Regex Pattern Matching: Distribution by patterns in file paths
  • Substring-Based Partitioning: Organization by customer segments
• Combined partitioned data with shared reference data support

S3 Publishing Enhancements

• Plain encoding support for direct access to job outputs without compression
• Individual file publishing rather than compressed archives
• Efficient access to individual files with no decompression overhead
• Default gzip encoding still available for complete result sets

Enhanced Networking Capabilities

• Networking enabled by default (shifted from opt-in to opt-out)
• Configuration option for restricting network access
• Experimental port mapping support
• Multiple network modes: bridge (default), host, and none

Comprehensive Authentication and Authorization

• Redesigned authentication system with multiple methods:
  • Basic authentication
  • API tokens
  • Single Sign-On (SSO)
• Fine-grained authorization controls
• New CLI commands for authentication management

Host Environment Variable Forwarding

• Access to specified environment variables from the host system
• Secure credential passing through a controlled allowlist mechanism
• Reference host variables using env: prefix in job specifications

Improved WebAssembly Support

• Refactored WASM job specification with better separation of concerns
• Remote modules defined as input sources for alignment with other executors
• HTTP networking support for WebAssembly through new host function implementation
• Configurable network access modes and security controls
• TinyGo-compatible client library for HTTP functionality

Docker DinD Integration

• Docker-in-Docker images for simplified containerized deployment
• Two image variants:
  • Standard base image for client usage and orchestrator nodes
  • DinD image for compute nodes running Docker workloads
• Pre-configured Docker daemon for simple integration with container ecosystems

Performance and Quality of Life Improvements

• Enhanced resource management and more efficient state tracking
• count field now defaults to 1 instead of 0
• NetworkDefault type for automatic networking configuration

Documentation & Resources

Explore the complete release details including FAQs, implementation guides and enterprise licensing information in our comprehensive v1.7.0 documentation. The full documentation includes detailed answers to common questions about partitioning strategies, authentication methods, S3 integration, and enterprise support options.

New Contributors

• @virajbhartiya made their first contribution in #4848
• @haixuanTao made their first contribution in #4857
• @Olg-K made their first contribution in #4873

Full Changelog: v1.6.0...v1.7.0