Release Notes: GoMFT v0.1.8

⚠️ Breaking Change: Directory Structure Changes

Overview

This release introduces several significant new features and improvements to GoMFT, Webhook Notifications for real-time job status updates, Multiple Configurations per Job, Log Viewer, Multi-threaded File Transfers for improved performance, and comprehensive testing improvements.

New Features

Admin Tools

• System Dashboard: Monitor system health, database size, and job statistics
• Log Viewer: Browse, view, and download application log files directly from the UI
• Database Management: Create backups, restore from backups, and perform database optimization
• Configuration Import/Export: Import and export job and transfer configurations

Webhook Notifications

• Real-time HTTP notifications when jobs complete or fail
• Configurable webhook URLs and custom HTTP headers
• HMAC-SHA256 signing for webhook payload authentication
• Conditional notifications (success/failure)

Multi-threaded File Transfers

• Configurable concurrent transfer settings per transfer configuration
• Improved performance for large file transfers
• Default of 4 concurrent transfers per configuration

Multiple Configurations per Job

• Jobs can now use multiple transfer configurations
• Search and select configurations when creating/editing jobs
• Improved job form with configuration checkboxes

Logging Improvements

• Configurable log levels and rotation
• Structured logging with time, level, and component information
• Log rotation with size limits and archiving

Improvements

User Interface

• Enhanced form validation and error messages
• Improved help text and field descriptions
• Updated configuration and job creation workflows

Performance

• Optimized database queries
• Improved job scheduling and execution
• Better handling of large file transfers

Testing

• Comprehensive testing guide
• New test utilities and mocking strategies
• Increased test coverage across components
• Webhook and Admin Tool specific test cases

Breaking Changes

Database Schema

• Jobs Table: Added config_ids field replacing the single config_id
• Transfer Configs Table: Added max_concurrent_transfers field
• Jobs Table: Added webhook-related fields (webhook_url, webhook_secret, etc.)
• Migration scripts are included and automatically applied on startup, but database backups before upgrading are strongly recommended

Configuration Structure

• Transfer configurations now include max concurrent transfers setting
• Jobs now reference multiple configuration IDs instead of a single ID
• Webhook fields added to job configurations

Directory Structure

• Added separate backups directory for database backups
• Updated logging directory structure
• Modified default paths in configuration

Environment Variables

• Added LOGS_DIR for configuring log file location
• Updated DATA_DIR handling for improved flexibility
• Added BACKUPS_DIR for database backup location

Upgrade Guide

Before Upgrading

1. Back up your database
2. Back up your configuration files
3. Document your current environment variables

Upgrading Steps

1. Pull the latest code
2. Update environment variables as needed
3. Restart the application (database migrations will be applied automatically)

Post-Upgrade Verification

1. Verify existing jobs have been migrated correctly
2. Check transfer configurations for new concurrent transfer settings
3. Test webhook notifications if configured

Technical Details

New Dependencies

• github.com/joho/godotenv v1.5.1
• github.com/stretchr/testify v1.10.0
• gopkg.in/natefinch/lumberjack.v2 v2.2.1
• github.com/davecgh/go-spew v1.1.1
• github.com/pmezard/go-difflib v1.0.0
• github.com/stretchr/objx v0.5.2

Docker Volumes

• Added gomft-backups volume for database backups
• Updated volume mounts for logs directory