dagu-org/dagu v2.0.2

on GitHub

Changed

• OS Environment Variable Expansion: OS environment variables (e.g., $HOME, $PATH) are no longer expanded by Dagu for non-shell executor types (docker, http, ssh, jq, mail, s3, redis, etc.) and DAG-level configuration fields (ssh, smtp, s3, registry_auths). Only variables explicitly defined in the DAG scope (env:, params:, secrets:, step outputs) are expanded. OS variables pass through unchanged, letting the target environment (container, remote shell, etc.) resolve them. Shell command execution is unaffected. See RFC 007.

  To use a local OS variable in a config field, explicitly import it via the env: block:

  env:
    - HOME_DIR: ${HOME}  # Import OS $HOME into DAG scope
  
  ssh:
    user: deploy
    host: app.example.com
    key: ${HOME_DIR}/.ssh/deploy_key  # Expanded — HOME_DIR is DAG-scoped

• Auth by Default: The default authentication mode changed from none to builtin. New installations require creating an admin account via the /setup page on first visit. The JWT token secret is auto-generated and persisted to {dataDir}/auth/token_secret if not explicitly configured. See RFC 018.

• DAG Type Validation: DAGs with type: chain (the default) no longer allow the depends field on steps. Chain execution runs steps sequentially in definition order, making explicit dependencies redundant. To use the depends field for custom execution order, set type: graph explicitly. This change prevents confusion between chain's implicit sequential ordering and graph's explicit dependency-based execution.

Removed

+- Legacy API: The old legacy API routes have been removed. All API access uses the current /api/v1 endpoints, which require authentication when builtin auth is enabled.

• Static API Token (auth.token): The auth.token.value configuration field and DAGU_AUTH_TOKEN environment variable have been removed. Use API keys (builtin auth mode) or basic auth instead. API keys provide role-based access control and usage tracking. See API Keys for migration.

• Basic Auth Flat Fields: The top-level is_basic_auth, basic_auth_username, and basic_auth_password configuration fields have been removed. Use auth.mode: basic with auth.basic.username and auth.basic.password instead. The legacy DAGU_BASICAUTH_* environment variables have also been removed; use DAGU_AUTH_MODE=basic, DAGU_AUTH_BASIC_USERNAME, and DAGU_AUTH_BASIC_PASSWORD.

• Standalone OIDC Mode (auth.mode: oidc): Removed. The valid auth modes are now none, basic, and builtin. Use builtin + OIDC mode for SSO with user management and RBAC. See Builtin Authentication - OIDC/SSO.

• Admin Config (auth.builtin.admin): The auth.builtin.admin.username / auth.builtin.admin.password config fields and DAGU_AUTH_ADMIN_USERNAME / DAGU_AUTH_ADMIN_PASSWORD environment variables have been removed. The first admin account is now created via the /setup page on first browser visit.

• Basic Auth enabled Field: The auth.basic.enabled field and DAGU_AUTH_BASIC_ENABLED environment variable have been removed. Basic auth is activated by setting auth.mode: basic.

• Deprecated YAML Fields: The following deprecated fields have been removed from the YAML spec. Migrate to the replacement fields:

  Removed Field	Replacement	Context
  run	call	Step field for sub-DAG execution
  dir	working_dir	Step field for working directory
  executor	type + config	Step field for executor configuration
  precondition (singular)	preconditions (array)	Both DAG-level and step-level
  container.workDir	container.working_dir	Container working directory

Added

• Access Log Configuration: New access_log_mode setting controls HTTP request logging. Values: "all" (default — log all requests), "non-public" (skip public/static asset paths), or "none" (disable access logging). Environment variable: DAGU_ACCESS_LOG_MODE.

• Auth Mode Selection: Authentication mode is now configured via auth.mode field. Valid modes: none, basic, builtin (default). Basic auth uses auth.mode: basic with auth.basic.username and auth.basic.password. Environment variable: DAGU_AUTH_MODE.

• Coordinator Enabled Config: New coordinator.enabled config option (default: true) and DAGU_COORDINATOR_ENABLED environment variable to explicitly enable or disable the coordinator service. When disabled, start-all skips the coordinator and DAGs are never dispatched to workers. Accepts true/false/1/0.

• Self-Upgrade Command: New dagu upgrade command for in-place binary updates with SHA256 verification, backup support, and cross-platform compatibility.

• Literal Dollar Escape for Non-Shell Executors: Use \$ to emit a literal $ in non-shell contexts (docker, http, ssh, jq, mail, etc.) and config fields. Shell-executed commands preserve native semantics. To emit a literal $$ in non-shell contexts, escape both dollars: \$\$.

• Unified Execution Dispatch (defaultExecutionMode): New server-level defaultExecutionMode setting controls whether DAGs run locally or are dispatched to workers. When set to distributed, all DAGs are automatically dispatched to the coordinator for worker execution, even without an explicit worker_selector. DAGs that must remain on the main instance can use worker_selector: local to override this behavior.

• Agent: AI assistant integrated into the Web UI for workflow management. The agent helps create, review, debug, and manage DAG workflows through an interactive chat interface. Supports multiple LLM providers (Anthropic, OpenAI, Google, OpenRouter, local models). Features include shell command execution with approval for dangerous operations, file reading/editing, DAG schema lookup, UI navigation, and web search.

• Agent Delegate Tool: New delegate tool for the agent that spawns up to 8 parallel sub-agents for independent sub-tasks. Each sub-agent runs in its own session with the same tools (minus delegate to prevent recursion) and returns a summary to the parent. Sub-agent messages stream through the parent SSE connection.

• Trigger Type Visibility: DAG runs now display how they were initiated (scheduler, manual, webhook, subdag, retry). The trigger type is shown in the DAG runs list and detail views with distinct icons and labels. Available via the triggerType field in the API response.

• CLI History Command: New dagu history command displays execution history for DAG runs with comprehensive filtering (by date, status, tags, run ID), pagination (limit support up to 1000 results), and multiple output formats (table, JSON, CSV).

• LLM Model Fallback: model field accepts array of model objects. First is primary, rest are fallbacks tried in order on any error. Per-model overrides for temperature, max_tokens, top_p, base_url, api_key_name.

• Container Shell Wrapper: New shell field wraps step commands with a shell interpreter, enabling pipes, redirects, and command chaining without manual wrapping. Available in both image and exec modes.

  container:
    image: alpine:latest
    shell: ["/bin/sh", "-c"]
  
  steps:
    - command: cat file.txt | grep error | wc -l
    - command: npm install && npm test

• Step Defaults (defaults): New DAG-level defaults field that defines default values inherited by every step and handler_on step. Supports 8 fields: retry_policy, continue_on, repeat_policy, timeout_sec, mail_on_error, signal_on_stop, env, preconditions.

  defaults:
    retry_policy:
      limit: 3
      interval_sec: 5
    env:
      - LOG_LEVEL: info
  
  steps:
    - name: fetch-data
      command: curl https://api.example.com/data
      # Inherits retry_policy and LOG_LEVEL from defaults
  
    - name: custom-step
      command: ./run.sh
      retry_policy:
        limit: 1
        interval_sec: 0
      # Overrides retry_policy; still gets LOG_LEVEL from defaults

• Major UI Redesign: Complete redesign of the user interface with improved dark mode support, modernized color palette, and streamlined navigation. Enhanced visual hierarchy across all pages including DAG lists, execution views, system status, and admin pages.

• LLM Secret Masking: Secrets defined in the secrets block are now automatically masked before being sent to LLM providers in chat steps. This prevents accidental exposure of sensitive values to external AI APIs while still allowing secrets to be used in message content via ${VAR} substitution.

• LLM Tool Calling: Chat executor now supports function calling / tool use, enabling AI agents to execute workflows as tools during sessions. Tools are defined as DAGs with automatic parameter discovery from defaultParams.

  steps:
    - type: chat
      llm:
        provider: anthropic
        model: claude-sonnet-4-20250514
        tools:
          - search_tool
        max_tool_iterations: 10
      messages:
        - role: user
          content: "What's the latest news about AI?"
  
  ---
  name: search_tool
  description: "Search the web for information"
  defaultParams: "query max_results=10"
  
  steps:
    - command: echo "Searching for: $1"
    - command: curl "https://api.example.com/search?q=$1&limit=$2"
      output: SEARCH_RESULT

• Tailscale Tunnel: Built-in remote access via embedded Tailscale node. Access Dagu from anywhere without port forwarding or VPN setup.

  dagu server --tunnel

  Modes:

  • --tunnel - HTTP on tailnet (WireGuard encrypted, no setup)
  • --tunnel --tunnel-https - HTTPS on tailnet (requires HTTPS enabled in Tailscale admin)
  • --tunnel --tunnel-funnel - Public internet access (requires Funnel enabled in Tailscale admin)

• System Status Page: New admin-only page consolidating system health monitoring in one place. Includes scheduler service status, coordinator service status, worker status with health/pollers/running tasks, and resource usage charts (CPU, Memory, Disk, Load).

• OIDC Integration for Builtin Auth (Recommended): Added OIDC/SSO login capability under builtin authentication mode. This is now the recommended way to use OIDC with Dagu, as it combines SSO convenience with full user management and RBAC. Key features: auto-signup on first login, role mapping from IdP groups, email domain filtering, email whitelist, customizable login button.

• Synchronous Execution API: New endpoint POST /api/v1/dags/{fileName}/start-sync that executes a DAG and waits for completion before returning. Returns full execution details including all node statuses. Useful for automation scripts, CI/CD pipelines, and any scenario where you need to wait for a DAG to finish.

  curl -X POST "http://localhost:8080/api/v1/dags/my-dag/start-sync" \
    -H "Content-Type: application/json" \
    -d '{"timeout": 300, "params": "{}"}'

• SFTP Executor: New step type for transferring files between local and remote servers via SFTP. Supports upload and download of files and directories, with atomic uploads using temporary files to prevent partial transfers.

  ssh:
    user: deploy
    host: server.example.com
  
  steps:
    - name: upload-config
      type: sftp
      config:
        direction: upload
        source: /local/config.yaml
        destination: /remote/config.yaml

• SSH Bastion Host Support: SSH executor now supports connecting through a jump/bastion host for accessing servers in private networks.

• SSH Connection Timeout: Added timeout field to SSH configuration (default: 30s) for controlling connection timeouts.

• S3 Executor: New step type for S3 operations with support for AWS S3 and S3-compatible services (MinIO, Google Cloud Storage, DigitalOcean Spaces, Backblaze B2). Operations: upload, download, list, delete. DAG-level connection defaults. Automatic multipart uploads for large files.

  s3:
    region: us-east-1
    access_key_id: ${AWS_ACCESS_KEY_ID}
    secret_access_key: ${AWS_SECRET_ACCESS_KEY}
    bucket: my-bucket
  
  steps:
    - name: upload-report
      type: s3
      config:
        key: reports/daily.csv
        source: /tmp/report.csv
      command: upload

• SQL Executor: New step types for database operations with PostgreSQL and SQLite support. Execute queries, import data from CSV/TSV/JSONL, and export results in multiple formats. Parameterized queries for SQL injection prevention. Transaction support with isolation levels.

  steps:
    - name: query-users
      type: postgres
      config:
        dsn: "postgres://user:pass@localhost:5432/mydb"
      command: "SELECT * FROM users WHERE active = true"
      output: USERS

• Redis Executor: New step type for Redis operations with support for all major Redis commands, pipelines, transactions, Lua scripts, and distributed locking. Connection modes: Standalone, Sentinel, Cluster. TLS support.

  redis:
    host: localhost
    port: 6379
    password: ${REDIS_PASSWORD}
  
  steps:
    - name: cache-lookup
      type: redis
      config:
        command: GET
        key: user:${USER_ID}
      output: CACHED_USER

• PostgreSQL Connection Pool Management for Workers: Added global PostgreSQL connection pool configuration at the worker level to prevent connection exhaustion when multiple DAGs run concurrently in shared-nothing mode.

• DAG Runs Tag Filter: Filter DAG runs by tags in the UI and API. Select multiple tags to filter runs from DAGs that have ALL specified tags (AND logic). Available via the new tags query parameter on /api/v1/dag-runs endpoint (comma-separated).

• Key-Value Tags: Tags now support key-value pairs in addition to simple tags. Tag filtering supports four modes: exact key, exact key-value, negation, and wildcard patterns.

  tags:
    env: prod
    team: platform

• Tag Validation: Tags are validated at YAML load time. Keys must be 1-63 characters (alphanumeric, -, _, .), values 0-255 characters (alphanumeric, -, _, ., /).

• Shared-Nothing Worker Architecture: Workers can now operate without shared filesystem access. Enables deployment in Kubernetes, multi-cloud, and containerized environments where NFS/shared volumes are not available. Static discovery, status pushing, log streaming, and zombie detection.

• Web Terminal: Added optional web-based terminal for executing shell commands directly from the Dagu UI. Disabled by default for security. Enable via DAGU_TERMINAL_ENABLED=true.

• Audit Logging Configuration: Added configuration option to enable/disable audit logging. Enabled by default. Configure via DAGU_AUDIT_ENABLED=false.

• Git Sync: Synchronize DAG definitions with a Git repository.

• Catchup (Missed Run Replay): New catchup_window field enables automatic replay of missed cron runs when the scheduler restarts after downtime. The overlap_policy field ("skip", "all", or "latest") controls behavior when the DAG is still running during catchup.

• Unified Stop/Restart Scheduling: Stop and restart schedules are now evaluated by the same TickPlanner module that handles start schedules.

• Remote Agent Tool: New remote tool enabling task delegation to remote Dagu nodes. Agents can spawn and manage work across distributed instances with optional session ID support for idempotent session creation.

• Remote Node Management: Admin UI page for managing remote worker nodes with full CRUD operations and connection testing. REST API endpoints for remote node configuration.

• Document Management: Comprehensive document management system with full CRUD and search capabilities. Dedicated Docs page with tree navigation and Markdown rendering for project documentation alongside DAG workflows.

• Git-Sync Reconciliation: Enhanced Git-sync with forget, delete, and move operations for sync item management. Cleanup functionality to remove all missing items at once.

• License Management: License validation, status tracking, and feature gating system for RBAC and audit logging.

• Agent Personality (Soul): Agent personality management system with full CRUD capabilities. Users can select and customize agent personalities for chat sessions.

• Agent API Cost Tracking: Per-message cost tracking for LLM API calls with USD pricing metadata. Enhanced agent step message persistence for cost analysis.

• Provider-Native Web Search for Agent: Agents can use provider-native web search capabilities. Users can enable web search and configure maximum uses per request in agent settings.

• Singleton Enqueue Option: New singleton constraint for the /dag-runs/enqueue endpoint ensures only one DAG with the same name can be running or queued at any time. (#1672)

Deprecated

• DAG-level max_active_runs field: The max_active_runs field in DAG files is now deprecated for local (DAG-based) queues. Local queues now always use FIFO processing with concurrency of 1. For concurrency control, define global queues in ~/.config/dagu/config.yaml and assign DAGs using the queue field.

• max_active_runs: -1 (queue bypass): Setting max_active_runs to -1 to bypass queueing is deprecated. All DAGs now go through the queue system with local queues defaulting to FIFO (concurrency 1).

Fixed

• Sub-DAG Loading with Long Names: Fixed a bug where sub-DAG execution failed for DAG names near the 40-character limit.

• Sub-DAG Spec View: Fixed "file not found" error when viewing the spec tab for sub-DAG runs in the UI.

• Container Step Output Capture: Fixed an issue where container.command was not executed when specified inside the container block without a top-level command field.

• Sub-DAG Error Masking in Parallel Execution: Fixed an issue where sub-DAG failures produced the cryptic error "no results available for node status determination" instead of the actual cause.

• Windows Process Timeout and Subprocess Termination: Fixed timeout_sec not being enforced on Windows. (#1635)

• Security: Path Traversal in DAG Creation API: DAG creation endpoint now validates names to prevent directory traversal attacks. (GHSA-6v48-fcq6-ff23, #1691)

• Docker HOME Folder Substitution: Added -H flag to sudo in entrypoint.sh so ~ correctly expands to /home/dagu instead of /root inside Docker containers. (#1699)

• Cache Memory Bloat: Optimized cache implementation with more efficient eviction strategy to prevent excessive memory usage. (#1679)

• Webhook Fallback Body: Webhooks now support arbitrary JSON payloads with intelligent format detection and automatic fallback handling. (#1668)

• Queue Processing Stability: Enhanced queue processor stability with improved error handling and recovery mechanisms for DAG file operations during concurrent processing. (#1595)

• Variable Expansion for Unknown Variables: Variable expansion now preserves unknown or undefined variables in their literal form instead of expanding to empty strings. (#1606)

• Scheduler Queue Capacity on Retry: Retries for DAGs using global queues are now properly enqueued instead of executed immediately, respecting queue capacity limits. (#1676)

• Working Directory Resolution: Enhanced working directory resolution to properly inherit from base configuration when not explicitly specified in DAG files. (#1641)

• Upgrade Command Stability: Improved upgrade download reliability with retry logic and exponential backoff. Strengthened Windows binary replacement. (#1646)

• Parallel + Call Parameter Splitting: Fixed expanded variable references being incorrectly split during parallel step execution with call. (#1665)

• Environment Variable Expansion for Non-Unix Shells: Improved variable expansion handling for edge cases involving single quotes and adjacent characters on Windows PowerShell and cmd. (#1666)

• Start Command Parameter Validation: Support for JSON-formatted parameters when executing DAG commands. Enhanced parameter validation for the start CLI command. (#1663)

• Frontend Build Segfaults: Replaced TerserPlugin with esbuild for faster, more reliable frontend builds. (#1645)

• Tag Sorting: Alphabetical sorting of tag filter combobox and DAG table tags list for better usability. (#1617)

• Queue Count Display: Fixed queue status reporting to exclude currently running items from the queued count. (#1602)

Contributors

This release represents a massive effort with 108 merged pull requests since v1.30.3. We are deeply grateful to every contributor who helped make Dagu v2 a reality through code, bug reports, feature requests, reviews, and community support.

Special Thanks to @ghansham and @kriyanshii for providing many valuable feedback and ideas throughout the v2.0.0 development cycle. Their consistent engagement across issues, pull requests, and discussions has been instrumental in shaping this release.

Code Contributors

Contributors who authored merged pull requests for v2.0.0:

Bug Reporters

Contributors who identified and reported bugs that were fixed in v2.0.0:

Feature Requesters

Contributors who proposed features that were implemented in v2.0.0:

Community Support and Reviewers

Contributors who provided code reviews, helped diagnose bugs, answered questions, and supported the community throughout the v2.0.0 development cycle:

Historical Code Contributors

Contributors who authored code in previous releases that forms the foundation of v2.0.0:

@ArseniySavin,
@ddddddO,
@garunitule,
@Kiyo510,
@Lewiscowles1986,
@liooooo29,
@rafiramadhana,
@RamonEspinosa,
@rocwang,
@stefaan1o,
@x2ocoder,
@x4204,
@fishnux,
@triole,
@yarikoptic,
@zph,
@arky,
@Arvintian,
@jerry-yuan,
@jonnochoo,
@lvoeg,
@reneleonhardt,
@david-waterworth,
@Tagnard,
@thefishhat,
@AdityaTel89,
@Sarvesh-11,
@SiwonP,
@christinoleo

Community Participants

Everyone who participated in discussions, reported feedback, or helped other users during the v2.0.0 cycle:

@2012ZGZYY,
@accforgithubtest,
@admerzeau,
@agajic-modoolar,
@alangrafu,
@alext-extracellular,
@alfhj,
@alienscience,
@aptemus,
@AX-AMote,
@bellackn,
@bielids,
@biraj21,
@borestad,
@bremyozo,
@cernoel,
@chrishoage,
@CMiksche,
@codinggeeks06,
@Daffdi,
@DarkWiiPlayer,
@dat-adi,
@dAtBigFish,
@dev-a,
@dmitriy-b,
@don-philipe,
@eerison,
@erwan-airone,
@eugenechyrski,
@fbartels,
@frafra,
@georgeck,
@GhisF,
@gyger,
@hbina,
@helmut72,
@hgeritzer,
@HtcOrange,
@iainad,
@imkebe,
@jarnik,
@jeremydelattre59,
@jhuang732,
@JohnMatthiasWabwire,
@jonasban,
@jonathonc,
@jrisch,
@JuchangGit,
@jyroscoped,
@kacamific,
@kachida,
@kamandir,
@kylejbrk,
@lnlion,
@mdanilakis,
@Mice7R,
@mingjianliu,
@mitchplze,
@mnmercer,
@Netmisa,
@nicokant,
@nightly-brew,
@normal-coder,
@overflowy,
@Pangolin2097,
@peterbuga,
@piotrwalkusz1,
@plc-dev,
@pratio,
@rrottmann,
@saishreyakumar,
@samuelgodoy,
@sascha-andres,
@Sedymariama,
@semyon-t,
@SGRelic,
@SoarinFerret,
@tapir,
@tetedange13,
@tguructa,
@thibmart1,
@topjor,
@TrezOne,
@Vad1mo,
@vhespanha,
@volong113322-tech,
@wakatara,
@xinxinxinye,
@yangkghjh,
@ylaizet,
@yosefy,
@yurivish,
@ZivenLu,
@zobzn