broadinstitute/cromwell 67

on GitHub

67 Release Notes

Configuration updates for improved scaling

Some configuration changes were introduced in Cromwell 67 to support improved scaling. See Cromwell's reference.conf for details on new parameters.

• I/O throttling moved from io to its own io.throttle stanza; config updates may be required if these values are currently being overridden in local deployments.

• The default system.job-rate-control has been changed from 50 per second to 20 per 10 seconds.

• New configuration parameters have been introduced for values which were previously hardcoded constants:

  • system.file-hash-batch-size, value updated from 100 to 50.
  • io.gcs.max-batch-size, value stays the same at 100.
  • io.gcs.max-batch-duration, value stays the same at 5 seconds.

• New configuration parameters which should not require updating:

  • io.command-backpressure-staleness
  • io.backpressure-extension-log-threshold
  • load-control.io-normal-window-minimum
  • load-control.io-normal-window-maximum

• io.nio.parallelism was previously misspelled in reference.conf but not in Cromwell's configuration reading code. Only correct spellings of this configuration key had or will have effect.