github concourse/concourse v7.4.0
on GitHub

latest releases: v7.11.2, v7.11.1, v7.11.0...
2 years ago

✈️ Features

  • Fly clear-resource-cache command (#7003) @EstebanFS 🔗

    • Added fly command clear-resource-cache, you could use this following the next format
      fly -t ci clear-resource-cache -r pipeline/resource [--version some:version]

  • Build page shows name of who triggered the build in header line of build page (#7112) @evanchaoli 🔗

    • The build page now shows the username of who triggers the build if the build is triggered manually.

  • Add page to view all builds/resource versions downstream/upstream from a root resource version (#7125) @chenbh 🔗

    • Disabled by default since computing causality for large datasets can be expensive, use --enable-resource-causality or $CONCOURSE_ENABLE_RESOURCE_CAUSALITY=true to enable the web UI and API endpoint.
      • Most datasets (like the merge commit for this PR) have < 100 builds and/or resource versions and take < 100ms, but it's possible for some "slow paced" resource versions (i.e. very infrequent new versions) to generate extremely large datasets
      • There is an automatic cutoff at 5000 builds or 25000 resource versions. On our deployment, the call for our slowest paced resource took about ~7 seconds to process, most of which is spent in the DB query
    • The causality page can be navigated to from the resource page
      Screen Shot 2021-06-03 at 11 37 08 AM
    • The causality page displays all the builds and resource versions that was generated from (downstream) or resulted in (upstream) the creation of a particular resource version
      Screen Shot 2021-06-03 at 11 25 03 AM
    • The downstream graph will put the root resource version on the left whereas the upstream graph will put it on the right
    • It takes into account all the intermediate resource versions when computing the final graph. In the picture above, while the resource page only shows that git version: 123 is a direct input to integrate #4 & #5, there is also an indirect link from git version: 123 -> test #19 -> ... -> intermediate-3 version:123 -> integrate #6 & #6.1

  • Support soft policy enforcement (#7139) @evanchaoli 🔗

    • This feature doesn't break the existing OPA policy check. If you have enabled OPA policy check, and you don't need "soft" policy enforcement, then you just don't need to do any configuration change.
    • 3 new ATC cli options are added:
      • CONCOURSE_OPA_RESULT_ALLOWED_KEY: specifies a key of allow flag in OPA returned result
      • CONCOURSE_OPA_RESULT_SHOULD_BLOCK_KEY: specifies a key of should-block flag in OPA returned result
      • CONCOURSE_OPA_RESULT_MESSAGES_KEY: specifies a key of messages in OPA returned result

    For example, if OPA returns the following result: 

    {
    "result": {
        "allow": true,
        "block": true,
        "reasons": ["foo", "bar"]
    }
}

    then CONCOURSE_OPA_RESULT_ALLOWED_KEY should be set to result.allow; CONCOURSE_OPA_RESULT_SHOULD_BLOCK_KEY should be result.block, and CONCOURSE_OPA_RESULT_MESSAGES_KEY should be result.reasons.

    NOTE: allow and block in OPA result should be boolean type, because it's easy to convert other types to boolean in an OPA policy.

  • Add ability to comment on a build (#7147) @multimac 🔗

    • You can now leave comments on builds. For instance, this can be used to give context to your coworkers about why a particular build failed:
      Screen Shot 2021-06-30 at 5 40 45 PM

    • If a build has a comment, it is displayed with a small marker to help you quickly find builds of interest. Hovering over the build displays a portion of the comment:
      Screen Shot 2021-06-30 at 5 41 52 PM

  • Add teamName to concourse_steps_wait_duration metrics (#7154) @Esysc 🔗

  • Use browser cache API for dashboard caching (#7247) @aoldershaw 🔗

    • The cached API responses on the dashboard no longer need to get truncated, which was previously introduced to work around localStorage limits

  • Allow interpolation in the across step values (#7252) @aoldershaw 🔗

    • The across step now supports dynamic interpolation of values. For instance, this can be combined with the set_pipeline step and instanced pipelines to set a dynamic list of pipelines: 
      - load_var: branches
  file: branches/branches.json
- across:
  - var: branch
    values: ((.:branches))
  set_pipeline: my-app
  file: ci/pipelines/my-app.yml
  instance_vars: {branch: ((.:branch))}

  • Cache the list of workers in memory (#7268) @aoldershaw 🔗

    • Scheduling containers should be more performant by reducing the number of required database calls

  • Optimize build log collector (#7327) @evanchaoli 🔗

    • Optimized a SQL statement used to remove build logs. This optimization will specially benefit large deployments that have a lot of pipelines.

  • Enable emitting dogstatsd metrics over uds (#7338) @jmhwang7 🔗

    • The Datadog emitter can now be configured to communicate with the Datadog agent over Unix Domain Sockets

🐞 Bug Fixes

  • containerd: properly populate /etc/hosts and /etc/hostname (#7041) @muntac 🔗

    • containerd: /etc/hosts and /etc/hostname are correctly populated

  • Handle 403 for vault preflight check of V2 (#7057) @xtremerui 🔗

  • atc: across step logs errors (#7090) @taylorsilva 🔗

    • Across step emits an error event when one of the sub-steps errors

  • containerd: Mount /dev/fuse to privileged containers (#7098) @aoldershaw 🔗

  • atc(fix): fixed a bug in resource check rate limiter. (#7102) @evanchaoli 🔗

    • Fixed a bug in check rate limiter that caused slow checks.

  • fix BaseResourceType for streamed volumes (#7108) @vito 🔗

  • Fix worker restart issue with containerd daemon and beacon (#7113) @muntac 🔗

    • Fix worker stall issue when restarting with containerd. Exit the worker's beacon process gracefully if any other top level process like the containerd daemon fails. Wait for containerd daemon to come up before starting the containerd Garden server.

  • Fix memory leak in notification bus (#7120) @aoldershaw 🔗

  • containerd: default to root if /etc/passwd is missing (#7124) @aoldershaw 🔗

    • Fixes a regression introduced in 7.3.0 that prevented containers that don't have an /etc/passwd file from running

  • Fix algorithm considering reruns as new builds (#7144) @taylorsilva 🔗

    • Fixes pipelines getting stuck with the same inputs when a job upstream of a job with version: every succeeds and is rerun

  • containerd: keep tasks running after concourse worker restarts gracefully (#7148) @aoldershaw 🔗

    • The containerd runtime is now more resilient to the concourse worker process gracefully restarting (e.g. via monit restart)
      • Tasks that were started prior to restart will continue to run when the worker process comes back up
      • This matches the behaviour of the Guardian runtime

  • Fixed build log reaper not respecting when both Days and Builds are set (#7179) @EstebanFS 🔗

    • The build log reaper has two options for determining when to reap logs. Before, if both of the options are set, it would reap if either of the two options were true, rather than requiring both of them to be satisfied

  • Apply a minimum rate limit for resource checking (#7218) @aoldershaw 🔗

    • If CONCOURSE_MAX_CHECKS_PER_SECOND is unset, Concourse will try to distribute checks evenly over the course of the check interval to reduce the concurrent load on external systems.
    • If there are few resources in a Concourse deployment (~1-20), checks may have to wait a substantial amount of time to run in order to space the checks out evenly. However, there's no real benefit to doing this, since having just a few resources doesn't cause significant load in the first place.
    • Now, Concourse ensures that at least one check is allowed to run per second

  • atc/db: prevent creation of duplicate check builds (#7221) @taylorsilva 🔗

    • Prevent duplicate checks from being created for a single resource

  • Fix browser back button after selecting a group (#7249) @aoldershaw 🔗

    • Previously, if a pipeline group was selected in the UI, the back button would not work (you'd have to press it twice to go back)

  • set_pipeline unpauses previously archived pipelines (#7255) @aoldershaw 🔗

    • When an archived pipeline is un-archived via the set_pipeline step, it will be unpaused

  • GC task caches belonging to archived pipelines (#7272) @aoldershaw 🔗

  • containerd: Clean up networking files in /tmp (#7276) @taylorsilva 🔗

    • Fixed a bug where the containerd runtime would create networking related files under /tmp and never delete them. They are now made under the --work-dir set for the worker and are cleaned up when the container is deleted. You can delete any lingering network files under your workers /tmp directory after upgrading.

  • Fix prometheus emitter not setting default attributes (#7294) @chenbh 🔗
    Additional metrics attributes configured by --metrics-attribute now propagates to the prometheus emitter correctly.

  • run check builds GC in batch (#7323) @xtremerui 🔗

🤷 Miscellaneous

📦 Bundled resource types

Check out latest releases or
releases around concourse/concourse v7.4.0

Don't miss a new concourse release

NewReleases is sending notifications on new releases.

Get notifications