Release Highlights

Streaming for replication (WIP)

This version of server added gRPC streaming API for replications between clusters. This feature is still working in progress, is currently disabled by default. It will be enabled in future release.

Count workflow by ExecutionStatus

The Count API feature allows the user to count the number of workflow matching the given query. For example, the command temporal workflow count -q "WorkflowType='foo'" returns the number of workflows that matches WorkflowType='foo'.

Now, you can send queries to the Count API to group by a given search attribute. Eg: -q "WorkflowType='foo' GROUP BY ExecutionStatus returns the number of workflows that matches WorkflowType='foo' grouped by ExecutionStatus.

The GROUP BY clause is only supported in the Count API, and, at this moment, we only support grouping by ExecutionStatus. We plan to support group by any search attribute in a future release.

Default authorizer changes

The default authorizer now allows OperatorService.ListSearchAttributes as a namespace-level read-only API. Additionally, the implementation was changed: if you have custom authorizer code based on the default authorizer, you may want to examine the new implementation.

Schedules improvements

Schedules that are “idle”, meaning they are only configured to run for a fixed number of times (either with “limited actions” or with a spec that doesn’t match any times in the future), will now automatically be deleted after one week since their last action. This applies to existing schedules, however existing schedules that are already idle won’t be deleted unless they’re woken up by a signal.

Jitter is now calculated correctly so that jitter across separate schedules is not correlated. This automatically applies to existing schedules.

HTTP API

The frontend workflow service API can now exposed via HTTP in addition to the existing gRPC exposed form. The port is set on services.frontend.rpc.httpPort and is disabled if unset or 0. The default Docker template enables the HTTP API on port 7243. See the HTTP annotations on the workflow service API proto file for more information about which operations are available at which URL paths.

All changes

2023-09-05 - 9b877fa - increase max allowed shard linger limit (#4790)
2023-09-05 - c73430b - Mark QueryWorkflow as read-only in api metadata (#4834)
2023-09-05 - 86d8839 - Add method metadata and use for default authorizer (#4794)
2023-08-28 - bba21ca - Always persist guessed set id on unknown build id (#4800)
2023-08-28 - c13afdc - Fix mutable state change not persisted bug (#4813)
2023-08-21 - df2167e - Update API to v1.24.0 (#4785)
2023-08-21 - f6755a5 - Return NonRetryableApplicationError if Namespace is not found on remote cluster during verify (#4786)
2023-08-21 - f42d19e - Split sync workflow state logic to dedicated file (#4784)
2023-08-18 - c784796 - HTTP API support (#4543)
2023-08-17 - 8309152 - Update some task refresh logic (#4779)
2023-08-17 - 0080bd2 - Simplify XDC logic (#4546)
2023-08-14 - 84b1e8b - Generate txn id for each event (#4774)
2023-08-14 - aa5c3db - Add api to Eager retrieve the unknown namespace from remote cluster (#4769)
2023-08-14 - 9f9db3f - Increaes RecordHeartbeat frequency in VerifyReplicationTask (#4771)
2023-08-11 - 815c8a3 - Skip verifying workflow which has already passed retention time (#4770)
2023-08-11 - 0ce1234 - Rename leftover master to main (#4768)
2023-08-11 - 7982f81 - Fix delete search attributes in mutable state (#4767)
2023-08-11 - 545c963 - Count group by status capability (#4750)
2023-08-11 - 88f8c1c - Load task queue managers for demoted version set ids (#4762)
2023-08-11 - 2b3d52e - Improve otel gauge memory usage (#4722)
2023-08-10 - 673d563 - Add corruption protection to ms (#4564)
2023-08-11 - e4ce970 - Turn ErrWorkflowClosing into a retryable error (#4765)
2023-08-10 - 3a9e344 - Use raw history/matching client in task executor (#4710)
2023-08-10 - 97f15fc - Fix SQLite FTS query expression with dashes (#4755)
2023-08-10 - 2da4743 - Remove debug logger from visibility manager (#4751)
2023-08-10 - 2eb9609 - Make client code generation safer (#4747)
2023-08-10 - 274de98 - Make cassandra schema update idempotent (#4761)
2023-08-10 - 5ced7bf - Improve task range completion prerequisite check (#4754)
2023-08-10 - 7a11f91 - Move getTaskQueueManager call into redirect methods (#4759)
2023-08-10 - 97dbe09 - Fix retry policy max attempts (#4752)
2023-08-10 - 31eada0 - Reduce frontend handler retry max attempts (#4753)
2023-08-09 - 6d4a76b - Adjust API priorities (#4756)
2023-08-09 - cfad693 - Clean up some matching errors (#4758)
2023-08-09 - 0737a8b - Avoid potential double close on userDataChanged (#4757)
2023-08-09 - 63cd051 - Do not load sticky queue if not loaded (#4694)
2023-08-09 - d87acbd - Add frontend.globalNamespaceCount (#4743)
2023-08-09 - 039de3d - Add descriptions for task processing metrics (#4746)
2023-08-08 - e7dd2ff - Clean task queue name from matching before writing to WorkflowTaskScheduledEvent (#4565)
2023-08-08 - b1804c2 - eager namespace refresher POC (#4725)
2023-08-08 - 881b923 - APS limit error handling (#4739)
2023-08-08 - a608721 - remove the event cache initial size and refactor putInternal (#4698)
2023-08-07 - a3a4ac5 - Add descriptions for schedule metrics (#4728)
2023-08-07 - 58e731a - Enable new schedule workflow logic (#4741)
2023-08-07 - 60f733a - Fix bug causing duplicates when listing s3-archived workflows (#4712)
2023-08-07 - 5889f1c - Continue replication verification by skipping workflow which should be or soon to be deleted (#4734)
2023-08-07 - c861c7e - Update main branch in Makefile (#4740)
2023-08-07 - c49bdc6 - Change schedule jitter to depend on schedule + namespace ids (#4685)
2023-08-07 - 3bece5f - Collapse duplicate visibility tasks (#4727)
2023-08-07 - 0541c81 - Send versioned tasks to dlq when user data disabled (#4668)
2023-08-07 - bd909c9 - Return errors from task executors when processing actions (#4719)
2023-08-07 - df262d6 - Remove debug logging from Cassandra schema update tool (#4738)
2023-08-04 - 23aac05 - Speed up passive task validation (#4700)
2023-08-04 - 10a6f6e - Return ResourceExhaustedError for consistent query buffer full (#4731)
2023-08-04 - ecb625a - Update timer task error handling (#4718)
2023-08-03 - dcccbf0 - Log actual error for get history data loss events (#4726)
2023-08-03 - 5285b80 - Update running server locally README (#4723)
2023-08-03 - ccd23cb - Bump UI version to 2.17.1 (#4720)
2023-08-03 - db49117 - Add worker versioning calls to read-only namespace APIs for authorizer (#4721)
2023-08-03 - 9de9ccf - Add ActivityType to ActivityInfo for mutable state (#4717)
2023-08-02 - 4928b73 - Reduce the heartbeat size of VerifyReplicationTasks activity (#4705)
2023-08-02 - 1eb3217 - Add replay tests for scheduler workflow (#4701)
2023-08-02 - 4efda3c - Add more errors to targeted fault injector (#4711)
2023-08-01 - f4ca57f - Special some handle history client generation (#4708)
2023-08-01 - fb1d507 - Provide a shard ownership-based rate limit quota scaler from the history/shard package (#4704)
2023-08-01 - ac1d174 - Simplify newTaskQueueManager arguments (#4691)
2023-07-31 - 2832918 - Support injecting additional interceptors in history and matching (#4680)
2023-07-31 - 82fec84 - Inject custom task executor wrapper (#4697)
2023-07-28 - c595eb8 - Finish idle schedule when a timer fires (#4602)
2023-07-28 - 4fb4dd9 - move IsValid to ControllableContext (#4699)
2023-07-27 - 9269507 - Add versioned tag to loaded task queue gauge (#4672)
2023-07-27 - 83a3a45 - Persist unknown build ids (#4667)
2023-07-27 - 0281243 - enable shard linger with time limit config (#4696)
2023-07-28 - a13efee - Add quotas.DelayedRateLimiter (#4688)
2023-07-27 - 48b80c0 - Add shard.OwnershipScaledRateBurst (#4687)
2023-07-27 - 851e26b - Support count group by in visibility with SQL (#4666)
2023-07-27 - f9b2db5 - Support count group by in visibility with Elasticsearch (#4619)
2023-07-27 - a3b5911 - Matching service task GC (#4418)
2023-07-27 - 5690107 - Put user data replication tasks in DLQ (#4609)
2023-07-27 - 922cc27 - Convert version history bad request to internal error (#4690)
2023-07-27 - fbd0de7 - Fix Go version string goreleaser.yml (#4689)
2023-07-27 - 541dee3 - Add back generated mocks (#4671)
2023-07-26 - 4cd4f7e - Refactor CalculateEffectiveResourceLimit and NamespaceCountLimitInterceptor (#4665)
2023-07-26 - 3be986c - support delaying shard close for membership change (#4659)
2023-07-25 - 8753b8e - Adjust CLI page size (#4682)
2023-07-25 - 157bf67 - Fix tdbg unable to dump all workflow raw history issue (#4681)
2023-07-25 - c3fa088 - Return nil instead of task discarded error if namespace is invalid locally (#4678)
2023-07-25 - e7bd028 - add configurable ownership caching to history client (#4652)
2023-07-24 - fa1597b - Assign fx pass in raw task converter (#4679)
2023-07-24 - 5008255 - Add more replication error logs (#4677)
2023-07-24 - 386a651 - Minor fixes in force replication verification (#4675)
2023-07-24 - 48d3107 - Restore defaultListWorkflowsPageSize back to 1000 (from 100) (#4674)
2023-07-22 - ce3f42e - Add clock.TimeSource.AfterFunc (#4611)
2023-07-21 - c57f587 - Add XDC replication cache (#4379)
2023-07-21 - e10f213 - Fix flaky TestForceReplicationWorkflow_VerifyReplicationTaskNonRetryableError (#4669)
2023-07-21 - f708c81 - Move redirect logic into taskQueueManager (#4657)
2023-07-21 - baa964e - Fix flaky versioning integration test (#4651)
2023-07-21 - bd18f49 - defer shard.Unlock() (#4664)
2023-07-20 - 00587f6 - Execute VerifyReplicationTasks as an individual activity (#4656)
2023-07-21 - a834ea4 - Validate workflow task start time when complete (#4663)
2023-07-20 - a4a2ec0 - Fix future TestSetReadyGet_Parallel (#4661)
2023-07-20 - fc414a8 - Custom search attributes validation per store (#4655)
2023-07-20 - 61ff86b - update linter version in update-linters (#4653)
2023-07-19 - fdb94a5 - Do not use Unvailable as error type for expected error cases (#4650)
2023-07-19 - 724f23f - Cleanup new update related functional tests (#4563)
2023-07-19 - 2605c0b - Add support for ES Scroll for Scan API (#4614)
2023-07-19 - 0286211 - restructure history client for alternate ownership handling (#4636)
2023-07-19 - d7b4da8 - Extract common code from service/frontend/{fx,fx_test}.go (#4643)
2023-07-19 - 9a0745c - change the history cache key name to *Bytes (#4649)
2023-07-19 - a903b40 - Change nil metrics handle to no op handler (#4646)
2023-07-19 - 35ab8bd - Add a test for NamespaceCountLimitInterceptor (#4639)
2023-07-19 - 0646c0a - Log when health rate limiter value changes (#4648)
2023-07-18 - e8ae165 - Fail force-replication fast if any GenerateReplicationTasks/GenerateAndVerifyReplicationTasks activity returns error (#4642)
2023-07-19 - bf23997 - Better handle admin handler stream replication API lifecycle (#4647)
2023-07-18 - 8a75aed - Allow configurating history cache non-user context lock timeout (#4645)
2023-07-18 - 964a600 - do not cache large blob history event (#4621)
2023-07-18 - 1caf08d - Make replication task convertor injectable by fx (#4635)
2023-07-18 - ba5b23d - Prioritize UI/tctl when system is throttled (#4623)
2023-07-17 - a492906 - Ignore global rate limit if there are no hosts (#4627)
2023-07-17 - 72fe26c - Exclude history APIs from persistence health signals (#4637)
2023-07-14 - 7efba9f - Add verification of replication tasks in force replication (#4630)
2023-07-14 - 2d86390 - Do not generate visibility task for unchanged BuildIds (#4632)
2023-07-14 - a01a309 - Support cluster metadata customized tags (#4622)
2023-07-14 - 2fab90b - Drop standby tasks in standby taks executor (#4626)
2023-07-14 - 7e2de69 - Fix some minor warnings in the matching package (#4608)
2023-07-13 - cc38a41 - Add activity / workflow task version check (#4580)
2023-07-13 - b326a4a - Fix metrics tagging in stream replication task processing (#4625)
2023-07-13 - 6e811dd - Handle replicate workflow state with exist workflow data (#4617)
2023-07-12 - 1b82336 - Send default tasks to unversioned queue when user data disabled (#4610)
2023-07-12 - 9193a90 - Disable user data when fetch fails (#4604)
2023-07-12 - 2fd6123 - Deprecate QueueAckLevel (#4616)
2023-07-11 - 83faadf - Clean up cluster metadata initialization logic (#4531)
2023-07-11 - 87b0f60 - Fix metrics double emission in admin and operator handler (#4613)
2023-07-11 - 7dddf72 - Support cluster metadata customized tags (#4544)
2023-07-10 - d0c2a88 - Add frontend global RPS limit (#4583)
2023-07-10 - 3918d93 - emit history service panic metric for all requests (#4497)
2023-07-10 - 687125a - move controller shard membership handling (#4607)
2023-07-10 - ec1a010 - use a factory to create shard contexts (#4586)
2023-07-10 - 2374cc4 - Add a Clock option to our LRU cache (#4593)
2023-07-10 - 3847f90 - Remove durable archival flag (#4594)
2023-07-10 - 9547c83 - Disable confusing-naming linter and tune unhandled-error (#4606)
2023-07-10 - 0b0c398 - add stop reason for context (#4601)
2023-07-10 - 8f798a9 - Use time.NewTimer instead of time.After (#4596)
2023-07-07 - ec89a82 - properly fix migration in mem ack aggregation (#4571)
2023-07-07 - 5c9600d - Remove verbose logging (#4603)
2023-07-07 - 52997fd - Remove time.Sleep from util_test (#4592)
2023-07-07 - 3860610 - Remove flusher package (#4595)
2023-07-07 - a3f530d - Update go version to 1.20 (#4597)
2023-07-07 - 619e2f9 - Update trigger-publish GHA (#4599)
2023-07-07 - 731cc3c - Fix MySQL 8 upgrade script to schema v1.3 (#4598)
2023-07-06 - 061dbb2 - wait for drain duration before stopping shard controller (#4587)
2023-07-06 - 92c8418 - Refactor fx service Start/Stop (#4584)
2023-07-06 - bd241c0 - Log VisibilityStoreConfig for debugging (#4588)
2023-07-06 - a84c2e0 - support delay before history joins membership (#4582)
2023-07-06 - 15f2017 - Allow context disable frontend redirection (#4547)
2023-07-06 - 6d175c2 - move ringpop start call to service start (#4510)
2023-07-05 - 1ca6979 - Add help text to a few more metrics (#4529)
2023-07-05 - 17259db - Use DefaultServices for server_test (#4566)
2023-07-03 - db8b67a - Test speculative workflow task creation while completing previous workflow task (#4517)
2023-07-03 - 05e6ea0 - Disable goerr113 in test files (#4574)
2023-07-03 - 4b4dbf3 - Make all Go source file names snake case (#4558)
2023-07-03 - 2a22821 - Remove common.Daemon interface (#4573)
2023-06-30 - 76b5f33 - Improve build id scavenger (#4568)
2023-06-30 - 3bb5c58 - Fix migration in mem ack aggregation (#4569)
2023-06-29 - 6a92ddd - Versioning cleanup (#4556)
2023-06-29 - 2a2734e - Skip final task queue update if lost ownership (#4554)
2023-06-29 - 911d7c3 - Measure backlog after completing current task (#4562)
2023-06-29 - 598aa47 - Add corruption protection to ms (#4560)
2023-06-29 - f97237f - Handle get namespace by id error (#4559)
2023-06-29 - 5e03077 - Use lowest persistence priority for standby tasks (#4551)
2023-06-29 - 435cffe - Add a test to our gRPC resolver (#4548)
2023-06-29 - c0fb121 - Consider non default sets reachable by new workflows for a while after they stop being queue default (#4545)
2023-06-28 - 87cc6f4 - Get execution start/close time when using cassandra visibility store (#4549)
2023-06-27 - f51b96d - Add an error log detection test to the server (#4555)
2023-06-27 - a7b0afd - Handle new fx event type (#4553)
2023-06-27 - 11a30b2 - Per shard per namespace RPS warning log (#4525)
2023-06-27 - 39bf6a8 - Add Semgrep scanning (#4550)
2023-06-26 - 1e18621 - Add more metrics for stream replication (#4539)
2023-06-26 - 7198aa7 - Remove unused metrics def (#4540)
2023-06-23 - b5979f9 - Update go dependencies (#4534)
2023-06-23 - 326020a - Update versionChecker.go for the next release (#4533)
2023-06-23 - 7314ea4 - Update updateRegistry store on mutable state reload (#4532)
2023-06-22 - c52957b - Make replication tasks units 'dimensionless' (#4515)
2023-06-22 - cca978f - Remove ReplicationTasksReturned (#4514)
2023-06-22 - 9405cf8 - Perform task range completion on shard rangeID update (#4528)
2023-06-21 - 61b7d64 - Add an option for metrics descriptions (#4486)

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.22.0)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools