Release Highlights
Schema Changes
This release does not includes any schema changes.
Notable Changes
Advanced Visibility
This release changes how custom search attributes are managed. If you are not using advanced visibility / Elasticsearch, you are not affected and can skip this section.
Before v1.10.0 it wasn't easy to add new custom search attributes to a cluster. It was a two step process:
- A cluster admin had to first manually change the Elasticsearch schema.
- Then they had to manually update the dynamic config.
v1.10.0 adds new commands and modifies existing commands to manage and view search attributes:
tctl admin cluster add-search-attributes --name ProductId --type Keyword
tctl admin cluster get-search-attributes
tctl cluster get-search-attributesNote: Due to Elasticsearch limitations you can only add new custom search attributes but not rename or remove existing ones. This is why we removed 6 predefined Custom*Field search attributes from the default server config.
If you don't use them, you can remove them from your dynamic config before v1.10.0 deployment.
After the upgrade to v1.10.0 , when the Temporal Server starts up, it will automatically read the current search attributes from the dynamic config and save them to the cluster metadata database table. After that, the entire frontend.validSearchAttributes section can be removed from your dynamic config.
To browse existing search attributes, run tctl cluster get-search-attributes command. You will see search attribute output like the following:
Search attributes:
+-----------------------+---------+
| NAME | TYPE |
+-----------------------+---------+
| BinaryChecksums | Keyword |
| CloseTime | Int |
| CustomNamespace | Keyword |
| ExecutionStatus | Int |
| ExecutionTime | Int |
| Operator | Keyword |
| RunId | Keyword |
| StartTime | Int |
| TaskQueue | Keyword |
| TemporalChangeVersion | Keyword |
| WorkflowId | Keyword |
| WorkflowType | Keyword |
+-----------------------+---------+To add new search attributes to your cluster you need to run the tctl admin cluster command:
tctl admin cluster add-search-attributes --name ProductId --type Keyword
You are about to add search attributes [ProductId:Keyword]. Continue? Y/N y
Custom search attributes:
+-----------+---------+
| NAME | TYPE |
+-----------+---------+
| ProductId | Keyword |
+-----------+---------+
System search attributes:
+-----------------------+---------+
| NAME | TYPE |
+-----------------------+---------+
| BinaryChecksums | Keyword |
| CloseTime | Int |
| CustomNamespace | Keyword |
| ExecutionStatus | Int |
| ExecutionTime | Int |
| Operator | Keyword |
| RunId | Keyword |
| StartTime | Int |
| TaskQueue | Keyword |
| TemporalChangeVersion | Keyword |
| WorkflowId | Keyword |
| WorkflowType | Keyword |
+-----------------------+---------+
Storage mappings:
+----------------------------+-------------+
| COLUMN NAME | COLUMN TYPE |
+----------------------------+-------------+
| Attr.BinaryChecksums | keyword |
| Attr.CustomNamespace | keyword |
| Attr.Operator | keyword |
| Attr.ProductId | keyword |
| Attr.TemporalChangeVersion | keyword |
| CloseTime | long |
| ExecutionStatus | long |
| ExecutionTime | long |
| NamespaceId | keyword |
| RunId | keyword |
| StartTime | long |
| TaskQueue | keyword |
| WorkflowId | keyword |
| WorkflowType | keyword |
+----------------------------+-------------+
Workflow info:
{
"execution": {
"workflowId": "temporal-sys-add-search-attributes-workflow",
"runId": "45386457-3015-47cd-806a-5f40ead7f327"
},
"startTime": "2021-06-02T21:11:40.984120413Z",
"closeTime": "2021-06-02T21:11:41.364028543Z",
"status": "Completed"
}
Search attributes have been added successfully.The tctl admin cluster get-search-attributes command will give you the exact same output.
The admin command output has more info than the non-admin tctl cluster get-search-attributes command. It shows custom and system search attributes separately. In addition to this, it shows the actual Elasticsearch schema and state of the system Workflow Execution which is adding the search attributes to the cluster (can be useful for debugging).
Search attributes are cached internally and will be available 1 minute after being added.
Dockerfile changes
Dockerfile changes are mostly related to advanced visibility. Previously, if ENABLE_ES was set to true, the startup script would create the Elasticsearch index for you. Now this happens only with the auto-setup image (used by our docker-compose files) or with helm-chart. If you are using the server image you need to create the index manually.
Also Custom*Field search attributes are added automatically for auto-setup image to support compatibility with samples.
All changes:
2021-04-27 - 0886827 - Add priority throttling (#1500)
2021-04-27 - 836a638 - Update most of dependencies to the latest versions (#1494)
2021-04-27 - 99a5fb2 - Fix reset workflow pending activity schedule time (#1502)
2021-04-27 - a9fb3e1 - Fix history service metrics double emission issue (#1498)
2021-04-27 - f951d58 - Update README.md (#1493)
2021-04-28 - 07abf54 - Add metrics propagation across services and no user-induced latency metric (#1476)
2021-04-28 - 1ddd9ba - Bump Web to v1.9.0 (#1504)
2021-04-28 - 63f2c83 - Pin go-sql-driver/mysql to v1.5.0 (#1503)
2021-04-29 - 2d7d63d - Default to use DB record version for mutable state (#1506)
2021-04-29 - f8e6cad - Add SearchAttributesManager (#1489)
2021-04-30 - aeaef66 - Expose workflow state transition count (#1505)
2021-05-02 - 44cdadf - Add check for certificate expiration on start (#1511)
2021-05-03 - 1d020b5 - Modify Elasticsearch client for the new search attribute implementation (#1513)
2021-05-03 - 4f9473e - Emit shard lock metrics (#1508)
2021-05-03 - 7a69e2b - Disable visibility config validation if advanced visibility is configured (#1519)
2021-05-03 - 9d610b4 - Do not try set metrics trailer if context is closed (#1516)
2021-05-04 - 3e19d63 - Add thread safety to metrics context (#1517)
2021-05-04 - 9dc9ac8 - Report correct workflow type name tag (#1509)
2021-05-05 - 4b390d7 - Do not persist to DB when resetting sticky task queue (#1520)
2021-05-06 - 22e5045 - Align CODEOWNERS with repository access (#1525)
2021-05-06 - 242a0be - Fix go test coverage log warrning (#1528)
2021-05-06 - 476a12f - Fix readme slack invite link (#1527)
2021-05-06 - 7962111 - Add "addsearchattributes" worker (#1518)
2021-05-06 - a223a11 - Add fossa scan to buildkite (#1526)
2021-05-07 - 26cbc5f - Bump current version to 1.10.0 (#1534)
2021-05-07 - 7606c6a - Set branch on fossa analyze (#1529)
2021-05-07 - d2c5477 - Add fossa badge to readme (#1533)
2021-05-07 - dab33d4 - Upsert and Delete visibility record in the configured database only (#1532)
2021-05-10 - 76f55e6 - Use priority based API throttling (#1512)
2021-05-10 - e7cc7f2 - Add back GlobalNamespaceRPS (#1537)
2021-05-11 - 0f81c55 - Disabe build fail on fossa (#1541)
2021-05-11 - 3fd1fd8 - Pass logger from server options instead of calling log.NewDefaultLogger() (#1542)
2021-05-11 - 5580e86 - Disable EnableDBRecordVersion for 1.10 (#1539)
2021-05-12 - 91a6593 - Rename log.NewDefaultLogger() to log.NewTestLogger() (#1543)
2021-05-13 - 223ac38 - Bump Web to v1.9.1 (#1547)
2021-05-13 - 82b05a2 - Feature: data converter websocket (#1437)
2021-05-17 - 36c4bd9 - Use ClusterMetadata based search attributes (#1540)
2021-05-17 - 37583eb - Remove CustomField search attributes from index template (#1553)
2021-05-17 - a073b64 - Update jmoiron to the latest version 1.3.4 (#1550)
2021-05-18 - 71f073b - Allow search attributes when Elasticsearch is not configured (#1556)
2021-05-18 - b3f10a7 - Split integration_tests.go (#1548)
2021-05-19 - 959fb34 - Properly dedup signal request ID (#1558)
2021-05-19 - eff23dc - Refactor Dockerfile and scripts (#1501)
2021-05-20 - 6a0ea05 - Set search attribute types for history events attributes (#1551)
2021-05-20 - 8fbc615 - Fix LastFirstEventTxnId not being properly propagated (#1560)
2021-05-21 - 03225b6 - Add ES_USER and ES_PWD to auto-setup.sh (#1563)
2021-05-21 - 4f54be5 - Add shellcheck to the base builder image (#1524)
2021-05-21 - fb25ef3 - Remove dynamic config UpdateValue API (#1557)
2021-05-22 - c09cd45 - Remove Custom*Field search attributes (#1562)
2021-05-22 - e727600 - Use shellcheck in ci-build target (#1565)
2021-05-22 - f04753f - Use ONBUILD in Dockerfile for develop image (#1564)
2021-05-24 - 1867fa7 - Improve namespace ack level calculation (#1568)
2021-05-24 - 23ced89 - Fix issue that namespace replication name is not set in req (#1569)
2021-05-24 - e153684 - Add option to specify custom metrics reporter (#1561)
2021-05-24 - f355adc - Remove dry run flag and test run from documentation (#1567)
2021-05-25 - 103ba39 - Merge serialization methods into new Serializer (#1570)
2021-05-25 - 1688801 - Use 10s timeout instead of long poll (2m) timeout as default for visibility commands in tctl (#1571)
2021-05-25 - a8c9606 - Fix tctl visibility timeout to 10s (#1572)
2021-05-26 - 2e3332f - Bump Web to v1.9.2 (#1574)
2021-05-27 - a69bc42 - Header providers plugin support (#1488)
2021-05-27 - b040653 - Show connection error details for PostgreSQL layer (#1581)
2021-06-01 - 790af7c - Include namespace tag to API error metrics (#1582)
2021-06-02 - 44131b5 - Add Attr properties to Elasticsearch GetMapping API (#1587)
Helpful links to get you started with Temporal
Temporal Docs
Server
Docker Compose
Helm Chart