github open-metadata/OpenMetadata 2.0.3-release

pre-release7 hours ago

Changelog

OpenMetadata 2.0.3 is a maintenance release focused on connector reliability across Snowflake, BigQuery, MSSQL, Tableau, Looker, and DynamoDB, two data-integrity migrations, several security fixes including a certificate-verification bypass and a SQL-injection fix, and correctness fixes across search, lineage, and data quality.

🔌 Connectors & Ingestion

  • Airflow: pipelines showed a stale run instead of the latest one #33153: Pipeline status now selects the latest run correctly.
  • Unity Catalog: reading lineage and its SQL required two separate queries #33141: Native lineage and its SQL are now read together in one streamed query.
  • Unity Catalog: path-based lineage to external tables didn't resolve #32565: Path-based lineage to external tables now resolves correctly.
  • Mode: report discovery pagination could stop before reaching the last page #33189: Fixes the pagination termination condition, so reports past the first page are no longer missed.
  • Looker: lineage and project filtering were incomplete #32531: Improves both lineage resolution and project filtering.
  • REST: OpenAPI Path Item fields weren't handled #32958: Path Item-level fields are now parsed correctly.
  • DynamoDB: no way to see a table's primary key #33207: Extracts the partition key and sort key as the primary key.
  • PostgreSQL and Greenplum: materialized views weren't ingested #31549: Materialized views are now included.
  • dbt: 1.12+ inline metric specs failed to ingest #33535: Falls back to type_params.expr for dbt 1.12+ inline metric specs.
  • MSSQL: descriptions were read from the wrong database, encryption wasn't honored per driver, and query timeouts were unbounded #32574: All three are fixed.
  • Snowflake: the semantic catalog cache collided across databases sharing a schema name #33926: The cache is now keyed by database, not schema alone.
  • BigQuery: auto-detected partition windows could silently return empty samples #33221: Widens the auto-detected partition window to avoid silent empty samples.
  • Tableau: filtered-out workbooks still triggered API calls, and some workbooks threw null dataLength errors #33609: Filtered workbooks are now skipped before the API call, and the null dataLength errors are fixed.
  • Snowflake: procedure bodies and quoted tag values weren't read correctly #34146: Reads the DESC PROCEDURE body property and skips quoted tag values.
  • A batch of deferred foreign keys targeting the same table could overwrite each other #34099: Deferred FKs are now grouped by target table before being applied.

📊 Data Quality

  • Data Quality chart widgets showed stale responses #33186: Stale responses are now ignored in favor of the latest request.
  • Test Suites pagination wasn't reset when the search or owner filter changed #33467: Pagination now resets to page one on filter changes.
  • Test case and data contract scoping filters didn't resolve what they claimed to scope by #33393: The filters now resolve correctly against their stated scope.
  • A logical test suite's search docs could be dropped when recreated by upsert #33498: Search docs now stay intact across an upsert-triggered recreate.
  • The test-case picker's infinite scroll stalled on scaled displays #33708: Infinite scroll now works correctly at non-default display scaling.
  • A test suite pipeline's entityFullyQualifiedName didn't point at its table #33839: The pipeline now points at the correct table.
  • "Select all" on a filtered bundle suite added every test, not just the filtered subset #34047: "Select all" now adds only the filtered subset.
  • Table diff memory usage was unbounded on large diffs #33985: Table diff now keeps memory usage bounded on large diffs.

🔍 Search & Discovery

  • A collection-mutating Painless script could fail on missing fields #33788: The script now guards against missing fields.
  • Bulk pipeline status updates dropped relationship fields from the search doc #34046: Relationship fields are now preserved on bulk status updates.
  • A lineage operation waited on a global write queue instead of its own edge #33606: The operation now waits on the specific projected edge instead of the global queue.
  • Adding or updating lineage could throw a script exception #33449: Fixes the script exception in the add/update lineage path.

🛡️ Data Governance & Quality

  • Bot impersonation restrictions had gaps #32353: Hardens impersonation restriction filtering, closing a privilege-escalation path in workflow triggers and user import.
  • PII (Personally Identifiable Information) classification averaged recognizer scores across the sample instead of taking the strongest match #32153: Uses the maximum recognizer score instead of a sample-size average.
  • PII classification didn't corroborate spaCy's named-entity results #33720: spaCy named entities are now corroborated for more accurate classification.
  • Users could be added to a team that disallows joining #33423: The joinable-team check now applies to both user creation and update.
  • Data Product ports were keyed by request index instead of entity ID #32939: Ports are now keyed by entity ID, preventing mismatches.
  • Incident Manager permissions were coupled to Test Case Edit permissions #32074: The two permission sets are now decoupled.
  • Any user could PATCH another user's profile #34095: PATCHing another user now requires EditAll.
  • ODPS (Open Data Product Specification) import keyed Data Product identity on display name instead of product ID #34193: Identity is now keyed on productID, preventing duplicate or mismatched imports on rename.
  • ODPS import didn't resolve the tag source, so glossary terms failed to attach #34191: The tag source now resolves correctly, so glossary terms attach as expected.
  • The Data Completeness field picker didn't send custom properties correctly #33836: Custom properties are now sent as extension.<name>.
  • Migration: a stale L1 cache could drop the DataConsumerPolicy CreateTask rule #33763: Repairs the rule where it was dropped.
  • hashlib.md5 calls blocked FIPS (Federal Information Processing Standards) mode deployments #33261: Adds usedforsecurity=False to every hashlib.md5 call for FIPS compatibility.
  • Task form filter values were interpolated directly into generated SQL #34107: Filter values are now bound as named parameters, separate from the generated SQL.
  • verify_ssl=True actually disabled certificate verification in the SDK #34016: verify_ssl=True now enables certificate verification as expected.
  • Rule Library SQL expressions executed without a sandbox #34015: SQL expressions now render inside a Jinja sandbox.
  • PATCHing a Table could lose schemaDefinition and drop tablePartition #34053: Both are now preserved across a PATCH.

🔐 Authentication

  • LDAP (Lightweight Directory Access Protocol) login could wipe a user's team memberships and persona #33733: Loads the full user record on login instead of a partial one, so teams and personas are no longer dropped.
  • A regenerated SCIM (System for Cross-domain Identity Management) token's "Created On" date didn't update #33828: Regenerating a SCIM token now updates its "Created On" date.

⚙️ Platform

  • Migration: entity_extension.extension was too narrow for some custom property values #32121: Widens the column to VARCHAR(512).
  • RdfTagUpdater wrote tags through an invalid inline SPARQL writer #33510: Removes the invalid writer.
  • The Fuseki (RDF) extension was required even when unused, and readiness failures were unclear #33838: The extension is now optional, and readiness failures are reported accurately.
  • The history-by-timestamp page broke when an entity in the window was hard-deleted #33016: The page now serves correctly even when an entity in the window is hard-deleted.
  • Activity replies could deadlock on MySQL, and a database rollback could still be committed #34123: Fixes the MySQL deadlock, and a transaction the database rolled back is never committed.
  • There was no record of why an app run ended, and RDF reindexing could run on multiple servers at once #33901: App runs now record why they ended, and RDF reindexing runs on a single server.

🎛️ UI

  • Connector docs links and field docs on ingestion agent forms pointed to the wrong place #33470: Links and field docs are now correct.
  • A hard-deleted entity's page didn't redirect anywhere #33419: The page now redirects to the parent entity after a hard delete.
  • Task comments lost their delete option, hover affordance, and expand control #33629: All three are restored.
  • The markdown converter mangled nested lists in descriptions #33693: Nested lists now convert correctly.
  • Navigating to a non-existent glossary FQN silently fell back to the first glossary, and the right panel loader could block on an already-confirmed 404 #33724: A not-found placeholder shows instead of the fallback, and the loader no longer blocks.
  • Archiving a file with the same name as an existing one gave no explanation #33766: Context Center now returns an actionable error for a same-name archived file.
  • Dashboard widgets didn't respect their configured size and could flicker between tabs #33952: Widgets are now keyed correctly, respect their size, and no longer flicker on tab switch.
  • The ingestion Success count didn't include updated records #34037: updated_records now counts toward the ingestion Success count.

🔒 Security

📣 Notifications

  • Task change events weren't emitted from every resolve, close, and reopen path #33278: Change events now fire from every one of those paths.
  • Pipeline observability could emit for the wrong run, or more than once per DAG (Directed Acyclic Graph) #32352: Observability now emits exactly once per DAG, from the newest run.
  • A late event consumer waited for the next misfire scan instead of running immediately #34210: A late consumer now runs at once.

Full Chnagelog: 2.0.2...2.0.3

Don't miss a new OpenMetadata release

NewReleases is sending notifications on new releases.