datahub-project/datahub v1.5.0rc1 on GitHub

What's Changed

fix(redirect): fix open rediect on redirect_uri by @david-leifker in #16279
fix(search): treat datahub.urn as URN in structured property index ma… by @david-leifker in #16284
fix(pr-to-linear): fixing issue with pr to linear accidentally creating duplicate linear tickets in certain cases by @gabe-lyons in #16288
feat(adf): Add column lineage extraction for Copy activity by @askumar27 in #16265
chore(): Agent context kit - Add data product entities to entity details by @jjoyce0510 in #16270
ci(pre-commit) Use action lint docker hook by @devashish2203 in #16290
fix(ingest): fix extra forward slash for looker url generation by @kevinkarchacryl in #16274
fix(tableau): extract table level lineage if column metadata is missing by @alokr-dhub in #16222
fix(deps): pin CVE-safe versions for frontend and global resolution by @david-leifker in #16266
fix(cypress): v2_glossaryTerm fix filtering by tag (oss) by @v-tarasevich-blitz-brain in #16273
fix(cypress): fix ingestionv2/source cypress test by @v-tarasevich-blitz-brain in #16277
fix(ui) Fix casing typo in DataHub name by @chriscollins3456 in #16299
fix(clickhouse): include dictGet() table references in SQL lineage by @kyungsoo-datahub in #16183
feat(linear): test to route to linear based on hueristics by @gabe-lyons in #16301
ci: test connector-tests-trigger workflow by @rajatoss in #16224
fix(ci): fix PR lookup in connector-tests-trigger by @rajatoss in #16311
fix(tableau): removed the field num_upstream_table_skipped_no_columns by @alokr-dhub in #16297
feat(views): redesign view builder with updated filters and UX improvements AI-377 by @annadoesdesign in #16313
fix(lineage): improve efficiency of lineage index backfill filter by @anshbansal in #16315
feat(ui/lint): add ESLint rules enforcing semantic color tokens for d… by @annadoesdesign in #16282
fix(ui) Fix infinite loop with tracking calls edge case by @chriscollins3456 in #16316
fix(ui): Properly hide lineage badge from search cards; store value in local storage by @chriscollins3456 in #16305
fix(usage-events): Do not try to hydrate entity without entity key by @gallyamb in #16020
fix(agent context snowflake): fix snowflake generators for agent-cont… by @nwadams in #16324
chore(): Update release notes for DataHub Cloud v0.3.16.4 by @jjoyce0510 in #16325
feat(retention): move retention to system-update, optimize delete and… by @david-leifker in #16323
fix(ingestion/s3): disable Avro schema name validation in S3 connector by @treff7es in #16327
chore(test): Update test weights from CI runs (2026-02-22) by @github-actions[bot] in #16308
feat(mssql): mssql extension for query lineage parsing by @alokr-dhub in #16084
docs(applications): remove experimental/beta language by @max-datahub in #16328
linear(routing): route with prompt, include multiple teams by @gabe-lyons in #16320
fix(graphql) Fix discrepancy with count in entities per platform chart by @chriscollins3456 in #16318
feat(sql_parsing): add generate_column_lineage parameter to skip CLL computation by @kyungsoo-datahub in #16244
feat(ingest/sigma): add default_db and default_schema to PlatformDetail config by @kyungsoo-datahub in #16245
docs: fix broken ingestion source and deployment documentation links by @lakshay-nasa in #16237
feat(ui/theme): add semantic color tokens for shadows, overlays, inputs, and interactive states by @annadoesdesign in #16337
fix: add note missing from actions menu by @ani-malgari in #16304
feat(doc rate limit): Add limits to document ingestion AI-380 by @nwadams in #16321
fix(ui): prevent document editor toolbar from disappearing when click… by @annadoesdesign in #16334
fix(sql-parser): upgrade sqlglot to v29.0.1 and fix semantic view parsing + Block statement handling by @sgomezvillamor in #16120
fix(sql-parser): migrate from sqlglot[rs] to sqlglot[c] with runtime patches by @sgomezvillamor in #16335
feat(ingestion): add trace ID logging to REST emitter by @pedro93 in #15968
fix(ingest/sigma): add retry logic for 429/503 and handle 400 Bad Request by @kyungsoo-datahub in #16243
bug(ingestion/sigma): Exclude types without lineage by @alisa-aylward-toast in #16302
fix(docs): change event name for metadata change events by @kevinkarchacryl in #16346
fix(ingest): handle null error_after criteria in dbt source freshness assertions by @danielford831 in #16295
fix(docs): broken link for proposals by @kevinkarchacryl in #16349
feat(kafka): support more split configurations to support migrations by @RyanHolstien in #16322
fix(cypress) Skip failing siblings cypress test for now by @chriscollins3456 in #16350
refactor(ui): standardize all entity icons to Phosphor and fix sizing by @annadoesdesign in #16285
docs(deploy): Fixed two small sets of typos by @javabrett in #16291
docs(release-notes): add known issue for v0.3.16.3 scroll pagination regression by @alexjst in #16351
fix(deps): bump aiohttp and azure-core for CVE fixes by @rajatoss in #16343
fix(ci): Update paths to coverage reports by @devashish2203 in #16359
docs(snowflake): improve docs about requirements on ACCOUNT_USAGE by @sgomezvillamor in #16347
test(ingestion): fix (or mitigate) flakiness of sql parser aggregator perf tests by @sgomezvillamor in #16360
docs: update README to reflect latest developments by @shirshanka in #16364
refactor(ingest/vertexai): Major code quality improvements and comment cleanup by @acrylJonny in #16176
fix(test): drop 100-query config from CI perf benchmark by @kyungsoo-datahub in #16354
fix(views): align entity types across View builder tabs and pass search facets to filters by @annadoesdesign in #16352
fix(snowflake): handle Snowflake 2026_01 BCR property format change in DDL lineage by @max-datahub in #16363
chore(ingestion/vertex-ai): updating capabilities of vertexai by @acrylJonny in #16367
build(deps): bump swiper from 11.1.4 to 12.1.2 in /docs-website by @dependabot[bot] in #16289
chore(deps): bump basic-ftp from 5.0.5 to 5.2.0 in /docs-website by @dependabot[bot] in #16353
build(deps): bump aquasecurity/trivy-action from 0.33.1 to 0.34.0 in /.github/workflows by @dependabot[bot] in #16260
chore(deps): bump webpack from 5.94.0 to 5.105.0 in /docs-website by @dependabot[bot] in #16122
chore(): Update Docs for Ask DataHub, Slack, & More AI-397 by @jjoyce0510 in #16369
fix(ingestion/oracle): fix geospatial type recognition for SDO_GEOMETRY columns by @acrylJonny in #16361
chore(): bump aircompressor by @david-leifker in #16370
feat(ingestion/oracle): recognize XMLTYPE and geospatial columns instead of falling back to NullType by @acrylJonny in #16366
feat(metadata-io): reindex retry, createIndex retry, and ES7 test fixes by @david-leifker in #16355
ci(metrics): Add scripts to collect workflow metrics by @devashish2203 in #16371
ci(metrics): Fix Post workflow actions - Correct artifact path by @devashish2203 in #16377
feat(ingestion/Sigma): Allowing workbook filtering in addition to workspace filtering by @alisa-aylward-toast in #16303
fix(ingestion/oracle): replace freezegun with time_machine in integration tests by @acrylJonny in #16362
ci(metrics): Add workflow dispatch action for testing by @devashish2203 in #16379
feat(bigquery): adds convert_column_urns_to_lowercase flag by @sgomezvillamor in #13232
feat(): Add dataset assertions to agent context kit snowflake by @jjoyce0510 in #16374
feat(policies): Add ability to target policies based on Glossary Terms and Groups by @chriscollins3456 in #16365
fix(ui/lineage-module): fix lineage module to show custom annotation nodes by @purnimagarg1 in #16312
fix(ingestion): add possibility to configure timeout for es reindex operations by @aksmiyazaki in #16187
feat(ingestion/kafka): add option to disable Avro schema name validation by @Devarsh23 in #16310
Upgrade full build to 17 by @Dutt23 in #16380
feat(snowflake): Add metadata pattern pushdown and table type filtering for improved performance by @rajatoss in #16100
chore(deps): bump minimatch from 3.1.2 to 3.1.5 in /smoke-test/tests/cypress by @dependabot[bot] in #16376
build(gitignore): add Agents Skills to gitignore by @TimSchlitzer in #16395
fix(ingest/snowflake): Snowflake quoting fix by @treff7es in #16393
feat(dev-tooling): improve dev ergonomics for running datahub by @shirshanka in #16394
fix(mae): fix mae interaction with ES8 data streams by @aksmiyazaki in #16280
Build Python wheels from merge commit by @rajatoss in #16402
chore(docker): remove Alpine variants from datahub-actions and datahu… by @david-leifker in #16404
refactor(dataplex): streamline Dataplex ingestion by removing entity … by @NehaGslab in #16063
ci(workflow): Add step to publish CI metrics to Posthog. by @devashish2203 in #16410
feat(ingest/snowplow): Snowplow source by @treff7es in #15735
feat(ingestion): update the navigation buttons on create/edit ingestion source by @purnimagarg1 in #16156
chore(): upgrade Jackson to 2.18.6 and regenerate lockfiles by @david-leifker in #16405
fix(ui): Re-center Looker logo image by @chriscollins3456 in #16414
feat(ingest/mongodb): add pymongo[aws] extra for AWS IAM auth support by @javabrett in #16412
fix(ui): alchemy-components page title cleanups by @jayacryl in #16407
fix(ingest/kafka-connect): use canonical 'mssql' platform for Debezium SQL Server SchemaResolver by @treff7es in #16413
fix(dagster): preserve DataJob lineage on failed/canceled runs by @treff7es in #16386
chore(): Fix dataset assertions tool optional args by @jjoyce0510 in #16392
chore(ingestion): Adjusting iceberg source examples and default recipe by @skrydal in #16417
docs(): Update MCP Server Documentation to Reflect Auth Options + Minor refactorings by @jjoyce0510 in #16418
fix(ci): update to new trivy action release to fix breakages by @RyanHolstien in #16420
fix(ui): Fix aspect ratio for lineage platform icons by @chriscollins3456 in #16421
feat(search): set dynamic on object type fields in ES v2/v3 mapings by @david-leifker in #16423
fix(ingest/teradata): set DATABASE context for view HELP commands by @JohnRTurner in #16208
fix(redshift): use boundary-aware segment stitching for query reconstruction by @kyungsoo-datahub in #16253
fix(ingestion): update save button style by @v-tarasevich-blitz-brain in #16427
improvement(ui): design review changes for dataset summary and ingestion page by @purnimagarg1 in #16429
fix(ingestion/oracle): fix profiling crashes and silent table exclusions by @acrylJonny in #16396
docs(release): v0.3.16.5-acryl by @david-leifker in #16428
fix(ui): Remove filter we don't support from run results tab by @chriscollins3456 in #16433
feat(agent-context): support sql search filters in mcp tools by @nwadams in #16403
feat(gms) Add backend resolvers to support multiple data products per asset by @chriscollins3456 in #16387
fix(ui): Normalize column field paths in column level lineage existence check by @chriscollins3456 in #16430
feat(ui) Add support for multiple data products per asset - frontend by @chriscollins3456 in #16388
fix(web): revert default select placement from to 'bottomLeft' by @jayacryl in #16432
docs(): Corrected camel-casing of DataHub name from Datahub in normal text by @javabrett in #16307
fix(ingestion/snowplow): add missing cachetools dependency to snowplow connector by @treff7es in #16442
fix(security): Remove hardcoded token signing keys (BREAKING) by @devashish2203 in #16385
feat(pr-to-linear): notify authors when reviewer is assigned in linear by @gabe-lyons in #16440
refactor(search): relocate runId tracking to MAE-consumer by @david-leifker in #16446
docs(quickstart): Update quickstart documentation to clarify signing key value precedence by @devashish2203 in #16447
fix(deps): GHSA-72hv-8253-57qq jackson-core - Hazelcast 5.6, parquet … by @david-leifker in #16448
[ai-378] feat(doc-embed): automatically embed datahub-documents every hour by @nwadams in #16450
docs(quickstart) Fix inter doc links by @devashish2203 in #16449
feat(dbt): add convert_urns_to_lowercase support for dbt ingestion by @alfiyas-datahub in #16358
ci(metrics): Add more workflows to submit metrics to Posthog. by @devashish2203 in #16456
feat(kafka-connect): bundle JVM via jdk4py to remove system Java dependency by @StanDmitrievAiven in #16445
feat(trino): add column-level lineage on upstreamLineage to connector… by @alfiyas-datahub in #16292
fix(frontend): add missing applications field to glossaryTerm GraphQL query by @max-datahub in #16398
fix(ingestion/snowflake): map COPY query type to INSERT instead of UNKNOWN by @treff7es in #16461
chore(GMS): Reduced verbosity of logs for conditional write failure by @skrydal in #16460
refactor(ci): Consolidate Free Disk space steps into a reusable action by @devashish2203 in #16454
[AI-233] feat(google adk): set up google adk for agent-context by @nwadams in #16434
docs(ingestion): add datahub-skills connector development guide by @maggiehays in #16435
feat(ingest/iceberg): add ingestion-time domain assignment by @sergey-pozdnyakov-epam in #16443
ci: make Linear ticket creation comment more informative for PR authors by @Copilot in #16463
feat(ui): bring tab changes to OSS by @v-tarasevich-blitz-brain in #16441
chore(ingestion): urllib3 v2 by @sgomezvillamor in #16464
feat(dev-tooling): consolidate CLAUDE.md into AGENTS.md, add setup/frontend commands by @shirshanka in #16470
Docs: Update CTA announcement bar link to YouTube recording by @grayayer in #16468
fix(local-dev): Corrected yarn version in mise.toml to 1.22.22. by @javabrett in #16406
ingest(snowflake): Emulate tag inheritance in-memory to eliminate N+1 queries by @treff7es in #16400
fix(smoke-test): fix flaky OpenAPI test with proper sync and thread safety by @pedro93 in #16444
fix(UI): fix selected alignment for bulk editing by @kevinkarchacryl in #16484
Bring fixes from SaaS (Select, ingestion fixes) by @v-tarasevich-blitz-brain in #16477
docs(): Add Microsoft Copilot Context Kit Guide + Misc Docs Improvements by @jjoyce0510 in #16452
docs(): Add docs for AI Plugins for Ask DataHub by @jjoyce0510 in #16469
feat(cli): agent-friendly datahub graphql + datahub init improvements by @shirshanka in #16476
feat(ingestion): migrate setup.py dependencies to pyproject.toml (PEP621) by @kyungsoo-datahub in #16339
docs(managed-datahub): Add release notes for version 0.3.17 by @jayacryl in #16488
ci(deploy): Update docker-unified to deploy to datahub head on every commit by @devashish2203 in #16481
fix(versioning): VersionPropertiesSideEffect mutates shared DataMap causing 422 on transaction retry by @acrylJonny in #16474
fix(python-build): show actual PR commit in wheel build context by @askumar27 in #16467
fix(): Adding file source to bundled venv set by @jjoyce0510 in #16492
feat(ui): add Alchemy Alert component by @annadoesdesign in #16487
fix(web): clear stale view URN when selected view no longer exists by @jayacryl in #16466
docs(3.17): adding smart assertion field metric deprecation notes by @jayacryl in #16491
docs(managed-datahub): Add to release notes and fix legacy dates by @AdrianMachado in #16500
feat(upgrade): K8 scale-down with conditional evaluation and tests by @david-leifker in #16451
docs(remote-executor): add 'Update Kubernetes Deployment' section by @ligfx in #16161
fix(ingestion/tableau): apply project filters to embedded datasources when emit_all_embedded_datasources is enabled by @aviraj-gour in #16340
feat(ingest/dbt): add support for dbt semantic models by @alfiyas-datahub in #16236
feat(docker): add slim-mode PySpark check in bundled venv builder by @david-leifker in #16504
feat(ingestion/kafka-connect): add support for Debezium and Confluent JDBC sink connectors by @acrylJonny in #16483
fix(ingestion): fix navigating to source from executions by @v-tarasevich-blitz-brain in #16510
docs(observe): smart assertion field metric support recommendations by @jayacryl in #16501
fix(docs): solve merge conflict duplication in release notes for v0.3.17 by @jayacryl in #16511
feat(pr-to-linear): integrating linear ids with github ids for pr to linear actions by @gabe-lyons in #16502
fix(ingestion): Add missing dependency constraints by @kyungsoo-datahub in #16513
docs(mcp): add Claude Code setup instructions by @jewright in #16514
fix(ingest/glue): allow generating fine-grained lineage without a graph by @ligfx in #16494
feat(cli): add search command with semantic search, projection, and agent context by @shirshanka in #16471
fix(deps): apply CVE minimums via constraints and bump Authlib, filelock by @david-leifker in #16517
docs(observe): dataset monitoring rules by @jayacryl in #16490
refactor(deps): use externalDependency for commons-lang3, force zooke… by @david-leifker in #16520
docs(ingestion): improve structure consistency by @sgomezvillamor in #16431
feat(sdk): expose trace IDs for SYNC_PRIMARY and ASYNC emit modes by @pedro93 in #15744
feat(ingestion): add uv.lock and constraints.txt for reproducible builds by @kyungsoo-datahub in #16489
Optimise plugins by @Dutt23 in #16531
docs: fix broken 'Edit this page' links on generated docs and improve contributor UX by @lakshay-nasa in #16530
(fix: datahub-web-react) glossary-term-hover-crashes-summary-panel by @ani-malgari in #16535
fixing wording for internal team members by @gabe-lyons in #16532
docs(observe): dataset monitoring rules - video walkthrough by @jayacryl in #16537
docs(ingest/powerbi): update Power BI docs to reflect Entra configuration by @ligfx in #16519
fix(deps): align all Netty modules to 4.1.130.Final for CVE fixes by @david-leifker in #16538
fix(gms): align MySQL schema with mysql-setup to prevent getNextVersi… by @david-leifker in #16542
Update Quickstart CTA by @manuelawei8 in #16524
docs: update community references and fix additional broken documenta… by @lakshay-nasa in #16345
feat(ingestion/mode): Improve Mode connector performance with threading, SQL caching, and instrumentation by @treff7es in #16300
fix: compatibility quickstart tests by @Dutt23 in #16548
perf : optimize gms startup by @Dutt23 in #16241
fix quickstart config by @Dutt23 in #16551
feat(ingest/glue): add updateTime as lastModified in dataset properties by @alokr-dhub in #16508
chore(ingest/glue): dedent get_lineage_if_enabled by one by @ligfx in #16495
fix(test): fix flaky TestIsolatedClassLoader by cleaning up SecurityManager by @pedro93 in #16525
perf(ingest): fix schema resolver bulk-fetch caching by @treff7es in #16499
fix(ingest): emit browsePathsV2 for DataFlow/DataJob with platform_instance to avoid Default folder by @treff7es in #15270
feat(docker): pin transitive deps in FULL/SLIM ingestion images by @kyungsoo-datahub in #16498
fix(ece): directly use event producer for ece by @RyanHolstien in #16534
docs(vertexai): add vertex AI builder docs for agent-context by @nwadams in #16516
docs(assertions): Timeseries Bucketing & Backfill Documentation by @AdrianMachado in #16523
ci(ingestion): add checkLockFile to verify uv.lock and constraints.txt freshness by @kyungsoo-datahub in #16559
fix(ingestion): bump pyarrow minimum for CVE-2026-25087 by @david-leifker in #16563
fix(analytics): strip URN prefix from platform name in daily report by @shirshanka in #16566
feat: remove setup containers; skip S3/STS when no AWS config by @david-leifker in #16558
feat(ingestion): add GraphQL query projection system for schema compa… by @shirshanka in #16522
docs: temporarily disable version dropdown and add 1.4.0 badge by @lakshay-nasa in #16536
fix(ingestion): restore correct emit modes after PR #15968 regression by @askumar27 in #16521
fix(test/oracle): sorted db query result order to remove test flakiness by @alokr-dhub in #16571
fix(docker): remove UV_CONSTRAINT from datahub-ingestion Dockerfile by @kyungsoo-datahub in #16570
fix(test): remove flaky test_pattern_rebuild_no_lock_contention by @kyungsoo-datahub in #16567
doc(release-notes-3-17): Update release notes for signing key removal. by @devashish2203 in #16556
fix(ingest): merge generatePyprojectDeps into updateLockFile by @kyungsoo-datahub in #16574
fix(ingest/glue): treat table UpdateTime as UTC by @ligfx in #16561
fix(ui/entity-header): Fix context path overflow on entity header by @asikowitz in #16576
feat(ingest/powerbi): enable extract_column_level_lineage by default by @ligfx in #16568
fix(logging): reduce excessive logging across DataHub Java services by @max-datahub in #16577
fix(logging): reduce log volume phase 2 — demote per-event/per-request INFO logging by @max-datahub in #16578
build(deps): bump logback to 1.5.32 and refresh lockfiles by @david-leifker in #16581
fix(bigquery): pass explicit credentials to GCP clients for thread safety by @jayacryl in #16579
refactor(datahub-web-react): migrate Access Tokens page from antd/MUI to alchemy components by @annadoesdesign in #16539
Rdf ingestion mvp by @stephengoldbaum in #15741
docs(observe): add loom embed for smart assertions backfill by @jayacryl in #16580
feat(dev): improve datahub-dev ergonomics — stop command, conflict detection, worktree fix by @shirshanka in #16587
fix(docs): fix docGen failure for RDF connector by @shirshanka in #16589
feat(docs): streamline integrations page and catalog generation by @shirshanka in #16597
docs(dev): add docs development guide to AGENTS.md + datahub-dev docs command by @shirshanka in #16598
chore(ingestion): datacatalog lineage and protobuf upgrade by @sgomezvillamor in #16560
fix(ingest): kafka platform instance helper by @kevinkarchacryl in #16526
chore(test): Update test weights from CI runs (2026-03-15) by @github-actions[bot] in #16588
fix(ingestion/kafka-connect): produce datajobs for JDBC sink when runtime topics API is empty by @acrylJonny in #16557
ci: update build job names to be unique to avoid conflicting checks in Mergeable by @devashish2203 in #16605
feat(ingestion): achieve feature parity between GE and SQLAlchemy profilers by @sgomezvillamor in #16529
refactor(ui): migrate Roles & Permissions pages to Alchemy components by @annadoesdesign in #16565
ci: add PR title validation workflow by @askumar27 in #16569
feat: add test-review skill for smoke and integration test quality analysis by @pedro93 in #16604
feat(ingestion): add configurable report sample sizes and failure logging by @rob-1019 in #16165

New Contributors

@gallyamb made their first contribution in #16020
@lakshay-nasa made their first contribution in #16237
@danielford831 made their first contribution in #16295
@alexjst made their first contribution in #16351
@aksmiyazaki made their first contribution in #16187
@Devarsh23 made their first contribution in #16310
@TimSchlitzer made their first contribution in #16395
@StanDmitrievAiven made their first contribution in #16445
@sergey-pozdnyakov-epam made their first contribution in #16443
@Copilot made their first contribution in #16463
@stephengoldbaum made their first contribution in #15741

Full Changelog: v1.4.0.3rc1...v1.5.0rc1