datahub-project/datahub v1.4.0rc2 on GitHub

What's Changed

docs(bigquery): schema resolver strategy by @sgomezvillamor in #15164
fix(bigquery): apply case normalization consistently for temp table inference by @sgomezvillamor in #15252
fix(abs): Prevent crash when validating dict path_specs in ABS source by @sgomezvillamor in #15329
fix(searchBarV2): fix selecting option by hovering by @v-tarasevich-blitz-brain in #15349
feat(uploadFiles): extend asset links to support file upload by @v-tarasevich-blitz-brain in #15225
feat(datasetModules): add basic implementation of columns module by @v-tarasevich-blitz-brain in #15293
fix(ingestion/grafana): fix fails caused by text panels in ingestion by @acrylJonny in #15291
fix(datahub-actions): make config in ActionConfig optional by @daha in #15393
fix(ingest/snowflake): Add classification_lib to setup dependencies by @treff7es in #15407
feat(openapi v3) support sorting customization on missing value handling by @zhixuanjia in #15383
fix(ingestion/grafana): fixing uniqueness bug in grafana connector by @acrylJonny in #15397
feat(dataset-summary): create new lineage module for dataset summary page by @purnimagarg1 in #15313
feat(ingestion): add feature flag for ingestion onboarding by @v-tarasevich-blitz-brain in #15376
feat(ingestion): update ownership ux by @v-tarasevich-blitz-brain in #15374
fix(cypress): optimyze summary about section tests by @v-tarasevich-blitz-brain in #15332
(fix/metadata-ingestion): Update classification docs to mark deprecated by @ani-malgari in #15380
fix(readOnly): fixes read only mode for update queries by @RyanHolstien in #15398
fix(sdk): DataJob environment defaults to PROD when using flow_urn (#15381) by @dinesh-github in #15388
fix(docs): Adding fixes for GraphQL APIs & support hiding from global context by @jjoyce0510 in #15399
fix(ui): fix entity type search filter by @sakethvarma397 in #15406
fix(ui): Adding chart's last modified data on search index by @sakethvarma397 in #15088
feat(ingestion): create ingestion source skeleton by @v-tarasevich-blitz-brain in #15390
fix(ingest/ci): fix docker compose integration test failures by @kyungsoo-datahub in #15424
feat(ingestion/oracle): add support for materialized view, stored procs and usage by @acrylJonny in #15118
fix(ui): Enable stateful ingestion for sources where we support it by @treff7es in #15422
feat(ingestion): Add OAuth callback support for Kafka producers/sinks by @max-datahub in #15420
chore(deps): fix (ch.qos.logback : logback-classic) by @relaxedboi in #15409
fix(log): change to warning by @anshbansal in #15440
fix(change-events): fix NPE in EditableSchemaMetadataChangeEventGenerator by @trialiya in #13895
fix(pit): reduce overhead of PIT creation by @RyanHolstien in #15412
fix(graphql): listUsers resolver incorrect access check by @david-leifker in #15448
feat(context): Various UI improvements for Context Base (Part 1/2) by @jjoyce0510 in #15413
feat(openapi)-NPE_v3_relationships_api_fix by @aabharti-visa in #15432
chore(deps): fix (org.bouncycastle:bcprov-jdk18on) by @relaxedboi in #15439
fix(csv-enricher): Add HTTP status code checking for remote file fetches by @pedro93 in #15454
fix(search): fix name conflicts in search(mlmodel) by @deepgarg760 in #15415
feat(ingest/airflow): add teradata operator support for Airflow plugin by @btkcodedev in #15418
fix(ui): Replacing antd Modals to component library by @sakethvarma397 in #13801
fix(graphql): improve response messages by @anshbansal in #15455
chore(test): Update test weights from CI runs (2025-11-30) by @github-actions[bot] in #15451
perf(metadata-io, ebean): add deterministic ordering for SELECT ... FOR UPDATE by @raudzis in #15425
Don't crash when filtering Add assets to domain by @benjiaming in #15462
docs(kafka): clarify USE_CONFLUENT_SCHEMA_REGISTRY disables cleanup policy step by @mhadole-tc in #15297
tests(smoke): add proper log format by @anshbansal in #15464
feat(ui/ingestion): create new page for ingestion run details by @purnimagarg1 in #15443
refactor(tests): move print to logger by @anshbansal in #15466
fix(secrets): Remove incorrect JSON escaping in updateSecret that corrupts special characters by @acrylJonny in #15442
docs: Update wrong helm reference slack.md by @Tim-Visser in #15433
feat(java-sdk): Add Java SDK V2 with fluent builder API and entity support by @shirshanka in #15307
feat(ingestion): improve create/update source handling by @v-tarasevich-blitz-brain in #15446
feat(ui/ingestion): build new source selection screen by @purnimagarg1 in #15471
fix(search): fix suggestion conversion for ES8 by @deepgarg760 in #15426
feat(kafka): add lightweight Kafka connectivity validation by @max-datahub in #15472
chore(deps): update org.ow2.asm dependency in datahub-client by @deepgarg760 in #15478
feat(ingest): add convert to lowercase for s3 by @kevinkarchacryl in #15475
feat(events): improve events kafka pool & client retry by @david-leifker in #15429
ES clear: delete indices themselves, not all the docs by @jmacryl in #15449
fix(changeEvent): fix reinstate lifecycle event condition by @RyanHolstien in #15476
docs(release): Revise known issues in release notes for v0.3.15 by @jayacryl in #15465
fix(ui) Limit the number of entities per page in search (#5198) by @chriscollins3456 in #14871
feat(python): make MCE Topic in Kafka sink as optional by @Anshul759 in #14150
feat(structured-properties): add version field to StructuredProperties by @pedro93 in #15255
docs(mcp): mcp server setup instructions by @jamesmayfield-star in #15484
docs: reference field-path-spec-v2 in SchemaFieldPath by @libvarun in #15477
test: make flaky neo4j integration test less flaky by @daha in #15492
docs: Update GraqhQL query in external OAuth authentication docs by @daha in #15493
feat(ui/ingestion): add search by name on source selection screen by @purnimagarg1 in #15488
docs: revert versioned docs by @yoonhyejin in #15486
feat(context): Support Viewing and Adding Related Context to All Asset Types by @jjoyce0510 in #15453
fix(es8): fix role creation issues by @RyanHolstien in #15485
docs: Update announcement bar content for Dec Town Hall by @grayayer in #15501
fix: make published GMS and frontend ports configurable with quickstart by @daha in #15494
fix(platformsModule): do not show add sources button when an user doesn't have permissions to manage ingestions by @v-tarasevich-blitz-brain in #15507
feat(ingestion): add the new details form (step 2) by @v-tarasevich-blitz-brain in #15474
fix(ingest/snowflake/sqlparser): Ensure table-column lineage consistency across Snowflake source and SQL aggregator by @treff7es in #15377
feat(ingestion): add advanced settings section by @v-tarasevich-blitz-brain in #15506
feat(ui/ingestion): build step 3 of create/edit ingestion source by @purnimagarg1 in #15508
fix(security): disable server version disclosure in HTTP headers by @esteban in #15500
chore(gradle): bump multiple 3rd party dependencies versions by @esteban in #15431
feat(assertion): enforce strict urns for assertions by @david-leifker in #15512
feat(ui/backend): business attributes support for new ui by @kartikey-visa in #15452
Added username to pr labeler for Patrick Druley (pdruley) by @pdruley in #15509
feat(ingestion/grafana) add option to pass grafana user email as dashboard owner by @daha in #15489
perf(ingestion): compile regex patterns for ingestion filtering hot path by @rob-1019 in #15463
Datahub as a microfrontend host (react) by @philip-nikolov-jpmc in #15358
Datahub as a microfrontend host (frontend) by @philip-nikolov-jpmc in #15359
feat(ingestion): Restores "Extending file sink to support writing to S3 (#14160) (#14248)" by @esteban in #15504
chore(deps): bump actions/checkout from 5 to 6 by @dependabot[bot] in #15385
docs: improve docs for cdc mode configuration by @shirshanka in #15496
docs(search): datahub cloud search access controls by @david-leifker in #15513
fix(java-sdk): Ensure full aspect writes complete before patches by @shirshanka in #15522
feat(ingestion): add filters section by @v-tarasevich-blitz-brain in #15516
fix(ingestion): fix updated secret field by @v-tarasevich-blitz-brain in #15528
feat(ingestion): update test connection button by @v-tarasevich-blitz-brain in #15527
chore(): bump lz4 & jdbc wrapper by @david-leifker in #15531
docs(v0.3.15.5): update docs by @david-leifker in #15510
ci(pr tracking): pr to linear automation by @gabe-lyons in #15503
feat(ingest/kafka-connect): Kafka connect infer lineage from DataHub by @treff7es in #15234
chore(ci): use cache mount for apt install to speed up docker build by @kewats in #15441
feat(ingestion/hive-metastore): add upstream lineage to hive-metastore by @acrylJonny in #15435
fix(ui) Fix two spots that need resolveRuntimePath by @chriscollins3456 in #15535
fix(ui) Fix unsafe destructuring of assertion in UI by @chriscollins3456 in #15519
feat(ingestion): add analytics events by @v-tarasevich-blitz-brain in #15547
feat(ui/ingestion): update breadcrumb in create/edit source to have option to go to Manage Data Sources by @purnimagarg1 in #15540
feat(ui/ingestion): bring ingestion page redirection changes back to OSS by @purnimagarg1 in #15544
feat(files) Add basic file cleanup after related entity is deleted by @chriscollins3456 in #15271
feat(dataplex): New connector for Google Dataplex by @pdruley in #15379
docs(logical): Add note on separate logical platform by @asikowitz in #15551
fix(system-update): fix system-update entity-client by @david-leifker in #15555
fix(ingestion/dremio): fix schema_resolver for non-standard uri length for dremio by implementing custom schema resolver by @acrylJonny in #15514
fix(actions): Prepare doc propagation action for prefixed pipeline names by @skrydal in #15672
fix(ingestion): fix ingestion onboarding redesign feature flag not being set by @purnimagarg1 in #15608
feat(ui): show 404 if user manually navigates browser url to non-existent mfe by @philip-nikolov-jpmc in #15548
feat(context): Context Base V1 Fix Ups by @jjoyce0510 in #15532
fix(gh to linear): fix missing gh repo in action by @gabe-lyons in #15678
feat(ci): re-run only failed smokle-tests in workflow reruns by @chakru-r in #15552
feat(ci): add workflow monitor for automatic retrigger on first failure by @chakru-r in #15683
improvement(ui/ingestion): add external sources and update categories and pills of sources by @purnimagarg1 in #15542
docs: fix broken link on slack setup docs by @yoonhyejin in #15680
fix(dataplex): Several fixes and refactoring for the Dataplex MVP by @pdruley in #15677
feat(ui) Show ingestion onboarding popup on new select source screen by @chriscollins3456 in #15685
feat(ui/signup): redesign the user sign up page by @purnimagarg1 in #15684
feat(docs): Adding Context Base Python APIs + docs by @jjoyce0510 in #15319
feat(ui/login): implement redesign of login page by @purnimagarg1 in #15686
feat(ui) Update styling of the filters section in new ingestion forms by @chriscollins3456 in #15687
feat(context): Remove "drafts" concept pre-release by @jjoyce0510 in #15536
fix(ui) Fix linting error on master by @chriscollins3456 in #15692
feat: Add useFinalSidebarTabs hook for entity sidebar extensibility by @annadoesdesign in #15690
feat(search): add basic search validations by @RyanHolstien in #15369
fix(structured-properties): fix property definition version type by @david-leifker in #15688
docs: fix command for remote executor helm chart & aws deploy by @yoonhyejin in #15681
fix(pr-to-linear): make task non blocking by @gabe-lyons in #15694
fix(ci): handle empty smoke-test matrix by @chakru-r in #15697
feat(ci): publish workflow metrics to posthog by @chakru-r in #15700
fix: adds missing tenacity dep for dataplex plugin by @sgomezvillamor in #15701
Update pr-labeler.yml by @alokr-dhub in #15705
fix(ingestion/grafana): fix broken ownership ingestion by @daha in #15525
chore: add dinesh-verma-datahub to pr-labeler team members by @dinesh-verma-datahub in #15706
updated pr-labeler.yml by @alfiyas-datahub in #15708
fix(masking): mask passwords in SQLAlchemy URLs using correct encoding by @alokr-dhub in #15696
feat(ingestion): add discard changes confirmation by @v-tarasevich-blitz-brain in #15543
feat(ingestion): add descriptions to sources by @v-tarasevich-blitz-brain in #15671
feat(ui): update design of reset credentials page by @purnimagarg1 in #15711
feat(ingestion): Add Redshift Query Tagging by @AdrianMachado in #15676
feat(ci): test failure metric reporting by @chakru-r in #15717
fix(pr-to-linear): fixing possible error conditions by @gabe-lyons in #15726
fix(reset-credentials): submit reset credentials form on enter and fix tests by @purnimagarg1 in #15724
feat(ui) Fix issues with new sign up page and show them be default by @chriscollins3456 in #15716
feat(hive): Add Thrift connection mode with Kerberos support to hive-metastore connector by @max-datahub in #15691
feat(ingestion): Add OAuth callback support for Kafka producers/sinks by @max-datahub in #15673
fix(ui): bring back change for fixing search bar loading style bug by @purnimagarg1 in #15720
refactor(fivetran): update handling of database and schema names to use quoted identifiers by @askumar27 in #15321
fix(ui): update button styles for login/signup screens by @purnimagarg1 in #15722
feat(ingestion): add telemetry events for the new ingestion flow by @purnimagarg1 in #15721
Improve disable states and Modal compatibility by @AdrianMachado in #15693
Add rob-1019 (Rob J. Caskey) to team-member list by @rob-1019 in #15469
fix(redshift): fix lineage extraction ignoring disabled flags (regression from #14580) by @sgomezvillamor in #15545
feat(ingestion/airflow) Add Airflow 3.x support by @treff7es in #13790
feat: add transformer to convert tags to structured properties by @sgomezvillamor in #15423
fix(ci): cleanup prev run artifacts during retry by @chakru-r in #15734
feat(ingest/powerbi): add Amazon Athena lineage support by @kyungsoo-datahub in #15728
feat(ingestion): add forms for popular sources by @v-tarasevich-blitz-brain in #15712
feat(sdk): enable parametrized assertion run in python sdk by @petemango in #15447
bugfix(fivetran): Improve Fivetran REST API Error Handling and User Reporting by @askumar27 in #15323
improvement(search) adding support for semantic search by @alexsku in #15743
feat(ingestion): Use UA constant for dbrx by @AdrianMachado in #15745
feat(ingestion): adding ingestion security guidelines by @gabe-lyons in #15729
fix(ingest/mssql): Fix MSSQL stored procedure lineage extraction by @kyungsoo-datahub in #15340
chore(test): Update test weights from CI runs (2025-12-21) by @github-actions[bot] in #15747
fix(cli): apply latest cli version by @deepgarg760 in #15731
fix(metabase): add legacy-mbql parameter for Metabase 0.57+ compatibility by @J0hnG4lt in #15709
fix(actions): Added support for pydantic v2 to actions framework by @pdruley in #15742
docs: integrate metadata standards sidebar section by @yoonhyejin in #15718
feat(dataplex): Adding dataplex platform logo ahead of connector by @pdruley in #15354
feat(docs): migrate RFC process back to main repository by @shirshanka in #15242
fix(ui/lineageV3): Fix lineage graph (breaks lineage module) by @asikowitz in #15715
fix(ui) Fix yaml editor cursor alignment issues by @chriscollins3456 in #15758
feat(config): support split consumer/producer config for kafka by @RyanHolstien in #15751
feat(azure-data-factory): add Azure Data Factory connector by @askumar27 in #15499
feat(ingest/snowflake): Add Snowflake Semantic View ingestion support by @kyungsoo-datahub in #15395
docs(metadata-tests): added tuning recommendation by @david-leifker in #15761
fix(log): message of checkV2TableExists by @nhuantho in #15765
feat(cli): add --env option to ingest deploy command by @pedro93 in #15518
feat(docker): add Alpine-based variants for ingestion and actions images by @esteban in #15764
docs(release-notes): Add initial release notes for upcoming version by @jayacryl in #15767
docs(observe): smart assertions profiling by @jayacryl in #15750
feat(observability): add Kafka consumer lag monitoring by @shirshanka in #15769
chore(ci): cleanup pr labeler by @anshbansal in #15771
Update pr-labeler to include myself as DH team member by @kewats in #15772
feat(consistency-checks): api and upgrade consistency checks by @david-leifker in #15766
fix(sql-parsing): split MSSQL statements when previous expression ends with parentheses by @alfiyas-datahub in #15730
improvement(ui/ingestion): update copy of abandonment flow modal by @purnimagarg1 in #15778
improvement(ui/ingestion): minor UI improvements in ingestion flow by @purnimagarg1 in #15779
fix(ingestion/schema-resolver): add enabler for sql parsing for more than 3 parts and implement in dremio by @acrylJonny in #15733
Add HTTPS Support and Allow Disabling HTTP in Play Frontend by @relaxedboi in #15757
feat(ui) Update copy for ingestion page subtitles by @chriscollins3456 in #15759
feat(ci): add pipMirrorUrl and pipExtraIndexUrl support for Docker builds by @esteban in #15773
fix(openapi-timeseries): return scrollId by @david-leifker in #15784

New Contributors

@dinesh-github made their first contribution in #15388
@max-datahub made their first contribution in #15420
@btkcodedev made their first contribution in #15418
@mhadole-tc made their first contribution in #15297
@Tim-Visser made their first contribution in #15433
@jamesmayfield-star made their first contribution in #15484
@libvarun made their first contribution in #15477
@pdruley made their first contribution in #15509
@rob-1019 made their first contribution in #15463
@philip-nikolov-jpmc made their first contribution in #15358
@kewats made their first contribution in #15441
@alokr-dhub made their first contribution in #15705
@dinesh-verma-datahub made their first contribution in #15706
@alfiyas-datahub made their first contribution in #15708
@J0hnG4lt made their first contribution in #15709
@nhuantho made their first contribution in #15765

Full Changelog: v1.4.0-preview...v1.4.0rc2