Skip to content

v0.13.3

Latest
Compare
Choose a tag to compare
@david-leifker david-leifker released this 23 May 23:11

DataHub Release Notes

User Experience

  • NEW: Business Attributes: Business Attributes are used to standardize and manage data elements across multiple domains, projects, and applications. By linking dataset attributes to Business Attributes, organizations ensure uniformity and ease of updates, as changes made to a Business Attribute are automatically propagated across all linked datasets. #9863
  • Improved UI for Dataset Properties: Added collapse functionality for long dataset properties, making it easier to navigate and view relevant information. #10203
  • Pagination for Ingestion Tasks Listing: Added pagination to the tasks listing page, making it easier to manage and navigate through tasks. #10293
  • Rich Text Support for Form Descriptions: Added support for rich text in form descriptions, enhancing the user experience. #10425
  • New Analytics Charts: Added charts in the Analytics tab to identify Top Users and New Users. #10344
  • Enhanced search functionality with customizable autocomplete configuration. #10426

Developer Experience

  • Unified CI Workflow Updates: Improved CI build with unified workflow updates and disk space cleanup, making the build process more efficient. #10353
  • Improved Logging for GraphQL Requests: Enhanced logging for GraphQL requests, providing better insights and debugging capabilities. #10404
  • Enhanced Documentation for Lineage Feature Guide: Updated documentation for the lineage feature guide, making it easier to understand and implement. #10401
  • Improved Documentation for SchemaField.label: Updated documentation for SchemaField.label, providing clearer guidance for developers. #10251
  • Enhanced CI with Docker Image Publishing: Added Docker image publishing capabilities to the CI workflow, streamlining the deployment process. #10193
  • Redesigned Docs Site Feedback Button: Improved the design of the feedback button in the documentation, making it more user-friendly. #10182

Metadata Ingestion

  • Improved Data Profiling by early filtering of tables, correctly computing sample row counts, and combining unique count queries per table. #10378, #10319, #10322
  • Airflow: Introduced support for BigQueryInsertJobOperator. #10452
  • BigQuery: Added support for Table Clones and incremental column-level lineage.
  • Snowflake: Improved reporting for usage aggregation and handled lineage errors; Improved ingestion performance with system sampling on very large tables. #10279, #10430
  • Glue: Introduced support for delta schemas. #10299
  • Redshift: Improved usage extraction by filtering out system queries. #10247
  • Mode: Enhanced ingestion for Mode by adding dashboards into containers, improving data visualization and management. #10563
  • PowerBI: Added support to automatically extract table lineage between PowerBI and Databricks. #10416
  • dbt: Improved dbt ingestion by handling complex SQL and enhancing documentation, providing better data management and insights. #10323
  • NiFi: Enhanced ingestion for NiFi with process group as browse path and incremental lineage, improving data organization and tracking. #10202
  • Incubating Sigma and CockroachDB sources. #10037, #10226

Breaking Changes

  • DynamoDB Connector: aws_region is now a required configuration. The connector will no longer loop through all AWS regions; instead, it will only use the region passed into the recipe configuration. #10419
  • Custom Validators and Mutators: Dropped a previously required constructor. #10389
  • FabricType RVW: Added as a new FabricType. No rollbacks allowed once metadata with this fabric type is added without manual cleanups in databases. #10472

For full details on breaking changes, please refer to the updating DataHub documentation.

Contributors

A big thank you to all our contributors for this release!

First-Time Contributors

@bouaouda-achraf, @camilogutierrez, @dotan-mor, @egemenberk, @erikkvale, @guyr-ziprecruiter, @ishtartec, @jonasHanhan, @mrjefflewis, @noggi, @olgapenedo, @paguos, @richenc, @Rosmirose, @sagar-salvi-apptware, @timothyjin

Repeat Contributors

@ajoymajumdar, @deepgarg-visa, @dushayntAW, @filipe-caetano-ovo, @gaurav2733, @kevin1chun, @ksrinath, @Masterchen09, @mayurinehate, @ms32035, @Nelvin73, @rtekal, @sgomezvillamor, @shubhamjagtap639, @siladitya2, @skrydal

DataHub Maintainers

@anshbansal, @asikowitz, @chriscollins3456, @darnaut, @david-leifker, @eboneil, @gabe-lyons, @hsheth2, @jayacryl, @jjoyce0510, @RyanHolstien, @shirshanka , @sid-acryl, @treff7es, @yoonhyejin

Thank you all for your hard work and contributions!

What's Changed

New Contributors

Full Changelog: v0.13.1...v0.13.3