Merge branch 'current' into nfiann-editing

Author: nataliefiann

website/docs/docs/build/latest-metrics-spec.md

@@ -12,7 +12,9 @@ The latest Semantic Layer specification creates an open standard for defining me
 With the new spec, you get simpler configuration without losing flexibility, faster onboarding for new contributors, and a clearer path to consistent, governed metrics across your organization. 
 
 :::info Availability
-The new YAML spec is currently only available in the <Constant name="fusion_engine" />, and will be coming soon to the <Constant name="dbt_platform" /> **Latest** release track and <Constant name="core" /> v1.12 (coming in 1H 2026).
+The new YAML spec is currently only available in the <Constant name="fusion_engine" />, and will be coming soon to the <Constant name="dbt_platform" /> **Latest** release track and <Constant name="core" /> v1.12. 
+
+For more information about availability, reach out to your account manager or post in the [#dbt-semantic-layer](https://getdbt.slack.com/archives/C046L0VTVR6) channel in the [dbt Community Slack](https://www.getdbt.com/community/join-the-community/). 
 :::
 
 ## Changes in the latest spec 

website/docs/docs/mesh/iceberg/databricks-iceberg-support.md

@@ -5,43 +5,95 @@ sidebar_label: "Databricks Iceberg support"
 description: Understand Databricks support for Apache Iceberg.
 ---
 
-Databricks is built on [Delta Lake](https://docs.databricks.com/aws/en/delta/) and stores data in the [Delta table](https://docs.databricks.com/aws/en/introduction/delta-comparison#delta-tables-default-data-table-architecture) format. Databricks does not support writing to Iceberg catalogs. 
-Databricks can create both managed Iceberg tables and Iceberg-compatible Delta tables by storing the table metadata in Iceberg and Delta, readable from external clients. In terms of reading, Unity Catalog does support reading from external Iceberg catalogs.
+import BaseLocationEnvIsolation from '/snippets/_base-location-env-isolation-warning.md';
 
-When a dbt model is configured with the table property `UniForm`, it will duplicate the Delta metadata for an Iceberg-compatible metadata. This allows external Iceberg compute engines to read from Unity Catalogs. 
+dbt supports materializing Iceberg tables in Unity Catalog using the catalog integration, starting with the dbt-databricks 1.9.0 release, for two Databricks materializations:
 
-Example SQL:
+- [Table](/docs/build/materializations#table)
+- [Incremental](/docs/build/materializations#incremental)
 
-```sql
-{{ config(
-    tblproperties={
-      'delta.enableIcebergCompatV2': 'true'
-      'delta.universalFormat.enabledFormats': 'iceberg'
-    }
- ) }}
+## Databricks Iceberg tables
 
-```
-To set up Databricks for reading and querying external tables, configure [Lakehouse Federation](https://docs.databricks.com/aws/en/query-federation/) and establish the catalog as a foreign catalog. This will be configured outside of dbt, and once completed, it will be another database you can query. 
+Databricks is built on [Delta Lake](https://docs.databricks.com/aws/en/delta/) and stores data in the [Delta table](https://docs.databricks.com/aws/en/introduction/delta-comparison#delta-tables-default-data-table-architecture) format.
+
+Databricks supports two methods for creating Iceberg tables in its data catalog, [Unity Catalog](https://docs.databricks.com/aws/en/data-governance/unity-catalog/):
+
+- Creating [Unity Catalog managed Iceberg tables](https://docs.databricks.com/aws/en/tables/managed). Databricks Runtime 16.4 LTS and later support this feature.
+- Enabling [Iceberg reads](https://docs.databricks.com/aws/en/delta/uniform) on Delta tables. These tables still use the Delta file format, but generate both Delta and Iceberg-compatible metadata. Databricks Runtime 14.3 LTS and later support this feature.
+
+External Iceberg compute engines can read from and write to these Iceberg tables using Unity Catalog's [Iceberg REST API endpoint](https://docs.databricks.com/aws/en/external-access/iceberg). However, Databricks only supports reading from external Iceberg catalogs.
+
+To set up Databricks for reading and querying external tables, configure [Lakehouse Federation](https://docs.databricks.com/aws/en/query-federation/) and establish the catalog as a foreign catalog. Configure this outside of dbt. Once completed, it becomes another database you can query. 
+
+dbt does not yet support enabling [Iceberg v3](https://docs.databricks.com/aws/en/iceberg/iceberg-v3) on managed Iceberg tables.
+
+## Creating Iceberg tables
+
+To configure dbt models to materialize as Iceberg tables, you can use a catalog integration with `table_format: iceberg` (see [dbt Catalog integration configurations for databricks](#dbt-catalog-integration-configurations-for-databricks)).
+
+<VersionBlock lastVersion="1.99">
+dbt supports both creating managed Iceberg tables and Iceberg-enabled Delta tables (formerly [UniForm](https://www.databricks.com/blog/delta-uniform-universal-format-lakehouse-interoperability)). The behavior flag [`use_managed_iceberg`](/reference/global-configs/databricks-changes#use-managed-iceberg) determines whether dbt creates a managed Iceberg table or a Delta table.
+</VersionBlock>
+
+<VersionBlock firstVersion="2.0">
+dbt supports both creating managed Iceberg tables and Iceberg-enabled Delta tables (formerly [UniForm](https://www.databricks.com/blog/delta-uniform-universal-format-lakehouse-interoperability)). The adapter property `use_uniform` (see [Adapter properties](#adapter-properties)) determines whether dbt creates a managed Iceberg table or a Delta table.
+</VersionBlock>
+
+### External tables
+
+dbt also supports creating externally-managed Iceberg tables using the model configuration [`location_root`](/reference/resource-configs/databricks-configs#configuring-tables). Databricks' DDL for creating tables requires a fully qualified `location`. dbt defines this parameter on the user's behalf to streamline usage and enforce basic isolation of table data:
 
-We do not currently support the new Private Priview features of Databricks managed Iceberg tables. 
+- When you set a `location_root` string, dbt generates a `location` string of the form: `{{ location_root }}/{{ model_name }}`.
+If you set the configuration option `include_full_name_in_path` to true, dbt generates a `location` string of the form `{{ location_root }}/{{ database_name}}/{{ schema_name }}/{{ model_name }}`.
 
+<VersionBlock firstVersion="2.0">
+In Fusion, dbt also supports setting `location_root` as an [adapter property](#adapter-properties) in `catalogs.yml`.
+</VersionBlock>
 
-## dbt Catalog integration configurations for Databricks
+### dbt Catalog integration configurations for Databricks
 
-The following table outlines the configuration fields required to set up a catalog integration for [Iceberg compatible tables in Databricks](https://docs.databricks.com/aws/en/delta/uniform).
+<VersionBlock lastVersion="1.99">
+The following table outlines the configuration fields required to set up a catalog integration for [Iceberg compatible tables in Databricks](https://docs.databricks.com/aws/en/iceberg).
 
 | Field | Description | Required | Accepted values |
 | :---- | :---- | :---- | :---- |
-| name | Name of the Catalog on Databricks | Yes | “my_unity_catalog” |
-| catalog_type | Type of catalog  | Yes | unity, hive_metastore |
-| table_format | Format for tables created by dbt models.  | Optional | Automatically set to `iceberg` for `catalog_type=unity`; and `default` for `hive_metastore`. |
-| file_format | Format used for dbt model outputs.   | Optional | Defaults to `delta` unless overwritten in Databricks account.  |
+| `name` | Name of the catalog on Databricks | Yes | "my_unity_catalog" |
+| `catalog_type` | Type of catalog  | Yes | unity, hive_metastore |
+| `table_format` | Format for tables created by dbt models.  | Optional | Automatically set to `iceberg` for `catalog_type=unity`, and `default` for `hive_metastore`. |
+| `file_format` | Format used for dbt model outputs.   | Optional | Defaults to `delta` unless overwritten in Databricks account.  |
+</VersionBlock>
+
+<VersionBlock firstVersion="2.0">
+The following table outlines the configuration fields required to set up a catalog integration for [Iceberg compatible tables in Databricks](https://docs.databricks.com/aws/en/iceberg).
+
+| Field | Description | Required | Accepted values |
+| :---- | :---- | :---- | :---- |
+| `name` | Name of catalog integration | Yes | "my_write_integration" |
+| `catalog_type` | Type of catalog  | Yes | unity, hive_metastore |
+| `table_format` | Format for tables created by dbt models.  | Optional | Automatically set to `iceberg` for `catalog_type=unity`, and `default` for `hive_metastore`. |
+| `file_format` | Format used for dbt model outputs. | Optional | Defaults to `delta` unless overwritten in Databricks account.  |
+| `adapter_properties` | Additional configurations unique to Databricks | Optional | See [Adapter properties](#adapter-properties) |
+</VersionBlock>
 
 #### Note
 
 On Databricks, if a model has `catalog_name=<>` in its model config, the catalog name becomes the catalog part of the model's FQN. For example, if the catalog is named `my_database`, a model with `catalog_name='my_database'` is materialized as `my_database.<schema>.<model>`.
 
-### Configure catalog integration for managed Iceberg tables
+<VersionBlock firstVersion="2.0">
+### Adapter properties
+
+These are the additional configurations, unique to Databricks, that you supply and nest under `adapter_properties`. These configurations are specific to Unity Catalog; `adapter_properties` is not allowed for catalog integrations with `catalog_type: hive_metastore`.
+
+| Field | Required | Accepted values |
+| --- | --- | --- |
+| `use_uniform` | Optional | `True` or `False` |
+| `location_root` | Optional | "external/location/path"; for example, `"s3://cloud-storage-uri"` |
+
+- **use_uniform**: Specifies whether to use managed Iceberg tables or Iceberg-enabled UniForm tables. By default, `use_uniform` is false.
+- **location_root**: Specify an [external location](https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-external-tables) root to write to. dbt writes the table to `<location_root>/<identifier>`, or `<location_root>/<database>/<schema>/<identifier>` if `include_full_name_in_path` is true.
+</VersionBlock>
+
+## Configure catalog integration for Iceberg tables
 
 1. Create a `catalogs.yml` at the top level of your dbt project (at the same level as dbt_project.yml)<br />
 <br />An example of Unity Catalog as the catalog:
@@ -78,6 +130,3 @@ select * from {{ ref('jaffle_shop_customers') }}
 ```
 
 3. Execute the dbt model with a `dbt run -s iceberg_model`.
-
-
-

website/docs/reference/resource-configs/databricks-configs.md

@@ -16,6 +16,7 @@ dbt-databricks v1.9 adds support for the `table_format: iceberg` config. Try it
 | table_format   | Whether or not to provision [Iceberg](https://docs.databricks.com/en/delta/uniform.html) compatibility for the materialization     | Optional     | SQL, Python     | `iceberg`    |
 | file_format <sup>†</sup>        | The file format to use when creating tables (`parquet`, `delta`, `hudi`, `csv`, `json`, `text`, `jdbc`, `orc`, `hive` or `libsvm`).   | Optional     | SQL, Python     | `delta`     |
 | location_root       | The created table uses the specified directory to store its data. The table alias is appended to it.     | Optional  | SQL, Python     | `/mnt/root`  |
+| include_full_name_in_path   | Whether to use the full table path to qualify the location root. If this is set, the database, schema, and table alias are all appended to the location root. | Optional  | SQL, Python     | `true`  |
 | partition_by        | Partition the created table by the specified columns. A directory is created for each partition. | Optional   | SQL, Python     | `date_day`  |
 | liquid_clustered_by<sup>^</sup>  | Cluster the created table by the specified columns. Clustering method is based on [Delta's Liquid Clustering feature](https://docs.databricks.com/en/delta/clustering.html). Available since dbt-databricks 1.6.2. | Optional          | SQL, Python     | `date_day` |
 | auto_liquid_cluster\+ | The created table is [automatically clustered by Databricks](https://docs.databricks.com/aws/en/delta/clustering#automatic-liquid-clustering).  Available since dbt-databricks 1.10.0 | Optional | SQL, Python | `auto_liquid_cluster: true` |