Skip to content

Commit 973cf87

Browse files
committed
added documentation for external-tables and fallback_schema
1 parent 459424d commit 973cf87

File tree

1 file changed

+86
-0
lines changed

1 file changed

+86
-0
lines changed

website/docs/reference/resource-configs/teradata-configs.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -413,3 +413,89 @@ These steps collectively ensure that the valid_history strategy effectively mana
413413
)}}
414414
```
415415
See [Collecting Statistics documentation](https://docs.teradata.com/r/76g1CuvvQlYBjb2WPIuk3g/RAyUdGfvREwbO9J0DMNpLw) for more information.
416+
417+
## dbt-external-tables
418+
* [dbt-external-tables](https://github.com/dbt-labs/dbt-external-tables) are supported with dbt-teradata from dbt-teradata v1.9.3 onwards.
419+
* Under the hood, dbt-teradata uses the concept of foreign tables to create tables from external sources. More information can be found [here](https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Data-Definition-Language-Syntax-and-Examples/Table-Statements/CREATE-FOREIGN-TABLE)
420+
* User need to add the dbt-external-tables packages as dependency and can be resolved with `dbt deps` command
421+
```yaml
422+
packages:
423+
- package: dbt-labs/dbt_external_tables
424+
version: [">=0.9.0", "<1.0.0"]
425+
```
426+
* User need to add dispatch config for the project to pick the overridden macros from dbt-teradata package
427+
```yaml
428+
dispatch:
429+
- macro_namespace: dbt_external_tables
430+
search_order: ['dbt', 'dbt_external_tables']
431+
```
432+
* To define `STOREDAS` and `ROWFORMAT` for in dbt-external tables, one of the below options can be used:
433+
* user can use the standard dbt-external-tables config `file_format` and `row_format` respectively
434+
* Or user can just add it in `USING` config as mentioned in the Teradata's [documentation](https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Data-Definition-Language-Syntax-and-Examples/Table-Statements/CREATE-FOREIGN-TABLE/CREATE-FOREIGN-TABLE-Syntax-Elements/USING-Clause)
435+
436+
* For external source, which requires authentication, user needs to create authentication object and pass it in `tbl_properties` as `EXTERNAL SECURITY` object.
437+
For more information on Authentication object please follow this [link](https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Data-Definition-Language-Syntax-and-Examples/Authorization-Statements-for-External-Routines/CREATE-AUTHORIZATION-and-REPLACE-AUTHORIZATION)
438+
439+
* Sample external sources are provided below as references
440+
```yaml
441+
version: 2
442+
sources:
443+
- name: teradata_external
444+
schema: "{{ target.schema }}"
445+
loader: S3
446+
447+
tables:
448+
- name: people_csv_partitioned
449+
external:
450+
location: "/s3/s3.amazonaws.com/dbt-external-tables-testing/csv/"
451+
file_format: "TEXTFILE"
452+
row_format: '{"field_delimiter":",","record_delimiter":"\n","character_set":"LATIN"}'
453+
using: |
454+
PATHPATTERN ('$var1/$section/$var3')
455+
tbl_properties: |
456+
MAP = TD_MAP1
457+
,EXTERNAL SECURITY MyAuthObj
458+
partitions:
459+
- name: section
460+
data_type: CHAR(1)
461+
columns:
462+
- name: id
463+
data_type: int
464+
- name: first_name
465+
data_type: varchar(64)
466+
- name: last_name
467+
data_type: varchar(64)
468+
- name: email
469+
data_type: varchar(64)
470+
```
471+
472+
```yaml
473+
version: 2
474+
sources:
475+
- name: teradata_external
476+
schema: "{{ target.schema }}"
477+
loader: S3
478+
479+
tables:
480+
- name: people_json_partitioned
481+
external:
482+
location: '/s3/s3.amazonaws.com/dbt-external-tables-testing/json/'
483+
using: |
484+
STOREDAS('TEXTFILE')
485+
ROWFORMAT('{"record_delimiter":"\n", "character_set":"cs_value"}')
486+
PATHPATTERN ('$var1/$section/$var3')
487+
tbl_properties: |
488+
MAP = TD_MAP1
489+
,EXTERNAL SECURITY MyAuthObj
490+
partitions:
491+
- name: section
492+
data_type: CHAR(1)
493+
```
494+
495+
## Fallback Schema
496+
dbt-teradata internally created temporary tables to fetch the metadata of views for manifest and catalog creation. In case if user does not have permission to create tables on the schema they are working on, they can define a fallback_schema(to which they have proper create/drop privileges) in dbt_project.yml as variable.
497+
498+
```yaml
499+
vars:
500+
fallback_schema: <schema-name>
501+
```

0 commit comments

Comments
 (0)