Skip to content

Commit b55f106

Browse files
added documentation for external-tables and fallback_schema (#7610)
2 parents 158deeb + 40670b6 commit b55f106

File tree

1 file changed

+91
-0
lines changed

1 file changed

+91
-0
lines changed

website/docs/reference/resource-configs/teradata-configs.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -413,3 +413,94 @@ These steps collectively ensure that the valid_history strategy effectively mana
413413
)}}
414414
```
415415
See [Collecting Statistics documentation](https://docs.teradata.com/r/76g1CuvvQlYBjb2WPIuk3g/RAyUdGfvREwbO9J0DMNpLw) for more information.
416+
417+
## The external tables package
418+
419+
The [dbt-external-tables](https://github.com/dbt-labs/dbt-external-tables) package is supported with the dbt-teradata adapter from v1.9.3 onwards. Under the hood, dbt-teradata uses the concept of foreign tables to create tables from external sources. More information can be found in the [Teradata documentation](https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Data-Definition-Language-Syntax-and-Examples/Table-Statements/CREATE-FOREIGN-TABLE).
420+
421+
You need to add the `dbt-external-tables` package as a dependency:
422+
423+
```yaml
424+
packages:
425+
- package: dbt-labs/dbt_external_tables
426+
version: [">=0.9.0", "<1.0.0"]
427+
```
428+
429+
You need to add the dispatch config for the project to pick the overridden macros from the dbt-teradata package:
430+
431+
```yaml
432+
dispatch:
433+
- macro_namespace: dbt_external_tables
434+
search_order: ['dbt', 'dbt_external_tables']
435+
```
436+
437+
To define `STOREDAS` and `ROWFORMAT` for external tables, one of the following options can be used:
438+
- You can use the standard dbt-external-tables config `file_format` and `row_format` respectively.
439+
- Or you can add it in the `USING` config as mentioned in the [Teradata documentation](https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Data-Definition-Language-Syntax-and-Examples/Table-Statements/CREATE-FOREIGN-TABLE/CREATE-FOREIGN-TABLE-Syntax-Elements/USING-Clause).
440+
441+
For the external sources, which require authentication, you need to create an authentication object and pass it in `tbl_properties` as `EXTERNAL SECURITY` object. For more information on authentication objects, check out the [Teradata documentation](https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Data-Definition-Language-Syntax-and-Examples/Authorization-Statements-for-External-Routines/CREATE-AUTHORIZATION-and-REPLACE-AUTHORIZATION).
442+
443+
The following are examples of external sources configured for Teradata:
444+
445+
```yaml
446+
version: 2
447+
sources:
448+
- name: teradata_external
449+
schema: "{{ target.schema }}"
450+
loader: S3
451+
452+
tables:
453+
- name: people_csv_partitioned
454+
external:
455+
location: "/s3/s3.amazonaws.com/dbt-external-tables-testing/csv/"
456+
file_format: "TEXTFILE"
457+
row_format: '{"field_delimiter":",","record_delimiter":"\n","character_set":"LATIN"}'
458+
using: |
459+
PATHPATTERN ('$var1/$section/$var3')
460+
tbl_properties: |
461+
MAP = TD_MAP1
462+
,EXTERNAL SECURITY MyAuthObj
463+
partitions:
464+
- name: section
465+
data_type: CHAR(1)
466+
columns:
467+
- name: id
468+
data_type: int
469+
- name: first_name
470+
data_type: varchar(64)
471+
- name: last_name
472+
data_type: varchar(64)
473+
- name: email
474+
data_type: varchar(64)
475+
```
476+
477+
```yaml
478+
version: 2
479+
sources:
480+
- name: teradata_external
481+
schema: "{{ target.schema }}"
482+
loader: S3
483+
484+
tables:
485+
- name: people_json_partitioned
486+
external:
487+
location: '/s3/s3.amazonaws.com/dbt-external-tables-testing/json/'
488+
using: |
489+
STOREDAS('TEXTFILE')
490+
ROWFORMAT('{"record_delimiter":"\n", "character_set":"cs_value"}')
491+
PATHPATTERN ('$var1/$section/$var3')
492+
tbl_properties: |
493+
MAP = TD_MAP1
494+
,EXTERNAL SECURITY MyAuthObj
495+
partitions:
496+
- name: section
497+
data_type: CHAR(1)
498+
```
499+
500+
### Fallback Schema
501+
The dbt-teradata adapter internally creates temporary tables to fetch the metadata of views for manifest and catalog creation. In cases you don't have permission to create tables on the schema you are working on, you can define a fallback_schema (to which you have the proper `create`/`drop` privileges) in the dbt_project.yml as a variable.
502+
503+
```yaml
504+
vars:
505+
fallback_schema: <schema-name>
506+
```

0 commit comments

Comments
 (0)