From 3db4ce36de64f71599774368b07f46507912c2c2 Mon Sep 17 00:00:00 2001 From: "Talla, Mohan" Date: Wed, 4 Sep 2024 15:06:58 +0530 Subject: [PATCH 1/4] updated README and CHANGELOG --- CHANGELOG.md | 11 +++++ README.md | 119 +++++++++++++++++++++++++++++---------------------- 2 files changed, 80 insertions(+), 50 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a5cd4776..734d52bf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,20 @@ ## dbt-teradata 1.0.0a ### Features +* dbt-teradata is now fully compatible with Python 3.12. +* Added support for browser-based Single Sign-On (SSO) authentication when connecting to SSO-enabled Teradata databases. Please refer to the README for more details. ### Fixes +* Resolved an issue with the 'show' command. + * https://github.com/dbt-labs/dbt-adapters/pull/249 +* Fixed a TypeError in TeradataAdapter.get_catalog() where it incorrectly took three arguments instead of two. + * https://github.com/Teradata/dbt-teradata/issues/180 +* Corrected exception handling for failed database connections. + * https://github.com/Teradata/dbt-teradata/issues/183 ### Docs +* Updated the README to include additional Teradata profile fields and threads options. ### Under the hood +* Expanded test coverage for snapshots. +* Verified the compatibility of dbt's threads feature with dbt-teradata. diff --git a/README.md b/README.md index a28e1a64..e0ce4fc5 100644 --- a/README.md +++ b/README.md @@ -45,20 +45,20 @@ At a minimum, you need to specify `host`, `user`, `password`, `schema` (database ## Python compatibility -| Plugin version | Python 3.6 | Python 3.7 | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 | -|----------------| ----------- | ----------- | ----------- | ----------- | ----------- | ------------ | -| 0.19.0.x | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ -| 0.20.0.x | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ -| 0.21.1.x | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ -| 1.0.0.x | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ -| 1.1.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ -| 1.2.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ -| 1.3.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ -| 1.4.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ -| 1.5.x | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ -| 1.6.x | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ -| 1.7.x | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ -| 1.8.x | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ +| Plugin version | Python 3.6 | Python 3.7 | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 | Python 3.12 | +|----------------| ----------- | ----------- | ----------- | ----------- | ----------- |-------------|-------------| +| 0.19.0.x | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ +| 0.20.0.x | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ +| 0.21.1.x | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ +| 1.0.0.x | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ +| 1.1.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ +| 1.2.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ +| 1.3.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ +| 1.4.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ +| 1.5.x | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ +| 1.6.x | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ +| 1.7.x | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ +| 1.8.x | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ ## dbt dependent packages version compatibility @@ -189,42 +189,61 @@ my-teradata-db-profile: retry_timeout: 10 ``` -### Other Teradata connection parameters - -The plugin also supports the following Teradata connection parameters: -* account -* column_name -* cop -* coplast -* encryptdata -* fake_result_sets -* field_quote -* field_sep -* lob_support -* log -* logdata -* max_message_body -* partition -* sip_support -* teradata_values -* sslmode -* sslca -* sslcapath -* sslcrc -* sslcipher -* sslprotocol -* browser -* browser_tab_timeout -* browser_timeout -* sp_spl -* sessions -* runstartup -* logon_timeout -* https_port -* connect_timeout -* request_timeout - -For full description of the connection parameters see https://github.com/Teradata/python-driver#connection-parameters. +### Description of Teradata Profile Fields + +The following fields are required: + +Parameter | Default | Type | Description +----------------------- |------------| -------------- | --- +`user` | | string | Specifies the database username. Equivalent to the Teradata JDBC Driver `USER` connection parameter. +`password` | | string | Specifies the database password. Equivalent to the Teradata JDBC Driver `PASSWORD` connection parameter. +`schema` | | string | Specifies the initial database to use after logon, instead of the user's default database. +`tmode` | `"ANSI"` | string | Specifies the transaction mode. Only `ANSI` mode is currently supported. + + +The plugin also supports the following optional connection parameters: + +Parameter | Default | Type | Description +----------------------- | ----------- | -------------- | --- +`account` | | string | Specifies the database account. Equivalent to the Teradata JDBC Driver `ACCOUNT` connection parameter. +`browser` | | string | Specifies the command to open the browser for Browser Authentication, when logmech is BROWSER. Browser Authentication is supported for Windows and macOS. Equivalent to the Teradata JDBC Driver BROWSER connection parameter. +`browser_tab_timeout` | `"5"` | quoted integer | Specifies the number of seconds to wait before closing the browser tab after Browser Authentication is completed. The default is 5 seconds. The behavior is under the browser's control, and not all browsers support automatic closing of browser tabs. +`browser_timeout` | `"180"` | quoted integer | Specifies the number of seconds that the driver will wait for Browser Authentication to complete. The default is 180 seconds (3 minutes). +`column_name` | `"false"` | quoted boolean | Controls the behavior of cursor `.description` sequence `name` items. Equivalent to the Teradata JDBC Driver `COLUMN_NAME` connection parameter. False specifies that a cursor `.description` sequence `name` item provides the AS-clause name if available, or the column name if available, or the column title. True specifies that a cursor `.description` sequence `name` item provides the column name if available, but has no effect when StatementInfo parcel support is unavailable. +`connect_failure_ttl` | `"0"` | quoted integer | Specifies the time-to-live in seconds to remember the most recent connection failure for each IP address/port combination. The driver subsequently skips connection attempts to that IP address/port for the duration of the time-to-live. The default value of zero disables this feature. The recommended value is half the database restart time. Equivalent to the Teradata JDBC Driver `CONNECT_FAILURE_TTL` connection parameter. +`connect_timeout` | `"10000"` | quoted integer | Specifies the timeout in milliseconds for establishing a TCP socket connection. Specify 0 for no timeout. The default is 10 seconds (10000 milliseconds). +`cop` | `"true"` | quoted boolean | Specifies whether COP Discovery is performed. Equivalent to the Teradata JDBC Driver `COP` connection parameter. +`coplast` | `"false"` | quoted boolean | Specifies how COP Discovery determines the last COP hostname. Equivalent to the Teradata JDBC Driver `COPLAST` connection parameter. When `coplast` is `false` or omitted, or COP Discovery is turned off, then no DNS lookup occurs for the coplast hostname. When `coplast` is `true`, and COP Discovery is turned on, then a DNS lookup occurs for a coplast hostname. +`port` | `"1025"` | quoted integer | Specifies the database port number. Equivalent to the Teradata JDBC Driver `DBS_PORT` connection parameter. +`encryptdata` | `"false"` | quoted boolean | Controls encryption of data exchanged between the driver and the database. Equivalent to the Teradata JDBC Driver `ENCRYPTDATA` connection parameter. +`fake_result_sets` | `"false"` | quoted boolean | Controls whether a fake result set containing statement metadata precedes each real result set. +`field_quote` | `"\""` | string | Specifies a single character string used to quote fields in a CSV file. +`field_sep` | `","` | string | Specifies a single character string used to separate fields in a CSV file. Equivalent to the Teradata JDBC Driver `FIELD_SEP` connection parameter. +`host` | | string | Specifies the database hostname. +`https_port` | `"443"` | quoted integer | Specifies the database port number for HTTPS/TLS connections. Equivalent to the Teradata JDBC Driver `HTTPS_PORT` connection parameter. +`lob_support` | `"true"` | quoted boolean | Controls LOB support. Equivalent to the Teradata JDBC Driver `LOB_SUPPORT` connection parameter. +`log` | `"0"` | quoted integer | Controls debug logging. Somewhat equivalent to the Teradata JDBC Driver `LOG` connection parameter. This parameter's behavior is subject to change in the future. This parameter's value is currently defined as an integer in which the 1-bit governs function and method tracing, the 2-bit governs debug logging, the 4-bit governs transmit and receive message hex dumps, and the 8-bit governs timing. Compose the value by adding together 1, 2, 4, and/or 8. +`logdata` | | string | Specifies extra data for the chosen logon authentication method. Equivalent to the Teradata JDBC Driver `LOGDATA` connection parameter. +`logon_timeout` | `"0"` | quoted integer | Specifies the logon timeout in seconds. Zero means no timeout. +`logmech` | `"TD2"` | string | Specifies the logon authentication method. Equivalent to the Teradata JDBC Driver `LOGMECH` connection parameter. Possible values are `TD2` (the default), `JWT`, `LDAP`, `KRB5` for Kerberos, or `TDNEGO`. +`max_message_body` | `"2097000"` | quoted integer | Specifies the maximum Response Message size in bytes. Equivalent to the Teradata JDBC Driver `MAX_MESSAGE_BODY` connection parameter. +`partition` | `"DBC/SQL"` | string | Specifies the database partition. Equivalent to the Teradata JDBC Driver `PARTITION` connection parameter. +`request_timeout` | `"0"` | quoted integer | Specifies the timeout for executing each SQL request. Zero means no timeout. +`retries` | `0` | integer | Allows an adapter to automatically try again when the attempt to open a new connection on the database has a transient, infrequent error. This option can be set using the retries configuration. Default value is 0. The default wait period between connection attempts is one second. retry_timeout (seconds) option allows us to adjust this waiting period. +`runstartup` | "false" | quoted boolean | Controls whether the user's STARTUP SQL request is executed after logon. For more information, refer to User STARTUP SQL Request. Equivalent to the Teradata JDBC Driver RUNSTARTUP connection parameter. If retries is set to 3, the adapter will try to establish a new connection three times if an error occurs. +`sessions` | | quoted integer | Specifies the number of data transfer connections for FastLoad or FastExport. The default (recommended) lets the database choose the appropriate number of connections. Equivalent to the Teradata JDBC Driver SESSIONS connection parameter. +`sip_support` | `"true"` | quoted boolean | Controls whether StatementInfo parcel is used. Equivalent to the Teradata JDBC Driver `SIP_SUPPORT` connection parameter. +`sp_spl` | `"true"` | quoted boolean | Controls whether stored procedure source code is saved in the database when a SQL stored procedure is created. Equivalent to the Teradata JDBC Driver SP_SPL connection parameter. +`sslca` | | string | Specifies the file name of a PEM file that contains Certificate Authority (CA) certificates for use with `sslmode` values `VERIFY-CA` or `VERIFY-FULL`. Equivalent to the Teradata JDBC Driver `SSLCA` connection parameter. +`sslcrc` | `"ALLOW"` | string | Equivalent to the Teradata JDBC Driver SSLCRC connection parameter. Values are case-insensitive.
• ALLOW provides "soft fail" behavior such that communication failures are ignored during certificate revocation checking.
• REQUIRE mandates that certificate revocation checking must succeed. +`sslcapath` | | string | Specifies a directory of PEM files that contain Certificate Authority (CA) certificates for use with `sslmode` values `VERIFY-CA` or `VERIFY-FULL`. Only files with an extension of `.pem` are used. Other files in the specified directory are not used. Equivalent to the Teradata JDBC Driver `SSLCAPATH` connection parameter. +`sslcipher` | | string | Specifies the TLS cipher for HTTPS/TLS connections. Equivalent to the Teradata JDBC Driver `SSLCIPHER` connection parameter. +`sslmode` | `"PREFER"` | string | Specifies the mode for connections to the database. Equivalent to the Teradata JDBC Driver `SSLMODE` connection parameter.
• `DISABLE` disables HTTPS/TLS connections and uses only non-TLS connections.
• `ALLOW` uses non-TLS connections unless the database requires HTTPS/TLS connections.
• `PREFER` uses HTTPS/TLS connections unless the database does not offer HTTPS/TLS connections.
• `REQUIRE` uses only HTTPS/TLS connections.
• `VERIFY-CA` uses only HTTPS/TLS connections and verifies that the server certificate is valid and trusted.
• `VERIFY-FULL` uses only HTTPS/TLS connections, verifies that the server certificate is valid and trusted, and verifies that the server certificate matches the database hostname. +`sslprotocol` | `"TLSv1.2"` | string | Specifies the TLS protocol for HTTPS/TLS connections. Equivalent to the Teradata JDBC Driver `SSLPROTOCOL` connection parameter. +`teradata_values` | `"true"` | quoted boolean | Controls whether `str` or a more specific Python data type is used for certain result set column value types. +`query_band` | `"org=teradata-internal-telem;appname=dbt;"` | string | Specifies the Query Band string to be set for each SQL request. + +Refer to [connection parameters](https://github.com/Teradata/python-driver#connection-parameters) for the full description of the connection parameters. ## Supported Features From 2658faf799c7e3d729dd9d58c159a05803920638 Mon Sep 17 00:00:00 2001 From: "Talla, Mohan" Date: Wed, 4 Sep 2024 15:15:45 +0530 Subject: [PATCH 2/4] Update README.md --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index e0ce4fc5..8f6080f8 100644 --- a/README.md +++ b/README.md @@ -62,12 +62,12 @@ At a minimum, you need to specify `host`, `user`, `password`, `schema` (database ## dbt dependent packages version compatibility -| dbt-teradata | dbt-core | dbt-teradata-util | dbt-util | +| dbt-teradata | dbt-core | dbt-teradata-util | dbt-util | |--------------|----------|-------------------|----------------| | 1.2.x | 1.2.x | 0.1.0 | 0.9.x or below | | 1.6.7 | 1.6.7 | 1.1.1 | 1.1.1 | | 1.7.x | 1.7.x | 1.1.1 | 1.1.1 | -| 1.8.x | 1.8.x | 1.1.1 | 1.1.1 | +| 1.8.x | 1.8.x | 1.2.0 | 1.2.0 | ## Optional profile configurations From 3497c4bb3cf08371db9b72368671009b73677578 Mon Sep 17 00:00:00 2001 From: "Talla, Mohan" Date: Wed, 4 Sep 2024 17:41:05 +0530 Subject: [PATCH 3/4] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8f6080f8..4e0624f4 100644 --- a/README.md +++ b/README.md @@ -225,7 +225,7 @@ Parameter | Default | Type | Description `log` | `"0"` | quoted integer | Controls debug logging. Somewhat equivalent to the Teradata JDBC Driver `LOG` connection parameter. This parameter's behavior is subject to change in the future. This parameter's value is currently defined as an integer in which the 1-bit governs function and method tracing, the 2-bit governs debug logging, the 4-bit governs transmit and receive message hex dumps, and the 8-bit governs timing. Compose the value by adding together 1, 2, 4, and/or 8. `logdata` | | string | Specifies extra data for the chosen logon authentication method. Equivalent to the Teradata JDBC Driver `LOGDATA` connection parameter. `logon_timeout` | `"0"` | quoted integer | Specifies the logon timeout in seconds. Zero means no timeout. -`logmech` | `"TD2"` | string | Specifies the logon authentication method. Equivalent to the Teradata JDBC Driver `LOGMECH` connection parameter. Possible values are `TD2` (the default), `JWT`, `LDAP`, `KRB5` for Kerberos, or `TDNEGO`. +`logmech` | `"TD2"` | string | Specifies the logon authentication method. Equivalent to the Teradata JDBC Driver `LOGMECH` connection parameter. Possible values are `TD2` (the default), `JWT`, `LDAP`, `BROWSER`, `KRB5` for Kerberos, or `TDNEGO`. `max_message_body` | `"2097000"` | quoted integer | Specifies the maximum Response Message size in bytes. Equivalent to the Teradata JDBC Driver `MAX_MESSAGE_BODY` connection parameter. `partition` | `"DBC/SQL"` | string | Specifies the database partition. Equivalent to the Teradata JDBC Driver `PARTITION` connection parameter. `request_timeout` | `"0"` | quoted integer | Specifies the timeout for executing each SQL request. Zero means no timeout. From 7154528a0842a7a1405575f48a54dab92317cd8a Mon Sep 17 00:00:00 2001 From: "Talla, Mohan" Date: Wed, 4 Sep 2024 17:55:12 +0530 Subject: [PATCH 4/4] Removed connect_failure_ttl --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 4e0624f4..fad31efc 100644 --- a/README.md +++ b/README.md @@ -210,7 +210,6 @@ Parameter | Default | Type | Description `browser_tab_timeout` | `"5"` | quoted integer | Specifies the number of seconds to wait before closing the browser tab after Browser Authentication is completed. The default is 5 seconds. The behavior is under the browser's control, and not all browsers support automatic closing of browser tabs. `browser_timeout` | `"180"` | quoted integer | Specifies the number of seconds that the driver will wait for Browser Authentication to complete. The default is 180 seconds (3 minutes). `column_name` | `"false"` | quoted boolean | Controls the behavior of cursor `.description` sequence `name` items. Equivalent to the Teradata JDBC Driver `COLUMN_NAME` connection parameter. False specifies that a cursor `.description` sequence `name` item provides the AS-clause name if available, or the column name if available, or the column title. True specifies that a cursor `.description` sequence `name` item provides the column name if available, but has no effect when StatementInfo parcel support is unavailable. -`connect_failure_ttl` | `"0"` | quoted integer | Specifies the time-to-live in seconds to remember the most recent connection failure for each IP address/port combination. The driver subsequently skips connection attempts to that IP address/port for the duration of the time-to-live. The default value of zero disables this feature. The recommended value is half the database restart time. Equivalent to the Teradata JDBC Driver `CONNECT_FAILURE_TTL` connection parameter. `connect_timeout` | `"10000"` | quoted integer | Specifies the timeout in milliseconds for establishing a TCP socket connection. Specify 0 for no timeout. The default is 10 seconds (10000 milliseconds). `cop` | `"true"` | quoted boolean | Specifies whether COP Discovery is performed. Equivalent to the Teradata JDBC Driver `COP` connection parameter. `coplast` | `"false"` | quoted boolean | Specifies how COP Discovery determines the last COP hostname. Equivalent to the Teradata JDBC Driver `COPLAST` connection parameter. When `coplast` is `false` or omitted, or COP Discovery is turned off, then no DNS lookup occurs for the coplast hostname. When `coplast` is `true`, and COP Discovery is turned on, then a DNS lookup occurs for a coplast hostname.