Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Incorrect column data type with incremental contracted model and varchar data_type #10362

Closed
gshank opened this issue Jun 25, 2024 · 1 comment · Fixed by #10371
Closed
Assignees
Labels
bug Something isn't working

Comments

@gshank
Copy link
Contributor

gshank commented Jun 25, 2024

Current Behavior

A temporary table is created for incremental models when the table already exists. When this temporary table is created and one of the column contracted data_types is "character varying(1)" (size is not significant) the string size is lost in the creation of the temporary relation and instead the column is altered after the temporary table is created like: alter table "xxx"."xxxx"."xxx" add column "<column_name>__dbt_alter" character varying(256);

The column is created with the wrong string size.

Expected Behavior

The column is created with the string size that's specified in the contract.

Steps to recreate

Create an incremental model, update it with additional rows (causing the creation of a temporary table) and observe the the wrong string size is used.

Additional Context

Reported internally by Lee Bond-Kennedy.

@gshank gshank added the bug Something isn't working label Jun 25, 2024
@gshank gshank changed the title Incorrect column data type with incremental contracted model and varchar data_type [Bug] Incorrect column data type with incremental contracted model and varchar data_type Jun 25, 2024
@lbk-fishtown
Copy link

lbk-fishtown commented Jun 26, 2024

I was able to reproduce this for both varchar(1) and character varying(1). In both cases running dbt-postgres 1.6.16 the following occurs on the incremental run.

Both models are the same SQL

with source_data as (

    select 1 as id, 'a' as vchar

)

select *
from source_data

yml

models:
  - name: my_first_dbt_model
    config:
      materialized: incremental
      on_schema_change: append_new_columns
      contract:
        enforced: true
    description: "A starter dbt model"
    columns:
      - name: id
        description: "The primary key for this table"
        data_type: int
      - name: vchar
        description: "Test varying char contract"
        data_type: character varying(1)

  - name: my_other_model
    config:
      materialized: incremental
      on_schema_change: append_new_columns
      contract:
        enforced: true
    columns:
      - name: id
        description: "The primary key for this table"
        data_type: int
      - name: vchar
        description: "Test varying char contract"
        data_type: varchar(1)

logs

^[[0m10:17:46.720110 [debug] [Thread-1 (]: SQL status: SELECT 2 in 0.0 seconds
^[[0m10:17:46.720852 [debug] [Thread-1 (]: Changing col type from character varying(1) to character varying(256) in table database: "lee"
schema: "lee"
identifier: "my_first_dbt_model"

^[[0m10:17:46.724173 [debug] [Thread-1 (]: Using postgres connection "model.pgtest.my_first_dbt_model"
^[[0m10:17:46.724366 [debug] [Thread-1 (]: On model.pgtest.my_first_dbt_model: /* {"app": "dbt", "dbt_version": "1.6.16", "profile_name": "pgtest", "target_name": "dev", "node_id": "model.pgtest.my_first_dbt_model"} */

    alter table "lee"."lee"."my_first_dbt_model" add column "vchar__dbt_alter" character varying(256);
    update "lee"."lee"."my_first_dbt_model" set "vchar__dbt_alter" = "vchar";
    alter table "lee"."lee"."my_first_dbt_model" drop column "vchar" cascade;
    alter table "lee"."lee"."my_first_dbt_model" rename column "vchar__dbt_alter" to "vchar"
^[[0m10:21:12.614573 [debug] [Thread-1 (]: SQL status: SELECT 2 in 0.0 seconds
^[[0m10:21:12.615230 [debug] [Thread-1 (]: Changing col type from character varying(1) to character varying(256) in table database: "lee"
schema: "lee"
identifier: "my_other_model"

^[[0m10:21:12.618429 [debug] [Thread-1 (]: Using postgres connection "model.pgtest.my_other_model"
^[[0m10:21:12.618613 [debug] [Thread-1 (]: On model.pgtest.my_other_model: /* {"app": "dbt", "dbt_version": "1.6.16", "profile_name": "pgtest", "target_name": "dev", "node_id": "model.pgtest.my_other_model"} */

    alter table "lee"."lee"."my_other_model" add column "vchar__dbt_alter" character varying(256);
    update "lee"."lee"."my_other_model" set "vchar__dbt_alter" = "vchar";
    alter table "lee"."lee"."my_other_model" drop column "vchar" cascade;
    alter table "lee"."lee"."my_other_model" rename column "vchar__dbt_alter" to "vchar"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants