Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Allow partitioned tables in IMMVs #100

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jasonmp85
Copy link

This isn't feature-complete yet, but opening as a place to start a conversation around how to fully implement this feature.

At the moment, the prohibition on partitioned tables is removed, and hooks are added to handle ATTACH PARTITION and DETACH PARTITION.

To do:

  • Add check to prohibit multiple partitioned tables (causes crash)
  • Change ATTACH/DETACH to be incremental (presently does a REFRESH)
  • Figure out issue with multiple tables and correct it

This isn't feature-complete yet, but opening as a place to start a con-
versation around how to fully implement this feature.

At the moment, the prohibition on partitioned tables is removed, and
hooks are added to handle ATTACH PARTITION and DETACH PARTITION.

To do:

  * Add check to prohibit multiple partitioned tables (causes crash)
  * Change ATTACH/DETACH to be incremental (presently does a REFRESH)
  * Figure out issue with multiple tables and correct it
@yugo-n
Copy link
Collaborator

yugo-n commented Oct 19, 2024

Thank you for the pull request.
I understood that this is for discussing how to implement this feature.

At the moment, the prohibition on partitioned tables is removed, and hooks are added to handle ATTACH PARTITION and DETACH PARTITION.

As you suggested in this pull request, there are a few points to be discussed.

What the problem is when we remove the prohibition on partitioned tables, leaving ATTACH or DETACH aside for now.
As you reported, this can cause a crash, but there might be other issues we've not found yet. I'll investigate this more deeply.

As you proposed, we would need to consider ATTACH and DETACH support. From the view of parent partitioned table, this may seem insert and delete of batch of rows, so we might be able to handle this incrementally. However, when the size of attached or detached partition is large, it will takes a long time, so IVM would be not efficiently. In this case, indeed REFRESH might be better. I have no concrete idea whether we should control this, and if so how we can for now.

In addition, we might need to consider conditions where;

  • a partition is updated directly instead of the parent partitioned table
  • we use cascading partitioned table
  • a partition and its parent have different column order.
  • and so on.

Copy link
Collaborator

@yugo-n yugo-n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've commented above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants