Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(interactive): Implement HttpIrMetaReader to Get Meta Data From Remote Http Service #3908

Merged
merged 44 commits into from
Jun 26, 2024

Conversation

shirly121
Copy link
Collaborator

@shirly121 shirly121 commented Jun 11, 2024

What do these changes do?

  1. Implement HttpIrMetaReader to get meta data from remote http service.
  2. Implement DynamicIrMetaFetcher to fetch schema or statistics periodically, the interval can be set by configurations.

Related issue number

Fixes

siyuan0322
siyuan0322 previously approved these changes Jun 11, 2024
@@ -1,45 +0,0 @@
/*
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file should be completely removed?

@@ -23,6 +23,13 @@ compiler:
- FilterIntoJoinRule
- FilterMatchRule
- NotMatchToAntiJoinRule
meta:
reader:
mode: local
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still runs in local mode?

@@ -61,3 +61,15 @@ calcite.default.charset: UTF-8

# set the max capacity of the result streaming buffer for each query
# per.query.stream.buffer.max.capacity: 256

# set the mode to read ir meta, either from a 'local' file or via remote 'http'.
# ir.meta.reader.mode: local
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The configuration of ir.meta.xx, should be uniformly defined with graph.schema and graph.statistics. In addition, we can really configure those service scheme, to automatically differentiate from local and http. For example,

graph.meta.statistics.uri: http:https://localhost:8080 -> this is a http file, or
graph.meta.statistics.uri: file:https:///path/to/local/file -> this is a local file

Also give a uri for schema

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

BingqingLyu
BingqingLyu previously approved these changes Jun 13, 2024
siyuan0322
siyuan0322 previously approved these changes Jun 25, 2024
Committed-by: xiaolei.zl from Dev container
Committed-by: xiaolei.zl from Dev container
@siyuan0322 siyuan0322 merged commit 02c3bfa into alibaba:main Jun 26, 2024
58 of 59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants