Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor!: unify FrontendOptions and DatanodeOptions by using GrpcOptions #4088

Merged
merged 11 commits into from
Jun 18, 2024

Conversation

realtaobo
Copy link
Contributor

@realtaobo realtaobo commented Jun 2, 2024

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

As mentioned in the comment at #3957 (comment), try to unify FrontendOptions and DatanodeOptions by using GrpcOptions.

What's changed and what's your intention?

 *  Executing task: cargo test --package cmd --lib -- datanode::tests::test_deprecated_cli_options --exact --show-output 
...
running 1 test
2024-06-17T15:10:58.675193Z  INFO common_telemetry::logging: logs dir = /tmp/__unittest_logs
2024-06-17T15:10:58.682675Z  WARN cmd::datanode: Use the deprecated attribute `DatanodeOptions.rpc_addr`, please use `grpc.addr` instead.
2024-06-17T15:10:58.682740Z  WARN cmd::datanode: Use the deprecated attribute `DatanodeOptions.rpc_hostname`, please use `grpc.hostname` instead.
test datanode::tests::test_depreacted_cli_options ... ok

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.

@realtaobo realtaobo requested a review from a team as a code owner June 2, 2024 05:16
@github-actions github-actions bot added the docs-not-required This change does not impact docs. label Jun 2, 2024
Copy link

codecov bot commented Jun 2, 2024

Codecov Report

Attention: Patch coverage is 86.36364% with 12 lines in your changes missing coverage. Please review.

Project coverage is 84.85%. Comparing base (0fc18b6) to head (df5e284).
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4088      +/-   ##
==========================================
- Coverage   84.92%   84.85%   -0.07%     
==========================================
  Files        1023     1020       -3     
  Lines      179298   179296       -2     
==========================================
- Hits       152272   152146     -126     
- Misses      27026    27150     +124     

@zyy17
Copy link
Collaborator

zyy17 commented Jun 3, 2024

Most of the changes are look good to me, except:

  1. It's an breaking changes and it's worth for it? Or we should make it compatible?
  2. The PR failed the fuzz tests;

@realtaobo

This comment was marked as outdated.

src/datanode/src/config.rs Outdated Show resolved Hide resolved
src/datanode/src/config.rs Show resolved Hide resolved
@evenyag evenyag changed the title refactor: unify FrontendOptions and DatanodeOptions by using GrpcOptions refactor!: unify FrontendOptions and DatanodeOptions by using GrpcOptions Jun 4, 2024
@github-actions github-actions bot added the breaking-change This pull request contains breaking changes. label Jun 4, 2024
@evenyag
Copy link
Contributor

evenyag commented Jun 8, 2024

Since many people may already set the old rpc_addr, we might still keep these options but notify users that they are deprecated.

@realtaobo
Copy link
Contributor Author

Since many people may already set the old rpc_addr, we might still keep these options but notify users that they are deprecated.

Yes, I'm aware of this, but still exist some other questions for fuzz tests within distributed mode. I'll ping again when everything is ready.

@evenyag
Copy link
Contributor

evenyag commented Jun 11, 2024

Since many people may already set the old rpc_addr, we might still keep these options but notify users that they are deprecated.

Yes, I'm aware of this, but still exist some other questions for fuzz tests within distributed mode. I'll ping again when everything is ready.

I guess some of the fuzz tests use the old rpc options.

@realtaobo
Copy link
Contributor Author

realtaobo commented Jun 17, 2024

Hi, all, please take a look when you have some free time.

  1. It's an breaking changes and it's worth for it? Or we should make it compatible?

This PR moves GrpcOptions to servers/src/grpc and refactors the gRPC-related configurations of DatanodeOptions. Additionally, I have tagged rpc_addr, rpc_runtime_size, rpc_hostname, and rpc_max_recv/send_message_size as deprecated fields, setting them to None by default and printing a warning if they are set (The execution result example can be found in the description). Moreover, if the old RPC options are set, we read them first for compatibility.

  1. The PR failed the fuzz tests;

It has been resolved. I guess the distributed fuzz tests failed previously because they were not compatible with the greptimedb cluster's configuration used by CI.

Copy link
Contributor

@evenyag evenyag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

src/servers/src/grpc.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Thanks a lot.

@killme2008 killme2008 added this pull request to the merge queue Jun 18, 2024
Merged via the queue into GreptimeTeam:main with commit 22d1268 Jun 18, 2024
59 checks passed
@realtaobo realtaobo deleted the unify-options branch June 19, 2024 02:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change This pull request contains breaking changes. docs-not-required This change does not impact docs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants