Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bazel: Migrate py_proto_library to @com_github_grpc_grpc #30834

Merged
merged 1 commit into from
Nov 16, 2023

Conversation

sergiitk
Copy link
Contributor

@sergiitk sergiitk commented Nov 11, 2023

py_proto_library provided by @com_google_protobuf//:protobuf.bzl has been deprecated for a while now:

This is provided for backwards compatibility only. Bazel 5.3 will
introduce support for py_proto_library, which should be used instead.
https://github.com/protocolbuffers/protobuf/blob/32af7d211b85f71920acdfa9b796db008f8c2a45/protobuf.bzl#L642-L644

However, native py_proto_library has never been provided, see bazelbuild/bazel#3935. Instead @rules_python//python:proto.bzl is recommended. I attempted switching to this library, but it's not compatible with @com_google_googleapis's py_proto_library targets, see cncf/xds#69. I found a hacky workaround by using cc_proto_library to generate python targets, but downstream integration into Envoy failed (#30159).

This PR migrates py_proto_library implementation to to @com_github_grpc_grpc. This implementation is used by @com_google_googleapis's, and, more importantly, uses bazel aspects. Which decouples cncf/xds and Envoy's dependencies from concrete upstream py_proto_library implementations.

This resulted in a significant code improvement of bazel/api_build_system.bzl:

  • No more custom @com_google_googleapis dependency mapping needed via py_proto_library rules override.
  • No more hardcoded dependencies _xds_py_proto_library - proto dependency tree is determined from proto_library definitions via Basel aspects.
  • No more EXTERNAL_PROTO_PY_BAZEL_DEP_MAP dependency map needed - for the same reason.

Related PR in cncf/xds: cncf/xds#74.


Commit Message:

bazel: Migrate py_proto_library to @com_github_grpc_grpc

Additional Description:

`py_proto_library` provided by `@com_google_protobuf//:protobuf.bzl` has
been deprecated for a while now:

> This is provided for backwards compatibility only. Bazel 5.3 will
> introduce support for py_proto_library, which should be used
  instead.\
> https://github.com/protocolbuffers/protobuf/blob/32af7d211b85f71920acdfa9b796db008f8c2a45/protobuf.bzl#L642-L644

However, native `py_proto_library` has never been provided, see
https://github.com/bazelbuild/bazel/issues/3935. Instead
`@rules_python//python:proto.bzl` is recommended
(https://github.com/bazelbuild/bazel/issues/3935#issuecomment-1680850640).
I attempted switching to this library, but it's not compatible with
`@com_google_googleapis`'s py_proto_library targets, see
https://github.com/cncf/xds/pull/69. I found a hacky workaround by
using `cc_proto_library` to generate python targets, but downstream
integration into Envoy failed
(https://github.com/envoyproxy/envoy/pull/30159).

This PR migrates `py_proto_library` implementation to to
`@com_github_grpc_grpc`. This implementation is used by
`@com_google_googleapis`'s, and, more importantly, uses bazel aspects.
Which decouples cncf/xds and Envoy's dependencies from concrete
upstream py_proto_library implementations.

This resulted in a significant code improvement of
`bazel/api_build_system.bzl`:
- No more custom `@com_google_googleapis` dependency mapping needed via
  `py_proto_library` rules override.
- No more hardcoded dependencies `_xds_py_proto_library` - proto
  dependency tree is determined from `proto_library` definitions via
  Basel aspects.
- No more `EXTERNAL_PROTO_PY_BAZEL_DEP_MAP` dependency map needed - for
  the same reason.

Related PR in cncf/xds: https://github.com/cncf/xds/pull/74.

Risk Level: Low
Testing: do_ci.sh
Docs Changes: no
Release Notes: no

Copy link

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #30834 was opened by sergiitk.

see: more, trace.

@repokitteh-read-only repokitteh-read-only bot added the deps Approval required for changes to Envoy's external dependencies label Nov 11, 2023
Copy link

CC @envoyproxy/dependency-shepherds: Your approval is needed for changes made to (bazel/.*repos.*\.bzl)|(bazel/dependency_imports\.bzl)|(api/bazel/.*\.bzl)|(.*/requirements\.txt)|(.*\.patch).
envoyproxy/dependency-shepherds assignee is @mattklein123

🐱

Caused by: #30834 was opened by sergiitk.

see: more, trace.

Comment on lines +42 to +43
version = "83c031ea693357fdf7a7fea9a68a785d97864a38",
sha256 = "35bdcdbcd2e437058ad1f74a91b49525339b028a6b52760ab019fd9efa5a6d44",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this will be updated once cncf/xds#74 is merged.

@sergiitk sergiitk marked this pull request as ready for review November 15, 2023 18:55
@sergiitk
Copy link
Contributor Author

This is ready for a review now.
FYI @htuch @phlax @moderation @adisuissa

@yanavlasov yanavlasov merged commit 602d63f into envoyproxy:main Nov 16, 2023
109 of 110 checks passed
htuch pushed a commit to cncf/xds that referenced this pull request Nov 16, 2023
py_proto_library provided by @com_google_protobuf//:protobuf.bzl has been deprecated for a while now:

This is provided for backwards compatibility only. Bazel 5.3 will
introduce support for py_proto_library, which should be used instead.
https://github.com/protocolbuffers/protobuf/blob/32af7d211b85f71920acdfa9b796db008f8c2a45/protobuf.bzl#L642-L644

However, native py_proto_library has never been provided, see bazelbuild/bazel#3935. Instead @rules_python//python:proto.bzl is recommended. I attempted switching to this library, but it's not compatible with @com_google_googleapis's py_proto_library targets, see #69. I found a hacky workaround by using cc_proto_library to generate python targets, but downstream integration into Envoy failed (envoyproxy/envoy#30159).

This PR migrates py_proto_library implementation to to @com_github_grpc_grpc. This implementation is used by @com_google_googleapis's, and, more importantly, uses bazel aspects. Which decouples cncf/xds and Envoy's dependencies from concrete upstream py_proto_library implementations.

This resulted in a significant code improvement of bazel/api_build_system.bzl:

No more custom @com_google_googleapis dependency mapping needed via py_proto_library rules override.
No more hardcoded dependencies _xds_py_proto_library - proto dependency tree is determined from proto_library definitions via Basel aspects.
No more EXTERNAL_PROTO_PY_BAZEL_DEP_MAP dependency map needed - for the same reason.
Similar work in Envoy: envoyproxy/envoy#30834.

Signed-off-by: Sergii Tkachenko <[email protected]>
@sergiitk
Copy link
Contributor Author

Hey @yanavlasov - we needed to merge cncf/xds first: #30834 (comment).
I'll make a follow up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deps Approval required for changes to Envoy's external dependencies
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants