Bazel build python with multiple protobuf subpackages

Question

I have a directory structure as follows:

pb/
├─ service/
│  ├─ service.proto
│  ├─ BUILD
├─ common/
│  ├─ common.proto
│  ├─ BUILD

Each service defines a rules_proto_grpc targets for codegen

load("@rules_proto//proto:defs.bzl", "proto_library")
load("@rules_proto_grpc//python:python_proto_library.bzl", PY_PROTO_DEPS = "PROTO_DEPS")
load("@rules_python//python:defs.bzl", "py_library")

# gazelle: ignore
# gazelle: proto disable_global

proto_library(
    name = "common_proto",
    srcs = ["common.proto"],
    visibility = ["//visibility:public"],
)

python_proto_compile(
    name = "common_python_proto",
    protos = [":common_proto"],
    visibility = ["//visibility:public"],
)

py_library(
    name = "common_python_proto_lib",
    srcs = [":common_python_proto"],
    imports = ["common_python_proto"],
    visibility = ["//visibility:public"],
    deps = PY_PROTO_DEPS,
)

The problem is when python executable target depends on generated code like this

load("@rules_python//python:defs.bzl", "py_binary")

py_binary(
    name = "main",
    srcs = ["main.py"],
    main = "main.py",
    python_version = "PY3",
    visibility = ["//visibility:public"],
    deps = ["//pb/common:common_python_proto_lib", "//pb/service:service_python_grpc_lib"],
)

Each target creates it's own directory subtrees containing pb/common and pb/service folders which breaks python imports since it can import either common or service proto definitions.

I found a related issue at rules_proto_grpc repository https://github.com/rules-proto-grpc/rules_proto_grpc/issues/145 but it deals with conflicts with protos defined in the same directory.

What is the best approach to make generated python code resolve path correctly when each service is in it's own subpackage?

I suspect I need a parent level library in /pb directory but importing child libraries strips directories and breaks imports again.

Bazel build python with multiple protobuf subpackages

0 Answers