I ran into this with our rules_pyz rules, but since we basically use a version of this tool ourselves, I decided I might as well submit this improvement upstream: https://github.com/TriggerMail/rules_pyz

Got the same problem. Could you please rebase the change, so that I can verify it?

I just rebased this change. I also updated the version of wheel that is used by the test to the latest version.

I found one issue that does not work. Look at the last line of the generated BUILD file:

# ... /external/pypi__gapic_google_cloud_logging_v2_0_91_3/BUILD

package(default_visibility = ["//visibility:public"])

load("@ros_log_deps//:requirements.bzl", "requirement")

py_library(
  name = "pkg",
  srcs = glob(["**/*.py"]),
  data = glob(["**/*"], exclude=["**/*.py", "**/* *", "BUILD", "WORKSPACE"]),
  # This makes this directory a top-level in the python import
  # search path for anything that depends on this.
  imports = ["."],
  deps = [requirement("google-gax"),requirement("googleapis-common-protos[grpc]"),requirement("oauth2client"),requirement("proto-google-cloud-logging-v2[grpc]"
)],
     )

this results into:

key "googleapis_common_protos[grpc]" not found in dictionary

Also, do tools/piptool.par and tools/whltool.par need to be updated to include these changes?

any ETA when this will be resolved?

Soon! Jon will start working on this around middle August. Sorry for the continuing delays.

whl.py should not have any Python2-only syntax (such as basestring) -- the problem is that when you parse requirements with pkg_resources.evaluate_marker it will use the current script's Python version. A lot of libraries have extra requires if it's not Python3, so, when running whltool.par under Python2 you end up adding unresolvable requirements to the BUILD file. For example requirement('futures') which is a builtin Python3 lib that's been backported.

Note if whltool.par is Python2 then we will always only get Python2 dependencies.

I think the right way to do is is to merge #82 first and then use six to make whl.py Python2/3 compatible.

Hello! Are there any updates on this?

The workaround to declare transitive dependencies manually is only so much maintainable...

@brandjon for thoughts?

I merged this PR into my fork and I do see the BUILD files in the ./bazel-<proj>/external/* directories now contain the transitive dependencies, but they don't seem to be automatically picked up. Any idea what I need to add to my BUILD files?

I take that back. This does generate dependencies for some packages, but not for others. For example, the BUILD file for Flask-1.0.2 looks like this:

package(default_visibility = ["//visibility:public"])

load("@my_deps//:requirements.bzl", "requirement")

py_library(
    name = "pkg",
    srcs = glob(["**/*.py"]),
    data = glob(["**/*"], exclude=["**/*.py", "**/* *", "BUILD", "WORKSPACE"]),
    # This makes this directory a top-level in the python import
    # search path for anything that depends on this.
    imports = ["."],
    deps = [],
)

but deps should really be [requirement("Werkzeug"),requirement("Jinja2"),requirement("itsdangerous"),requirement("click")]. For Flask-RESTful-0.3.6, the BUILD looks like this:

package(default_visibility = ["//visibility:public"])

load("@my_deps//:requirements.bzl", "requirement")

py_library(
    name = "pkg",
    srcs = glob(["**/*.py"]),
    data = glob(["**/*"], exclude=["**/*.py", "**/* *", "BUILD", "WORKSPACE"]),
    # This makes this directory a top-level in the python import
    # search path for anything that depends on this.
    imports = ["."],
    deps = [requirement("Flask"),requirement("aniso8601"),requirement("pytz"),requirement("six")],
)

which is correct.

What is the status of this? @buchgr

@nayak16 I am not familiar with this change. Wrong ping?

@brandjon Any chance you can review this in connection to your triage of #70?

@evanj @brandjon What's the latest on this PR? We're interested in this, as we have had to pause any Python dependency updates for effected dependencies. E.g. kubernetes-client.

I couldn't convince my organization to use Bazel, so I no longer have the time to figure out how to get Python to work with it, sorry.

#121 works as-is with python2, but doesn't work on its latest commits with python3.

@brandjon Do you have any advice for how to move forward? If this PR is rebased is it mergeable?

Wondering the same. What is needed to get this merged?