Creating pip package for TensorFlow with GPU support results in 0 byte simple_console_for_windows.zip

user_529612 0
Viewed 1298

After successfully building TensorFlow with GPU support, I'm trying to build the pip package and I'm getting an error saying it can't read the simple_console_for_windows.zip file.

I've confirmed that the file is in C:\tensorflow\bazel-bin\tensorflow\tools\pip_package folder, but it is 0 bytes.

This is my pip build command:

bazel-bin\tensorflow\tools\pip_package\build_pip_package C:/tmp/tensorflow_pkg

This is the full error:

Unzipping simple_console_for_windows.zip to create runfiles tree...
[./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip or
        ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip.zip, and cannot find ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip.ZIP, period.

I'm on Windows 10, using Bazel 0.16.1, Tensorflow 1.11, CUDA 9.2 and CUDNN 7.2.1.

My build command was:

bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

Anyone have an idea what I could try or check to get the pip package to build?

bazel tensorflow windows
2 Answers

Here is an open issue with workaround. https://github.com/tensorflow/tensorflow/issues/20332

  1. Edit the following file to remove all the lines with ".zip".

    bazel-out/x64_windows-opt/bin/tensorflow/tools/pip_package/simple_console_for_windows.zip-0.params

  2. Run the following command line to generate the required zip file, which is about 107 MB in my case. The verbose option will list all the files.

    external\bazel_tools\tools\zip\zipper\zipper.exe vcC bazel-out/x64_windows-opt/bin/tensorflow/tools/pip_package/simple_console_for_windows.zip @bazel-out/x64_windows-opt/bin/tensorflow/tools/pip_package/simple_console_for_windows.zip-0.params

  3. Now the original pip build command will build a whl.

user_79229980

This is a known issue, and unfortunately its root cause is https://github.com/tensorflow/tensorflow/issues/22390 TF is a big codebase, and when the size of the zip file exceeds 2 GBs, bazel's zipping tools break because they only use zip32.

The ultimate solutions are to contribute a fix to bazel to use zip64, and TensorFlow to lose some weight, the package has grown substantially lately. However, bazel team was able to give us a workaround. Only when building the pip package, you should add --define=no_tensorflow_py_deps=true to your bazel command. You will still not be able to build a debug binary, but at least you will be able to build a release binary.

user_7320030
Related