Tensorflow 2.3 unresolved external symbols in machine-generated files when building C++ project on Windows

Viewed 1078

I was wondering if anyone has had success linking against Tensorflow 2.3 DLL on Windows. I am trying to integrate some tensorflow functionality in a tiny VS2019 test project to see if I can use tensorflow C++ API at all.

  • I managed to build tensorflow with MSVC 14.16.27023 and followed the instructions given in the official links:
  • I built .dll and .lib using the following Bazel commands:
    • bazel build --config=opt //tensorflow:tensorflow.dll
    • bazel build --config=opt //tensorflow:tensorflow.lib
  • VS2019 project properties:
    • add relevant Additional Include Directories
    • add Additional Library Directories, which is only path-to-tensorflow-source\bazel-bin\tensorflow, where bazel-bin directory is generated by the build system (ie Bazel)
    • in Linker->Input->Additional Dependencies add generated tensorflow.lib

The source code is tiny and is comprised of a few lines I looked up in Joe Antognini's example:

#pragma once

#define NOMINMAX

#include "tensorflow/core/public/session.h"
#include "tensorflow/cc/ops/standard_ops.h"

int main()
{
  tensorflow::Scope root = tensorflow::Scope::NewRootScope();
  auto X = tensorflow::ops::Placeholder(root.WithOpName("x"), tensorflow::DT_FLOAT, 
                            tensorflow::ops::Placeholder::Shape({ -1, 2 }));
}

As explained in Ashley Tharp's guide, building right away results in linker errors complaining about unresolved external symbols:

  • LNK2001 unresolved external symbol "private: class tensorflow::Scope __cdecl tensorflow::Scope::WithOpNameImpl(...
  • LNK2001 unresolved external symbol "public: static class tensorflow::Scope __cdecl tensorflow::Scope::NewRootScope(void)"...
  • LNK2001 unresolved external symbol "public: __cdecl tensorflow::Scope::~Scope(void)"...
  • LNK2001 unresolved external symbol "public: __cdecl tensorflow::ops::Placeholder::Placeholder(...

Here is the explanation from her guide:

The reason this is happening is because you can only expose 60,000 symbols in a dll. This is just some limitation of the dll format. The tensorflow library code has more than 60000 symbols, so as the programmer building this dll (a dll is just a binary file for accessing a library at runtime) you will have to manually indicate which symbols you want exposed if they are not already. Google has chosen some default set, but it may not work for everyone.

And, as suggested in the guide, I went into tensorflow headers and prepended relevant symbols with TF_EXPORT macro, which is your usual DLL import-export pattern:

#ifdef TF_COMPILE_LIBRARY
#define TF_EXPORT __declspec(dllexport)
#else
#define TF_EXPORT __declspec(dllimport)
#endif  // TF_COMPILE_LIBRARY

The above-mentioned workaround works for the first 3 errors:

// scope.h
~Scope(); // change to: TF_EXPORT ~Scope();
...
static Scope NewRootScope(); // change to: TF_EXPORT static Scope NewRootScope();
...
Scope WithOpNameImpl(const string& op_name) const; // change to TF_EXPORT Scope...

After I am done editing, I re-run bazel like so bazel build --config=opt //tensorflow:tensorflow.lib, and errors are gone.

However, the issue arises when I try to perform similar manipulation to fix the last remaining unresolved symbol, namely Placeholder. The symbol is located inarray_ops.h, which is a machine-generated file. So whatever I edit inside the file is overwritten and lost as soon as I try to build .lib.

Question: How can I expose symbols that appear in machine-generated files? Pointers in the right direction would be much appreciated, perhaps I'm missing something obvious.

1 Answers

The folder containing dynamic library tensorflow_cc.dll and import library tensorflow_cc.dll.if.lib has also two files:

  • tensorflow_filtered_def_file.def : contains import symbols
  • tensorflow_cc.dll-2.params : has all built libraries

now if you have some unresolved symbols when building your app, all you have to do is to rebuild dynamic-library with updated tensorflow_filtered_def_file.def file. To this file you have to add missing symbols, for your sample code, it is:

EXPORTS
    ??1Scope@tensorflow@@QEAA@XZ
    ?NewRootScope@Scope@tensorflow@@SA?AV12@XZ
    ??0Placeholder@ops@tensorflow@@QEAA@AEBVScope@2@W4DataType@2@AEBUAttrs@012@@Z
    ?WithOpNameImpl@Scope@tensorflow@@AEBA?AV12@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z
    the rest symbols ...

After you updated .def file, you rebuild dll by calling the following command:

link.exe /nologo /DLL /SUBSYSTEM:CONSOLE -defaultlib:advapi32.lib -DEFAULTLIB:advapi32.lib 
   -ignore:4221 /FORCE:MULTIPLE /MACHINE:X64 
    @bazel-out/x64_windows-opt/bin/tensorflow/tensorflow_cc.dll-2.params /OPT:ICF /OPT:REF 
    /DEF:bazel-out/x64_windows-opt/bin/tensorflow/tensorflow_filtered_def_file.def /ignore:4070

before rebuilding dll/lib remove these files (they are only read-only).

More info here , see rafix07 comment.

Related