Hide template implementation

so I have created a template class which is not trivial and everything works as expected. I want to use this class as a kind of library and hide the implementation details. The code looks something like this:

 //Foo.hpp     
 #include "Header1.hpp"
 #include "Header2.hpp"

 #include "IFoo.hpp"
 ...

 template<typename T>
 class Foo : public IFoo{
   virtual ~Foo() = default;
   void method1(const T &t) override{
     //Do something
   }
  //...
 };

Now I wanted to have some kind of factory to hide the concrete implementation:

//Factory_foo.hpp
#include "Foo.hpp"

template<typename T>
std::unique_ptr<IFoo> create() {
  return std::make_unique<Foo<T>>();
}

So even, if the application code would include "Factory_foo.hpp" I cannot prevent the application code to include Header files which are necessary for the concrete implementation. Normally, I would put the method "create()" into the source file but this is not possible because the template types can be anything.

Any ideas how I can hide implementation details in this case?