Why can templates only be implemented in the header file?

Viewed 687179

Quote from The C++ standard library: a tutorial and handbook:

The only portable way of using templates at the moment is to implement them in header files by using inline functions.

Why is this?

(Clarification: header files are not the only portable solution. But they are the most convenient portable solution.)

17 Answers

Caveat: It is not necessary to put the implementation in the header file, see the alternative solution at the end of this answer.

Anyway, the reason your code is failing is that, when instantiating a template, the compiler creates a new class with the given template argument. For example:

template<typename T>
struct Foo
{
    T bar;
    void doSomething(T param) {/* do stuff using T */}
};

// somewhere in a .cpp
Foo<int> f; 

When reading this line, the compiler will create a new class (let's call it FooInt), which is equivalent to the following:

struct FooInt
{
    int bar;
    void doSomething(int param) {/* do stuff using int */}
}

Consequently, the compiler needs to have access to the implementation of the methods, to instantiate them with the template argument (in this case int). If these implementations were not in the header, they wouldn't be accessible, and therefore the compiler wouldn't be able to instantiate the template.

A common solution to this is to write the template declaration in a header file, then implement the class in an implementation file (for example .tpp), and include this implementation file at the end of the header.

Foo.h

template <typename T>
struct Foo
{
    void doSomething(T param);
};

#include "Foo.tpp"

Foo.tpp

template <typename T>
void Foo<T>::doSomething(T param)
{
    //implementation
}

This way, implementation is still separated from declaration, but is accessible to the compiler.

Alternative solution

Another solution is to keep the implementation separated, and explicitly instantiate all the template instances you'll need:

Foo.h

// no implementation
template <typename T> struct Foo { ... };

Foo.cpp

// implementation of Foo's methods

// explicit instantiations
template class Foo<int>;
template class Foo<float>;
// You will only be able to use Foo with int or float

If my explanation isn't clear enough, you can have a look at the C++ Super-FAQ on this subject.

Templates need to be instantiated by the compiler before actually compiling them into object code. This instantiation can only be achieved if the template arguments are known. Now imagine a scenario where a template function is declared in a.h, defined in a.cpp and used in b.cpp. When a.cpp is compiled, it is not necessarily known that the upcoming compilation b.cpp will require an instance of the template, let alone which specific instance would that be. For more header and source files, the situation can quickly get more complicated.

One can argue that compilers can be made smarter to "look ahead" for all uses of the template, but I'm sure that it wouldn't be difficult to create recursive or otherwise complicated scenarios. AFAIK, compilers don't do such look aheads. As Anton pointed out, some compilers support explicit export declarations of template instantiations, but not all compilers support it (yet?).

Actually, prior to C++11 the standard defined the export keyword that would make it possible to declare templates in a header file and implement them elsewhere. In a manner of speaking. Not really, as the only ones who ever implemented that feature pointed out:

Phantom advantage #1: Hiding source code. Many users, have said that they expect that by using export they will no longer have to ship definitions for member/nonmember function templates and member functions of class templates. This is not true. With export, library writers still have to ship full template source code or its direct equivalent (e.g., a system-specific parse tree) because the full information is required for instantiation. [...]

Phantom advantage #2: Fast builds, reduced dependencies. Many users expect that export will allow true separate compilation of templates to object code which they expect would allow faster builds. It doesn’t because the compilation of exported templates is indeed separate but not to object code. Instead, export almost always makes builds slower, because at least the same amount of compilation work must still be done at prelink time. Export does not even reduce dependencies between template definitions because the dependencies are intrinsic, independent of file organization.

None of the popular compilers implemented this keyword. The only implementation of the feature was in the frontend written by the Edison Design Group, which is used by the Comeau C++ compiler. All others required you to write templates in header files, because the compiler needs the template definition for proper instantiation (as others pointed out already).

As a result, the ISO C++ standard committee decided to remove the export feature of templates with C++11.

Although standard C++ has no such requirement, some compilers require that all function and class templates need to be made available in every translation unit they are used. In effect, for those compilers, the bodies of template functions must be made available in a header file. To repeat: that means those compilers won't allow them to be defined in non-header files such as .cpp files

There is an export keyword which is supposed to mitigate this problem, but it's nowhere close to being portable.

It means that the most portable way to define method implementations of template classes is to define them inside the template class definition.

template < typename ... >
class MyClass
{

    int myMethod()
    {
       // Not just declaration. Add method implementation here
    }
};

Just to add something noteworthy here. One can define methods of a templated class just fine in the implementation file when they are not function templates.


myQueue.hpp:

template <class T> 
class QueueA {
    int size;
    ...
public:
    template <class T> T dequeue() {
       // implementation here
    }

    bool isEmpty();

    ...
}    

myQueue.cpp:

// implementation of regular methods goes like this:
template <class T> bool QueueA<T>::isEmpty() {
    return this->size == 0;
}


main()
{
    QueueA<char> Q;

    ...
}

I suggest looking at this gcc page which discusses the tradeoffs between the "cfront" and "borland" model for template instantiations.

https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Template-Instantiation.html

The "borland" model corresponds to what the author suggests, providing the full template definition, and having things compiled multiple times.

It contains explicit recommendations concerning using manual and automatic template instantiation. For example, the "-repo" option can be used to collect templates which need to be instantiated. Or another option is to disable automatic template instantiations using "-fno-implicit-templates" to force manual template instantiation.

In my experience, I rely on the C++ Standard Library and Boost templates being instantiated for each compilation unit (using a template library). For my large template classes, I do manual template instantiation, once, for the types I need.

This is my approach because I am providing a working program, not a template library for use in other programs. The author of the book, Josuttis, works a lot on template libraries.

If I was really worried about speed, I suppose I would explore using Precompiled Headers https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html

which is gaining support in many compilers. However, I think precompiled headers would be difficult with template header files.

Another reason that it's a good idea to write both declarations and definitions in header files is for readability. Suppose there's such a template function in Utility.h:

template <class T>
T min(T const& one, T const& theOther);

And in the Utility.cpp:

#include "Utility.h"
template <class T>
T min(T const& one, T const& other)
{
    return one < other ? one : other;
}

This requires every T class here to implement the less than operator (<). It will throw a compiler error when you compare two class instances that haven't implemented the "<".

Therefore if you separate the template declaration and definition, you won't be able to only read the header file to see the ins and outs of this template in order to use this API on your own classes, though the compiler will tell you in this case about which operator needs to be overridden.

You can actually define your template class inside a .template file rather than a .cpp file. Whoever is saying you can only define it inside a header file is wrong. This is something that works all the way back to c++ 98.

Don't forget to have your compiler treat your .template file as a c++ file to keep the intelli sense.

Here is an example of this for a dynamic array class.

#ifndef dynarray_h
#define dynarray_h

#include <iostream>

template <class T>
class DynArray{
    int capacity_;
    int size_;
    T* data;
public:
    explicit DynArray(int size = 0, int capacity=2);
    DynArray(const DynArray& d1);
    ~DynArray();
    T& operator[]( const int index);
    void operator=(const DynArray<T>& d1);
    int size();

    int capacity();
    void clear();

    void push_back(int n);

    void pop_back();
    T& at(const int n);
    T& back();
    T& front();
};

#include "dynarray.template" // this is how you get the header file

#endif

Now inside you .template file you define your functions just how you normally would.

template <class T>
DynArray<T>::DynArray(int size, int capacity){
    if (capacity >= size){
        this->size_ = size;
        this->capacity_ = capacity;
        data = new T[capacity];
    }
    //    for (int i = 0; i < size; ++i) {
    //        data[i] = 0;
    //    }
}

template <class T>
DynArray<T>::DynArray(const DynArray& d1){
    //clear();
    //delete [] data;
    std::cout << "copy" << std::endl;
    this->size_ = d1.size_;
    this->capacity_ = d1.capacity_;
    data = new T[capacity()];
    for(int i = 0; i < size(); ++i){
        data[i] = d1.data[i];
    }
}

template <class T>
DynArray<T>::~DynArray(){
    delete [] data;
}

template <class T>
T& DynArray<T>::operator[]( const int index){
    return at(index);
}

template <class T>
void DynArray<T>::operator=(const DynArray<T>& d1){
    if (this->size() > 0) {
        clear();
    }
    std::cout << "assign" << std::endl;
    this->size_ = d1.size_;
    this->capacity_ = d1.capacity_;
    data = new T[capacity()];
    for(int i = 0; i < size(); ++i){
        data[i] = d1.data[i];
    }

    //delete [] d1.data;
}

template <class T>
int DynArray<T>::size(){
    return size_;
}

template <class T>
int DynArray<T>::capacity(){
    return capacity_;
}

template <class T>
void DynArray<T>::clear(){
    for( int i = 0; i < size(); ++i){
        data[i] = 0;
    }
    size_ = 0;
    capacity_ = 2;
}

template <class T>
void DynArray<T>::push_back(int n){
    if (size() >= capacity()) {
        std::cout << "grow" << std::endl;
        //redo the array
        T* copy = new T[capacity_ + 40];
        for (int i = 0; i < size(); ++i) {
            copy[i] = data[i];
        }

        delete [] data;
        data = new T[ capacity_ * 2];
        for (int i = 0; i < capacity() * 2; ++i) {
            data[i] = copy[i];
        }
        delete [] copy;
        capacity_ *= 2;
    }
    data[size()] = n;
    ++size_;
}

template <class T>
void DynArray<T>::pop_back(){
    data[size()-1] = 0;
    --size_;
}

template <class T>
T& DynArray<T>::at(const int n){
    if (n >= size()) {
        throw std::runtime_error("invalid index");
    }
    return data[n];
}

template <class T>
T& DynArray<T>::back(){
    if (size() == 0) {
        throw std::runtime_error("vector is empty");
    }
    return data[size()-1];
}

template <class T>
T& DynArray<T>::front(){
    if (size() == 0) {
        throw std::runtime_error("vector is empty");
    }
    return data[0];
    }
Related