Does passing fundamental values by const reference really hurt performance?

There is at least one circumstance where passing by const reference might disable optimizations. However, the most popular compilers provide a way to re-enable them.

Let’s look at this function:

int cryptographicHash( int& salt, const int& plaintext )
{
  salt = 4; // Chosen by fair dice roll
            // guaranteed to be random
  return plaintext; // If we tell them there's a salt,
                    // this is the last hash function they'll
                    // ever suspect!
}

Looks pretty secure, right? But, since we’re writing in C++, is it as fast as it could possibly be? (Definitely what we want in a cryptographic hash.)

No, because what if you call it with:

int x = 0xFEED;
const int y = cryptographicHash( x, x );

Now the parameters passed by reference alias the same object, so the function should, as written, return 4, not 0xFEED. This means that, disastrously, the compiler can no longer optimize away the & in its const int& parameter.

However, the most popular compilers (including GCC, clang, Intel C++ and Visual C++ 2015 and up) all support the __restrict extension. So, change the function signature to int cryptographicHash( int& salt, const int& __restrict plaintext ) and all problems with it are solved forever.

Since this extension is not part of the C++ standard, you can improve portability with something like the following:

#if ( __GNUC__ || __clang__ || __INTEL_COMPILER || __ICL || _MSC_VER >= 1900 )
#  define RESTRICT __restrict
#else
#  define RESTRICT /**/
#endif

int cryptographicHash( int& salt, const int& RESTRICT plaintext );

(In GCC and clang, this does not appear to change the generated code.)

On platforms where the fundamental type in question fits into a register, a decent compiler should eliminate const references from parameters if it can see both sides of the call. For templates that is usually a given (unless they were explicitly instantiated somewhere). Since your library presumably has to be templated all the way down, this will apply to your case.

It's possible that your end users will have bad compilers or platforms where e.g. a double does not fit into a register. I don't see why you'd be incentivized to make micro-optimizations for these particular users, but maybe you do.

It's also possible that you want to explicitly instantiate all templates in your library for some set of types and provide implementation-less header files. In that case the user's compiler must obey whatever calling conventions exist on that platform and will probably pass fundamental types by reference.

Ultimately, the answer is "profile the relevant and representative use cases" if you don't have faith in the compiler(s).

Edit (removed macro solution): As suggested by Jarod42, the C++ way would be using an alias template. This also avoids the lack of deduction that the asker was running into with their original approach:

template<class T>
using CONSTREF = const T&; // Or just T for benchmarking.

https://godbolt.org/z/mopZ6B

As cppreference says:

Alias templates are never deduced by template argument deduction when deducing a template template parameter.

Passing something like int by reference (basically a pointer) is clearly sub-optimal since the extra indirection through the pointer can incur a cache miss and it may also prevent compiler optimizations since the compiler can't always know that the pointed-to variable cannot be changed by other entities, so it may in some cases be forced to do additional loads from memory. Passing by value removes the indirection and lets the compiler assume that noone else is changing the value.

If an argument is trivially constructible and isn't modified, pass by value. The calling convention will automatically pass large structs by reference.

struct alignas(4096) page {unsigned char bytes[4096];};

[[nodiscard]] constexpr page operator^(page l, page r) noexcept {
    for (int i = 0; i < 4096; ++i)
        l.bytes[i] = l.bytes[i] ^ r.bytes[i];
    return l;
}

Arguments that are modified and/or returned by nonconst reference must be passed by nonconst reference.

constexpr page& operator^=(page& l, page r) noexcept {return l = l ^ r;}

Pass any argument returned with const reference semantics by const reference.

using buffer = std::vector<unsigned char>;

[[nodiscard]] std::string_view to_string_view(const buffer& b) noexcept {
    return {reinterpret_cast<const char*>(b.data()), b.size()};
}

Pass any argument deep copied to a different type by const reference.

[[nodiscard]] std::string to_string(const buffer& b) {
    return std::string{to_string_view(b)};
}

Pass any non-trivially constructible, unmodified and non-deep copied argument by const reference.

std::ostream& operator<<(std::ostream& os, const buffer& b) {
    os << std::hex;
    for (const unsigned short u8 : b)
        os << u8 << ',';
    return os << std::dec;
}

Pass any argument deep copied to a value of the same type by value. There's no point in passing by reference an argument that's copied anyway, and the constructor for the returned copy is optimized away. See https://en.cppreference.com/w/cpp/language/copy_elision

[[nodiscard]] buffer operator^(buffer l, const buffer& r) {
    const auto lsize = l.size();
    const auto rsize = r.size();
    const auto minsize = std::min(lsize, rsize);

    for (buffer::size_type i = 0; i < minsize; ++i)
        l[i] = l[i] ^ r[i];
    if (lsize < rsize)
        l.insert(l.end(), r.begin() + minsize, r.end());

    return l;
}

This includes template functions as well.

template<typename T>
[[nodiscard]]
constexpr T clone(T t) noexcept(std::is_nothrow_constructible_v<T, T>) {
    return t;
}

Otherwise take arguments of template parameter type by forwarding reference (&&). Note: && only has forwarding (universal) reference semantics in arguments of template parameter type, and/or for auto&& or decltype(auto)&&.

template<typename T>
constexpr bool nt = noexcept(std::is_nothrow_constructible_v<int, T&&>);

template<typename T>
[[nodiscard]]
constexpr int to_int(T&& t) noexcept(nt<T>) {return static_cast<int>(t);}

const auto to_int_lambda = [](auto&& t) noexcept(noexcept(to_int(t))) {
    return to_int(t);
};

This is a complex question that depends on architecture, compiler optimizations and many other specifics as the variation in answers shows. Since the OP is about writing template functions there is also the option of controlling which function is called by using SFINAE.

#include <iostream>

template <typename T, typename = typename std::enable_if_t<std::is_fundamental_v<T>> >
void f(T t) {
    std::cout << "Pass by value\n";
}

template <typename T, typename = typename std::enable_if_t<not std::is_fundamental_v<T>> >
void f(T const &t) {
    std::cout << "Pass by const ref.\n";
}

class myclass {};

int main() {
    float x;
    int i;
    myclass c;

    std::cout << "float: ";
    f(x);

    std::cout << "int: ";
    f(i);

    std::cout << "myclass: ";
    f(c);

    return 0;
}

Output:

float: Pass by value
int: Pass by value
myclass: Pass by const ref.