Unexpected senseless optimization tries of a simple for loop (in Visual C++ 2017)

I was playing around with the Visual C++ 2017 compiler doing some tests on how to implement various things to get maximum performance from the code when I encountered a behaviour I did not expect nor can I explain it.

I created a simple foreach method to process all values in my container. The container itself only stores a size_t size and an int *ptr.

This is the code of the foreach method taking a Lambda in f:

template<class F>
__declspec(noinline) void foreach(F f)
{
    for (size_t i = 0; i < size; ++i)
        f(ptr[i]);
}

I call it with

int sum = 0;
v.foreach([&](int item) { sum += item; });

When it is compiled a part of the binary looks like this:

        for (size_t i = 0; i < size; ++i)
00007FF6B3382310  xor         eax,eax  
00007FF6B3382312  mov         qword ptr [f],rdx  
00007FF6B3382317  cmp         qword ptr [rcx],rax  
00007FF6B338231A  jbe         MyVector::foreach<<lambda_c1957c9a484ac2f96c41b63c392e4950> >+2Ah (07FF6B338233Ah)  
00007FF6B338231C  nop         dword ptr [rax]  
            f(ptr[i]);
00007FF6B3382320  mov         r8,qword ptr [rcx+8]  
00007FF6B3382324  mov         r9d,dword ptr [r8+rax*4]  
        for (size_t i = 0; i < size; ++i)
00007FF6B3382328  inc         rax  
            f(ptr[i]);
00007FF6B338232B  add         dword ptr [rdx],r9d  
        for (size_t i = 0; i < size; ++i)
00007FF6B338232E  cmp         rax,qword ptr [rcx]  
00007FF6B3382331  jae         MyVector::foreach<<lambda_c1957c9a484ac2f96c41b63c392e4950> >+2Ah (07FF6B338233Ah)  
00007FF6B3382333  mov         rdx,qword ptr [f]  
00007FF6B3382338  jmp         MyVector::foreach<<lambda_c1957c9a484ac2f96c41b63c392e4950> >+10h (07FF6B3382320h)  
    }
00007FF6B338233A  ret  

Analysis:

• So rcx is the address of the size and also the address of this (as size is the first member of the object).
• rdx seens to be the address of the Lambda functor and it is also the address if the sum member of the Lambda.
• In address xxx320 the ptr is loaded into register r8.
• From xxx324 to xxx338 is the main loop. Value of ptr[i] is loaded into r9d, rax (the ptr) is incremented and r9d is added to the sum-member of the Lambda object.

The first 2 of my 3 questions are:

• Why is address xxx320 part of the loop? r8 is not changed and the ptr-member is not marked volatile. Shouldn't the jmp in xxx338 point to xxx324 instead?

• Why is the lambda address cached from rdx to [f] in xxx312 and restored to rdx in xxx333 in every loop? rdx is not changed, so why does the compiler reload it?

I tried to get rid of these 'inefficiencies' and found that the following source creates a more reasonable machine code:

template<class F>
__declspec(noinline) void foreach(F f)
{
    register auto f2 = f;
    register auto p = ptr;

    for (size_t i = 0; i < size; ++i)
        f2(p[i]);
}

The generated machine code is

    register auto f2 = f;
        register auto p = ptr;
00007FF7E79A2340  mov         r9,qword ptr [rcx+8]  

        for (size_t i = 0; i < size; ++i)
00007FF7E79A2344  xor         eax,eax  
00007FF7E79A2346  cmp         qword ptr [rcx],rax  
00007FF7E79A2349  jbe         MyCheckedVector::foreach<<lambda_27103c2606044b6f9a288cfb44283d2c> >+1Fh (07FF7E79A235Fh)  
00007FF7E79A234B  nop         dword ptr [rax+rax]  
            f2(p[i]);
00007FF7E79A2350  mov         r8d,dword ptr [r9+rax*4]  

        for (size_t i = 0; i < size; ++i)
00007FF7E79A2354  inc         rax  
            f2(p[i]);
00007FF7E79A2357  add         dword ptr [rdx],r8d  

        for (size_t i = 0; i < size; ++i)
00007FF7E79A235A  cmp         rax,qword ptr [rcx]  
00007FF7E79A235D  jb          MyCheckedVector::foreach<<lambda_27103c2606044b6f9a288cfb44283d2c> >+10h (07FF7E79A2350h)  
    }
00007FF7E79A235F  ret  

It fixes to problems of the original source:

• ptr or r9 is only loaded at the beginning of the method.
• rdx (the lambda's address) is not cached and simply stored in the rdxregister.
• the main loop only iterates from xxx350 tp xxx35D (Load ptr[i] into r8d, increment i in rax, add ptr[i] cached in r8d to Lambda member sum addressed with rdx, check if rax >= size in address at rcx

And my third question is:

How comes this piece if machine code actually runs at the same speed as the original version? It's only having 5 instructions per loop versus 8 instructions in the original loop and all 5 instructions are also present in the original loop.

Maybe I should also mention my optimization settings:

• Full Optimization
• Inline Function Expansion: Any Suitable
• Intrinsics: Yes
• Favor fast code
• Whole Program Optimization
• Enhanced Instruction Set: Advanced Vector Extensions 2 (AVX2)

I know I should leave optimizations to the compiler, but sometimes I dig into machine code out of curiosity. I would really appreciate any explanations!

--- UPDATE ---

I have created an example which is not 100% the original code but creates the same machine code. The example uses only 1K values (4000 Byte data) in the inner loop so memory access should not be limiting performance.

#include <iostream>
#include <ctime>
#include <conio.h>
#include <algorithm>
#include <vector>

static const auto OuterLoops = 100000;

class MyContainer
{
    static const auto Items = 100000000 / OuterLoops;

public:
    MyContainer()
        : size(Items),
        ptr(new int[Items])
    {
        // Make sure memory is paged
        std::fill(ptr, ptr + size, 0xA5);
    }

    template<class F> __declspec(noinline) auto foreach1(F f)
    {
        for (size_t i = 0; i < size; ++i)
            f(ptr[i]);
    }

    template<class F> __declspec(noinline) auto foreach2(F f)
    {
        const register auto sizer = size;
        const register auto fr = f;
        const register auto ptrr = ptr;

        for (size_t i = 0; i < sizer; ++i)
            fr(ptrr[i]);
    }

private:
    size_t size;
    int *ptr;
};

template<class F>
void measureSpeed(const char *const caption, F f)
{
    std::vector<int> results(11);

    for (auto& result : results)
    {
        for (auto a = clock(); a == clock(); );
        const auto start1 = clock();

        for(int i = 0; i < OuterLoops; ++i)
            f();

        result = clock() - start1;
    }

    std::sort(results.begin(), results.end());
    std::cout << caption << ": " << results[results.size() / 2] << " (";
    for (const auto& result : results)
        std::cout << result << ' ';
    std::cout << "\b)" << std::endl;

}

int main(int argc, char **argv)
{
    MyContainer c;

    int s1 = 0;
    measureSpeed("foreach1", [&]() { c.foreach1([&](const auto v) { s1 += v; }); });

    int s2 = 0;
    measureSpeed("foreach2", [&]() { c.foreach2([&](const auto v) { s2 += v; }); });

    if (s1 != s2)
        std::cerr << "Comparing nonsense" << std::endl;

    _getch();
    return 0;
}

The results I get are a bit variing, but foreach2 is always about 5% slower than foreach1:

foreach1: 195 (190 191 192 193 195 195 195 195 196 196 197)
foreach2: 207 (202 202 205 206 206 207 208 208 212 213 214)

(The first value is the median of all test runs followed by the sorted timings of all test runs. The values are from clock() and not converted, but at least with Visual Studio Platform Toolset v140 these are milliseconds.)