Can anyone define the Windows PE Checksum Algorithm?

Viewed 8419

I would like to implement this in C#

I have looked here: http://www.codeproject.com/KB/cpp/PEChecksum.aspx

And am aware of the ImageHlp.dll MapFileAndCheckSum function.

However, for various reasons, I would like to implement this myself.

The best I have found is here: http://forum.sysinternals.com/optional-header-checksum-calculation_topic24214.html

But, I don't understand the explanation. Can anyone clarify how the checksum is calculated?

Thanks!

Update

I from the code example, I do not understand what this means, and how to translate it into C#

sum -= sum < low 16 bits of CheckSum in file // 16-bit borrow 
sum -= low 16 bits of CheckSum in file 
sum -= sum < high 16 bits of CheckSum in file 
sum -= high 16 bits of CheckSum in file 

Update #2

Thanks, came across some Python code that does similar too here

    def generate_checksum(self):

    # This will make sure that the data representing the PE image
    # is updated with any changes that might have been made by
    # assigning values to header fields as those are not automatically
    # updated upon assignment.
    #
    self.__data__ = self.write()

    # Get the offset to the CheckSum field in the OptionalHeader
    #
    checksum_offset = self.OPTIONAL_HEADER.__file_offset__ + 0x40 # 64

    checksum = 0

    # Verify the data is dword-aligned. Add padding if needed
    #
    remainder = len(self.__data__) % 4
    data = self.__data__ + ( '\0' * ((4-remainder) * ( remainder != 0 )) )

    for i in range( len( data ) / 4 ):

        # Skip the checksum field
        #
        if i == checksum_offset / 4:
            continue

        dword = struct.unpack('I', data[ i*4 : i*4+4 ])[0]
        checksum = (checksum & 0xffffffff) + dword + (checksum>>32)
        if checksum > 2**32:
            checksum = (checksum & 0xffffffff) + (checksum >> 32)

    checksum = (checksum & 0xffff) + (checksum >> 16)
    checksum = (checksum) + (checksum >> 16)
    checksum = checksum & 0xffff

    # The length is the one of the original data, not the padded one
    #
    return checksum + len(self.__data__)

However, it's still not working for me - here is my conversion of this code:

using System;
using System.IO;

namespace CheckSumTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var data = File.ReadAllBytes(@"c:\Windows\notepad.exe");

            var PEStart = BitConverter.ToInt32(data, 0x3c);
            var PECoffStart = PEStart + 4;
            var PEOptionalStart = PECoffStart + 20;
            var PECheckSum = PEOptionalStart + 64;
            var checkSumInFile = BitConverter.ToInt32(data, PECheckSum);
            Console.WriteLine(string.Format("{0:x}", checkSumInFile));

            long checksum = 0;

            var remainder = data.Length % 4;
            if (remainder > 0)
            {
                Array.Resize(ref data, data.Length + (4 - remainder));
            }

            var top = Math.Pow(2, 32);

            for (int i = 0; i < data.Length / 4; i++)
            {
                if (i == PECheckSum / 4)
                {
                    continue;
                }
                var dword = BitConverter.ToInt32(data, i * 4);
                checksum = (checksum & 0xffffffff) + dword + (checksum >> 32);
                if (checksum > top)
                {
                    checksum = (checksum & 0xffffffff) + (checksum >> 32);
                }
            }

            checksum = (checksum & 0xffff) + (checksum >> 16);
            checksum = (checksum) + (checksum >> 16);
            checksum = checksum & 0xffff;

            checksum += (uint)data.Length; 
            Console.WriteLine(string.Format("{0:x}", checksum));

            Console.ReadKey();
        }
    }
}

Can anyone tell me where I'm being stupid?

8 Answers

No one really answered the original question of "Can anyone define the Windows PE Checksum Algorithm?" so I'm going to define it as simply as possible. A lot of the examples given so far are optimizing for unsigned 32-bit integers (aka DWORDs), but if you just want to understand the algorithm itself at its most fundamental, it is simply this:

  1. Using an unsigned 16-bit integer (aka a WORD) to store the checksum, add up all of the WORDs of the data except for the 4 bytes of the PE optional header checksum. If the file is not WORD-aligned, then the last byte is a 0x00.

  2. Convert the checksum from a WORD to a DWORD and add the size of the file.

The PE checksum algorithm above is effectively the same as the original MS-DOS checksum algorithm. The only differences are the location to skip and replacing the XOR 0xFFFF at the end and adding the size of the file instead.

From my WinPEFile class for PHP, the above algorithm looks like:

    $x = 0;
    $y = strlen($data);
    $val = 0;
    while ($x < $y)
    {
        // Skip the checksum field location.
        if ($x === $this->pe_opt_header["checksum_pos"])  $x += 4;
        else
        {
            $val += self::GetUInt16($data, $x, $y);

            // In PHP, integers are either signed 32-bit or 64-bit integers.
            if ($val > 0xFFFF)  $val = ($val & 0xFFFF) + 1;
        }
    }

    // Add the file size.
    $val += $y;

The CheckSum field is 32 bits long and is calculated as follows

1. Add all dwords (32 bit pieces) of the entire file to a sum

Add all dwords of the entire file not including the CheckSum field itself, including all headers and all of the contents, to a dword. If the dword overflows, add the overflowed bit back to the first bit (2^0) of the dword. If the file is not entirely divisible into dwords (4 bit pieces) see 2.

The best way I know to realize this is by using the GNU C Compilers Integer Overflow Builtin function __builtin_uadd_overflow. In the original ChkSum function documented by Jeffrey Walton the sum was calculated by performing an add (%esi),%eax where
esi contains the base address of the file and eax is 0 and adding the rest of the file like this

adc 0x4(%esi),%eax
adc 0x8(%esi),%eax
adc 0xc(%esi),%eax
adc 0x10(%esi),%eax
...
adc $0x0,%eax

The first add adds the first dword ignoring any carry flag. The next dwords
are added by the adc instruction which does the same thing as add but
adds any carry flag that was set before executing the instruction in addition
to the summand. The last adc $0x0,%eax adds only the last carry flag if it
was set and cannot be discarded.

Please keep in mind that the dword of CheckSum field itself should not be added.

2. Add the remainder to the sum if there is one

If the file is not entirely divisible into dwords, add the remainder as a
zero-padded dword. For example: say your file is 15 bytes long and looks like this
0E 1F BA 0E | 00 B4 09 CD | 21 B8 01 4C | CD 21 54
You need to add the remainder as 0x005421CD to the sum. My system is a
little-endian system. I do not know if the checksum would change because of the
this order of the bytes on big-endian systems, or you would just simulate this
behaviour.
I do this by rounding up the buffer_size to the next bytecount divisible by 4
without remainder or put differently: the next whole dword count represented
in bytes. Then I allocate with calloc because it initializes the memory block
with all zeros.

if(buffer_size%4)
  {buffer_size+=4-(buffer_size%4);
...
calloc(buffer_size,1)

3. Add the lower word (16 bit piece) and the higher word of the sum together.

sum=(sum&0xffff)+(sum>>16);

4. Add the new higher word once again

sum+=(sum>>16);

5. Only keep the lower word

sum&=0xffff;

6. Add the number of bytes in the file to the sum

return(sum+size);

This is how I wrote it. It is not C#, but C. off_t size is the number of bytes in the file. uint32_t *base is a pointer to the file loaded into memory. The block of memory should be padded with zeros at the end to the next bytecount divisible by 4.

uint32_t pe_header_checksum(uint32_t *base,off_t size)
  {uint32_t sum=0;
  off_t i;
  for(i=0;i<(size/4);i++)
    {if(i==0x36)
      {continue;}
    sum+=__builtin_uadd_overflow(base[i],sum,&sum);}
  if(size%4)
    {sum+=base[i];}
  sum=(sum&0xffff)+(sum>>16);
  sum+=(sum>>16);
  sum&=0xffff;
  return(sum+size);}

If you want you can see the code in action and read a little bit more here.

Related