Atomically change two numbers without using locks

Sorry if this is not directly addressing your primary concern (non-locking atomicity), but I see that you are manipulating memory-mapped files by using the MemoryMappedFile and MemoryMappedViewAccessor classes.

I really don't know if current iterations of the .NET Framework have addressed this, but in a codebase we wrote around three years ago, we found out that memory-mapped file manipulation using those classes offered really poor performance (around 7x slower if I recall correctly), compared with using the Win32 API and direct pointer manipulation of the mapped memory, even inside a managed C++/CLI class.

I strongly suggest you give this method a test, you may be surprised of the performance gains (as we certainly did), and maybe the performance gain is so significant that it allows you to afford the cost of standard locking to achieve the atomicity you desire.

If you like to explore this avenue, here is a code snippet that shows the basics of the technique.

Int32 StationHashStorage::Open() {
   msclr::lock lock(_syncRoot);
   if( _isOpen )
      return 0;
   String^ fileName = GetFullFileName();

   _szInBytes = ComputeFileSizeInBytes(fileName);
   String^ mapExtension = GetFileExtension();
   String^ mapName = String::Format("{0}{1}_{2}", _stationId, _date.ToString("yyyyMMdd"), mapExtension);

   marshal_context context;
   LPCTSTR pMapName = context.marshal_as<const TCHAR*>(mapName);

   {
      msclr::lock lock( _openLock );
         // Try to see if another storage instance has requested the same memory-mapped file and share it
         _hMapping = OpenFileMapping(FILE_MAP_READ | FILE_MAP_WRITE, FALSE, pMapName);
         if( !_hMapping ) {
            // This is the first instance acquiring the file
            LPCTSTR pFileName = context.marshal_as<const TCHAR*>(fileName);
            // Try to open the existing file, or create new one if not exists
            _hFile = CreateFile(pFileName, 
                                GENERIC_READ | GENERIC_WRITE, 
                                FILE_SHARE_READ,
                                NULL,
                                OPEN_ALWAYS,
                                FILE_ATTRIBUTE_NORMAL,
                                NULL);
            if( !_hFile )
               throw gcnew IOException(String::Format(Strings::CreateFileFailed, GetLastError(), _stationId));
            _hMapping = CreateFileMapping(_hFile, 
                                          NULL,
                                          PAGE_READWRITE | SEC_COMMIT,
                                          0,
                                          _szInBytes,
                                          pMapName);
            if( !_hMapping ) 
               throw gcnew IOException(String::Format(Strings::CreateMappingFailed, GetLastError(), _stationId));
            _usingSharedFile = false;
         } else {
            _usingSharedFile = true;
         }
      }

// _pData gives you access to the entire requested memory range, you can directly
// dereference it,  memcopy it, etc.

   _pData = (UInt32*)::MapViewOfFile(_hMapping, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, 0);

   if( !_pData ) 
      throw gcnew IOException(String::Format(Strings::MapViewOfFileFailed, ::GetLastError(), _stationId));

   // warm-up the view by touching every page
   Int32 dummy = 0;
   for( int i = 0; i < _szInBytes / sizeof(Int32); i+= 1024 ) {
      dummy ^=  _pData[i];
   }
   // return the dummy value to prevent the optimizer from removing the apparently useless loop
   _isOpen = true;
   return dummy;
}

void StationHashStorage::Cleanup() {
     if( !_disposed ) {
      // dispose unmanaged resources here
      if( _pData ) {
         if( !UnmapViewOfFile(_pData) ) 
            LOG_ERROR(Strings::UnmapViewOfFileFailed, ::GetLastError(), _stationId);
         _pData = NULL;
      }

      if( _hMapping ) {
         if( !CloseHandle(_hMapping) ) 
            LOG_ERROR(Strings::CloseMappingFailed, ::GetLastError(), _stationId);
         _hMapping = NULL;
      }


      if( _hFile ) {
         if( !CloseHandle(_hFile) ) 
            LOG_ERROR(Strings::CloseFileFailed, ::GetLastError(), _stationId);
         _hFile = NULL;
      }
      _disposed = true;
   }
}

Now, regarding your real question. Is it posible that you embed the generated ID as part of the data stream? My idea would be something like this:

1. Pre-write the whole contents of your memory with a dummy known value (maybe 0xffffffff).

2. Use your current capacity check atomic logic.

3. After writing a message payload, you immediately write the computed message Id (your capacity check would need to account for this extra data)

4. Instead of using Interlocked.Add to get the next Id, you would enter a loop that checks the memory before the current message (the previous message Id) until it is different to your dummy known value. Once you exit the loop, the current message Id would be the read value + 1.

This would require some special manipulation of the first inserted message (as it needs to seed the first Id marker in your stream. You would also need to be careful (if you are using long Ids and you are in 32-bit mode) that your Id stream reads and writes are atomic.

Good luck, and I really encourage you to give the Win32 API a try, it would be very interesting to find out if things, hopefully, have improved! Feel free to contact me if you need assistance with the C++/CLI code.