Background
We have been working very hard to try come up with solutions for a "High Performance" application. The application is basically a high throughput in-memory manager, with a sync back to disk. The "reads" and "writes" are tremendously high, around 3000 transactions a second. We try and do as much as possible in memory, but eventually the data gets stale and needs to be flushed to disk, and this is where a huge "bottleneck" ensues. The app is multi-threaded, with about 50 threads. There is no IPC (inter-process comms)
Attempts
We initially wrote this in Java, and it worked quite well, up until a certain load, the bottleneck was hit and it just couldn't keep up. Then we tried it in C#, and the same bottle-neck was reached. We tried this with unmanaged code (C#), and though on initial tests was blindingly fast using MMF (Memory-map files), in production, reading was slow (are using Views). We did try CouchBase, but we stumbled into problems surround high network utilization. This might be poor configuration on our part!
Extra Info: In our Java attempt (non-MMF), our thread with the Queue of information that needs to get flushed to disk builds to the extent of being unable to keep up "writing" to disk. In our C# Memory-Map File Approach, the problems is that READS are very slow, and the WRITES working perfectly. For some reason, the Views are slow!
Question
So the question is, situations where you intend of transferring massive amounts of data; can someone please assist with a possible approach or architectural design that might be able to assist? I know this seems a bit broad, but I think the specific nature of high performance, high throughput should narrow down the answers.
Can anyone vouch for using Couchbase, MongoDB or Cassandra at such a level? Other ideas or solutions would be appreciated.