Sharing data between AppDomains

The only way to avoid serialisation is to represent your data using objects which derive from MarshalByRefObject, but in that case you will still have the cost of marshalling across the AppDomain boundaries. This may also involve the refactoring/re-writing of much of your code.

Assuming marshalling by reference is not an option, you will have to serialise at some point. It simply cannot be avoided. One way to do this is as Neil Barnwell suggests, with a database, another would be with a local file as you suggest yourself.

Another way which may or may not feasible depending on your delivery timeline and/or .NET 4.0 adoption, would be to use a memory mapped file, see .Net Framework 4.0: Using memory mapped files.

I tend to say just use remoting. Writing the data to a file requires serialization, too. Serialization seems to be almost unavoidable what ever technology you use. You have to transfer data from one application domain to another using some channel and you will have to serialize the data in order to get it through the channel.

The only way to avoid serialization seems to be using shared memory so that both application domains can access the data without ever going through a channel. Even deep cloning the data from one application domain's memory into the other's memory is at its core nothing more then a binary serialization (where the result is not necessarily stored in consecutive memory locations).

I do appreciate you want to keep this in-memory, but my first suggestion would be to write the data to a database and query from there. Remoting is still a remote call, which is where much of the "cost" of using a database server comes from, and you'd have to build in transaction-handling to make sure you don't lose data. If you write to a SQL Server database you have transaction support ready and waiting for you, and it's fast-fast-fast for queries.