I've recently come across Chapel and I'm very keen to try it out. I have a two-fold problem I'm hoping it can solve.
I typically work in Python or C++. Java when backed into a corner.
I have two matrices I and V. Both are sparse and of dimension about 600K x 600K, populated at about 1% density.
First, using SciPy, I can load both from a SQL database into memory at the moment. However, I expect our next iteration will be simply too large for our machines. Perhaps 1.5M^2. In a case like that, RDDs from Spark may work for the load. I wasn't able to get PyTables to make this happen. I understand this is described as an "Out-of-core" problem.
Even if they do get loaded, doing I'IV goes OOM in minutes. (Here I' is transpose), so I'm looking into distributing this multiplication over multiple cores (which SciPy can do) and multiple machines (which it cannot, so far as I know). Here, Spark falls down but Chapel appears to answer my prayers, so-to-speak.
A serious limitation is budget on machines. I can't afford a Cray, for instance. Does the Chapel community have a pattern for this?