graph regression detection in neo4j

Question

I've had a graph with such dimensions

75 millions of nodes
238 millions of relationships

for several years.

I regenerate it periodically from external data (by importing csv).

Then I do batches of allshortestpaths searches, single search take several milliseconds. But whole batch of 100k searches usually took 5-10 minutes.

Some time ago batch searches became several times slower - always 30++ minutes.

Are there any tools to investigate the problem? I use cypher and java driver, there were no neo4j updates at that time (now 4.0.11)

I tried beefier server with Oracle linux instead of Debian, and neo4j 4.2.4 instead of 4.0.11, but that didn't help. So, the problem is probably inside the graph.

I've set logging for slow searches, but there are no searches slower than dozens of milliseconds. Unfortunatelly, I cannot revert to old (faster) graph to measure mean time.

UPDATE:

Old search

MATCH (one:Obj{oid:'id1'}) with one MATCH (two:Obj{oid:'anyOf100kId'}), path=allshortestPaths((one) -[*0..4]-(two)) WHERE ALL (x IN RELATIONSHIPS(path) WHERE x.val<130) return path limit 100;
+----------------------------------------------------------------------------------------------------------+
| Plan      | Statement   | Version      | Planner | Runtime       | Time | DbHits | Rows | Memory (Bytes) |
+----------------------------------------------------------------------------------------------------------+
| "PROFILE" | "READ_ONLY" | "CYPHER 4.2" | "COST"  | "INTERPRETED" | 44   | 82495  | 7    | 7707104        |
+----------------------------------------------------------------------------------------------------------+


+----------------------------+-------------------------------------------------------------------------------------------------+----------------+------+---------+----------------+------------------------+
| Operator                   | Details                                                                                         | Estimated Rows | Rows | DB Hits | Memory (Bytes) | Page Cache Hits/Misses |
+----------------------------+-------------------------------------------------------------------------------------------------+----------------+------+---------+----------------+------------------------+
| +ProduceResults@graph.db   | path                                                                                            |              1 |    7 |     224 |                |                    0/0 |
| |                          +-------------------------------------------------------------------------------------------------+----------------+------+---------+----------------+------------------------+
| +Limit@graph.db            | 100                                                                                             |              1 |    7 |       0 |                |                    0/0 |
| |                          +-------------------------------------------------------------------------------------------------+----------------+------+---------+----------------+------------------------+
| +ShortestPath@graph.db     | path = (one)-[anon_112*0..4]-(two) WHERE all(x IN RELATIONSHIPS(path) WHERE x.val < $autoint_2) |              1 |    7 |   82267 |        7707104 |                    0/0 |
| |                          +-------------------------------------------------------------------------------------------------+----------------+------+---------+----------------+------------------------+
| +CartesianProduct@graph.db |                                                                                                 |              1 |    1 |       0 |                |                    0/0 |
| |\                         +-------------------------------------------------------------------------------------------------+----------------+------+---------+----------------+------------------------+
| | +NodeIndexSeek@graph.db  | two:Obj(oid) WHERE oid = $autostring_1                                                          |              1 |    1 |       2 |                |                    0/0 |
| |                          +-------------------------------------------------------------------------------------------------+----------------+------+---------+----------------+------------------------+
| +NodeIndexSeek@graph.db    | one:Obj(oid) WHERE oid = $autostring_0                                                          |              1 |    1 |       2 |                |                    0/0 |
+----------------------------+-------------------------------------------------------------------------------------------------+----------------+------+---------+----------------+------------------------+

7 rows available after 0 ms, consumed after another 44 ms

Now I've switched to parameters + UNWIND, but see no difference.

In java List I put 100k ids, then

final String qu = "MATCH (one:Obj {oid: 'id1' }) WITH one UNWIND $batch AS row MATCH (two:Obj {oid:row}), path=allshortestPaths((one) -[*0..4]-(two)) WHERE ALL (x IN RELATIONSHIPS(path) WHERE x.val<130) return path limit 10000";

Result r = session.run(qu, parameters("batch", orgs));

Search didn't stop even after 30 minutes. Also separate searches are better, because I can break the loop and at least process partial results.

So, any ideas? 1 month ago it worked much faster.

user_5319625 · Answer

Not really an answer: I didn't find/create a tool to detect large clusters of nodes. The problem was solved by adding more RAM to fit all 64GB of data.

Also, spent couple of days trying to boost performance even more, but failed. Different JVM settings almost do nothing (large heap, large pagecache). Community edition seems to be hard-limited to 10 cores, but effective limit is even lower, because difference between 4 and 10-20 core mode is just several percent.

graph regression detection in neo4j

1 Answers