Are there any limitations of writing custom cypher queries while running them through spark connector? I am using a query with call and apoc library procedures. The same query works fine in cypher shell, but fails with error -
pyspark.sql.utils.IllegalArgumentException: Please provide a valid WRITE query
when run through spark connector. Any documentation around what all are allowed in custom cypher using spark connector?
My query looks something like this -
WITH event.dVal as dVal, COLLECT(cVal) as cVals,
apoc.coll.toSet(apoc.coll.flatten(COLLECT(cVal.sdVals))) as sdVals
CALL {
WITH dVal, cVals, sdVals
MERGE (d:DVal {DVal:dVal})
WITH d, cVals, sdVals
CALL {
WITH d, sdVals
UNWIND sdVals as sdVal
WITH d, sdVal
MERGE (sd:DVal {DVal: sdVal})
WITH d, sd
OPTIONAL MATCH (d)-[rel3:Relation1]->(sd)
FOREACH (o IN CASE WHEN rel3 IS NULL THEN [1] ELSE [] END |
CREATE (d)-[:Relation1]->(sd)
)
RETURN sd
}
WITH d, cVals, COLLECT(id(sd)) as sds
CALL {
WITH d, cVals
UNWIND cVals as cVal
WITH d, cVal
MERGE (c:CVal {CId: cVal.id})
WITH d, c, cVal
OPTIONAL MATCH (c)-[rel1:Relation2]->(d)
FOREACH (o IN CASE WHEN rel1 IS NULL THEN [1] ELSE [] END |
CREATE (c)-[:Relation2]->(d)
)
}
}