I am trying to build a simple relationship in Neo4j using Spark-Neo4j connector. My dataframe looks like this:
df_new= spark.createDataFrame(
[("CompanyA",'A','CompanyA','B'),("CompanyB",'B','CompanyB','C') ],
["name",'gid','description','parent_gid']
)
The desired tree should look like this:
The query I wrote looks like this:
query = """
MERGE (c:Company {gid:event.gid})
ON CREATE SET c.name=event.name, c.description=event.description
ON MATCH SET c.name=event.name, c.description=event.description
MERGE (p:Company {gid:event.parent_gid})
MERGE (p)-[:PARENT_OF]->(c)
"""
df_new.write\
.mode("Overwrite")\
.format("org.neo4j.spark.DataSource")\
.option("url", "bolt://localhost:7687")\
.option("authentication.type", "basic")\
.option("authentication.basic.username", username)\
.option("authentication.basic.password", password)\
.option("query", query)\
.save()
However my code ends up creating node instead of merging it, and I end up with two nodes for company B
