Warmup Neo4j database after spring initialization

Viewed 204

I use a Neo4J database with nearly 500k nodes. When I startup my Spring application and do the first query, it takes about 4-5 seconds. This happens just for the first query, so I thought I could do a warmup after spring is initialized to make all subsequent queries faster.

This is my applicationContext.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:util="http://www.springframework.org/schema/util"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-2.5.xsd">

    <!-- neo4j database -->
    <util:map id="config">
        <entry key="enable_remote_shell" value="false" />
    </util:map>

    <bean id="graphDbFactory" class="org.neo4j.graphdb.factory.GraphDatabaseFactory" />

    <bean id="graphDbBuilder" factory-bean="graphDbFactory"
        factory-method="newEmbeddedDatabaseBuilder">
        <constructor-arg value="/path/to/db" />
    </bean>

    <bean id="graphDbBuilderFinal" factory-bean="graphDbBuilder"
        factory-method="setConfig">
        <constructor-arg ref="config" />
    </bean>

    <bean id="graphDatabaseService" factory-bean="graphDbBuilderFinal"
        factory-method="newGraphDatabase" destroy-method="shutdown" class="org.neo4j.graphdb.GraphDatabaseService"/>

    <context:component-scan base-package="com.app.components" />
</beans>

I saw that one way to wait for Spring beans initialization was to implement an ApplicationListener, and the regular way to warmup the database with Neo was to call the apoc.runtime.warmup() function, so I did as follows:

package com.app.components;

@Component
public class StartupTasks implements ApplicationListener<ContextRefreshedEvent> {

    private static final Logger LOG = LogManager.getLogger(StartupTasks.class);

    @Autowired
    GraphDatabaseService db;

    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {
        executeTestSelect();
    }

    private void executeTestSelect() {
        LOG.info("Warming up Neo4j...");

        Transaction tx = db.beginTx();
        db.execute("CALL apoc.warmup.run()");
        tx.close();

        LOG.info("Warmup complete.");
    }
}

This one didn't work, everything gets correctly logged but the first neo4j query is still slow.

Since this approach didn't work, I edited the executeTestSelect method to run a real query and process the results like this:

private void executeTestSelect() {
    String textToSearch = "a"; // returns almost all nodes, should make neo4j cache them all
    Transaction tx = db.beginTx();
    Map<String, Object> params = new HashMap<String, Object>();
    params.put("textToSearch", textToSearch );
    Result resultSet = db.execute("MATCH (n:PROD) WHERE n.description CONTAINS {textToSearch} RETURN n",
                params);
    Iterator<Node> results = resultSet.columnAs("n");
    int count = 0;
    while (results.hasNext()) {
        results.next();
        count++:
    }
    tx.close();
    LOG.info("Neo4j cache done. Processed " + count + " nodes.");
}

This time the startup takes 4-5 seconds just to execute the query, but then it prints

Neo4j cache done. Processed 0 nodes.

This is weird, because the exact same query run when the app is fully initialized returns 450k nodes.

What am I missing? Is it possible that when reaching onApplicationEvent the Neo4j DB hasn't been initialized yet and can't execute queries?

How do I correctly warmup a neo4j database right after the complete spring app initialization?

1 Answers
Related