Drools performance

user_14787700 0
Viewed 285

I have an issue regarding performance of Drools on different Machines. I made very simple JMH Benchmark test:

package ge.magticom.rules.benchmark;
import ge.magticom.rules.benchmark.Subscriber
rule "bali.free.smsparty"
    activation-group "main"
    salience 4492
    when
        $subs:Subscriber(chargingProfileID == 2)

    then
    $subs.setResult(15);
end

rule "bali.free.smsparty5"
    activation-group "main"
    salience 4492
    when
        $subs:Subscriber(chargingProfileID == 3)

    then
    $subs.setResult(14);
end

    @Benchmark
    public Subscriber send() throws Exception {
        Subscriber subscriber = new Subscriber();
        subscriber.setChargingProfileID(5);
        StatelessKieSession session = ruleBase.newStatelessKieSession();
        ArrayList<Object> objs = new ArrayList<Object>();
        objs.add(subscriber);
        session.execute(objs);
        return subscriber;
    }

On Home development machine NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)"

(Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz 12 Threads ) 64 GB Memory with JDK 11 and have very great performance: With 7 threads a have nearly 2M operation per second(Stateless)

Benchmark             Mode  Cnt        Score        Error  Units
RulesBenchmark.send  thrpt    5  2154292.750 ± 149405.498  ops/s

But on preproduction server Which is Intel(R) Xeon(R) Gold 6258R CPU @ 2.70GHz with 112 Threads and 1 TB RAM I have half of performance (Even increasing threads) NAME="Oracle Linux Server" VERSION="8.4"

Benchmark             Mode  Cnt        Score        Error  Units
RulesBenchmark.send  thrpt    5  1084939.195 ± 107897.663  ops/s

I'm trying to test our billing system with java 11 and Drolls 7.54.0.Final. Our system was based on Jrockit realtime 1.6 and drools version 4.0.3. We are moving system from Sun Solaris SPARK to Intel base system.

Running same rules with Jrockit 1.6 I got even worth performance issue with Home and Preproduction environment: Home test benchmark:

Benchmark             Mode  Cnt       Score      Error  Units
RulesBenchmark.send  thrpt   20  692054.563 ± 3507.519  ops/s

Preproduction benchmark:

Benchmark                   Mode  Cnt        Score       Error  Units
RulesBenchmark.send        thrpt   20   382283.288 ±  6405.953  ops/s

As you can see, it's nearly half performance of very simple rules.

But for real rules, such as our online charging system, it's even bad performance :

On home environment I got

Benchmark                     Mode  Cnt    Score    Error  Units
WorkerBenchmark.send         thrpt    5  152.846 ± 87.076  ops/s

this means 1 message contains nearly 100 iterations so in 00:01:49 benchmark processed 16287 sessions with 430590 events of rule calls. single rule call is about 2.33 millisecond in average, which is not very great, but not as bad as on preproduction

On Preproduction server

Benchmark                     Mode  Cnt   Score   Error  Units
WorkerBenchmark.send         thrpt    5  35.013 ± 9.565  ops/s

in 00:01:54 I got only 3723 sessions which contains wholly 98571 events of rule calls. Each call is 10.7299 msc in average.

During running all these benchmark nothing was running on preproduction system. But on home environment there is a lot of development tools, was running tests from Intellij IDEA

Can you suggest anything, which may cause such difference in performance. I tried different java versions and vendors. These results are based on oracle-jdk-11.0.8.

Here are kernel params of Preproduction server:


 fs.file-max = 6815744
 kernel.sem = 2250 32000 100 128
 kernel.shmmni = 4096
 kernel.shmall = 1073741824
 kernel.shmmax = 4398046511104
 net.core.rmem_default = 262144
 net.core.rmem_max = 4194304
 net.core.wmem_default = 262144
 net.core.wmem_max = 1048576
 net.ipv4.conf.all.rp_filter = 2
 net.ipv4.conf.default.rp_filter = 2
 fs.aio-max-nr = 1048576
 net.ipv4.ip_local_port_range = 9000 65500
jmh drools
2 Answers

This is just a very wild guess since I definitively don't have enough information, but are the 2 environments using the same garbage collectors configured in the same way? Maybe you're using ParallelGC (which in my experience is better for pure throughput as you're measuring) on one side and G1 on the other?

user_1127790

Thanks for answer.

I used several GC configuration, none of them were ParallelGC. I think GC is not a problem. I used ZGC in final tests and GC pause times are not above 5 msc (tested also with java 16 and pause times are below 100 microsecond ). :

@Fork(value = 2, jvmArgs = {"--illegal-access=permit", "-Xms10G", "-XX:+UnlockDiagnosticVMOptions", "-XX:+DebugNonSafepoints",
    "-Xmx10G","-XX:+UnlockExperimentalVMOptions", "-XX:ConcGCThreads=5", "-XX:ParallelGCThreads=10", "-XX:+UseZGC", "-XX:+UsePerfData", "-XX:MaxMetaspaceSize=10G", "-XX:MetaspaceSize=256M"}

java -version

    java version "11.0.8" 2020-07-14 LTS
    Java(TM) SE Runtime Environment 18.9 (build 11.0.8+10-LTS)
    Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.8+10-LTS, mixed mode)

Here is Flame graph generated with AsyncProfilers FlameGraph from home PC

This is from server

As you can see, at home enviroment Java process is using 95% of whole time, but on server only 65%. the time difference is also obviouse :

RulesBenchmark.send                       thrpt    5  1612318.098 ± 64712.672   ops/s

Home Result FlameGraph.html

RulesBenchmark.send                       thrpt    5  775498.081 ± 72237.890   ops/s

Server Flame Graph.html

user_147877000
Related