Why is lua so slow in redis? Any workarounds?

user_58109 0
Viewed 4096

I'm evaluating the use of lua scrips in redis, and they seem to be a bit slow. I a benchmark as follows:

  1. For a non-lua version, I did a simple SET key_i val_i 1M times
  2. For a lua version, I did the same thing, but in a script: EVAL "SET KEYS[1] ARGV[1]" 1 key_i val_i

Testing on my laptop, the lua version is about 3x slower than the non-lua version. I understand that lua is a scripting language, not compiled, etc. etc. but this seems like a lot of performance overhead--is this normal?

Assuming this is indeed normal, are there any workaround? Is there a way to implement a script in a faster language, such as C (which redis is written in) to achieve better performance?

Edit: I am testing this using the go code located here: https://gist.github.com/ortutay/6c4a02dee0325a608941

redis lua
3 Answers

so there is now a workaround using a module created by John Sully. It works for Redis and KeyDB and allows you to use the V8 JIT engine which runs complex scripts much faster than Lua scripts. https://github.com/JohnSully/ModJS

user_126710310

I stumbled on this thread and was also curious of the benchmark results. I wrote a quick Ruby script to compare them. The script does a simple "SET/GET" operation on the same key using different options.

require "redis"

def elapsed_time(name, &block)
  start = Time.now
  block.call
  puts "#{name} - elapsed time: #{(Time.now-start).round(3)}s"
end

iterations = 100000
redis_key = "test"

redis = Redis.new

elapsed_time "Scenario 1: From client" do
  iterations.times { |i|
    redis.set(redis_key, i.to_s)
    redis.get(redis_key)
  }
end

eval_script1 = <<-LUA
redis.call("SET", "#{redis_key}", ARGV[1])
return redis.call("GET", "#{redis_key}")
LUA

elapsed_time "Scenario 2: Using EVAL" do
  iterations.times { |i|
    redis.eval(eval_script1, [redis_key], [i.to_s])
  }
end

elapsed_time "Scenario 3: Using EVALSHA" do
  sha1 = redis.script "LOAD", eval_script1
  iterations.times { |i|
    redis.evalsha(sha1, [redis_key], [i.to_s])
  }
end

eval_script2 = <<-LUA
for i = 1,#{iterations} do
  redis.call("SET", "#{redis_key}", tostring(i))
  redis.call("GET", "#{redis_key}")
end
LUA

elapsed_time "Scenario 4: Inside EVALSHA" do
  sha1 = redis.script "LOAD", eval_script2
  redis.evalsha(sha1, [redis_key], [])
end

eval_script3 = <<-LUA
for i = 1,2*#{iterations} do
  redis.call("SET", "#{redis_key}", tostring(i))
  redis.call("GET", "#{redis_key}")
end
LUA

elapsed_time "Scenario 5: Inside EVALSHA with 2x the operations" do
  sha1 = redis.script "LOAD", eval_script3
  redis.evalsha(sha1, [redis_key], [])
en

I got the following results running on my Macbook pro

Scenario 1: From client - elapsed time: 11.498s
Scenario 2: Using EVAL - elapsed time: 6.616s
Scenario 3: Using EVALSHA - elapsed time: 6.518s
Scenario 4: Inside EVALSHA - elapsed time: 0.241s
Scenario 5: Inside EVALSHA with 2x the operations - elapsed time: 0.5s

In summary:

  • scenario 1 vs. scenario 2 show that the main contributor is the round trip time as scenario 1 makes 2 requests to Redis while scenario 2 only makes 1 and scenario 1 is ~2x the execution time
  • scenario 2 vs. scenario 3 shows that EVALSHA does provide some benefit and I am sure this benefit increases the more complex the script gets
  • scenario 4 vs scenario 5 shows the overhead of invoking the script is near minimal as we doubled the number of operations and saw a ~2x increase in execution time.
user_4623980
Related