Multithreading in Dynamic Data Driven Testing

Question

I have a scenario where I need to run the same test N number of times, each time with slightly different data. The goal here is to capture a rare behavior in the application.

So I come up with the following test codes:

Scenario:
  * json testJSONArray = # A JSON Array containing N number of generated test data entries, each of them is a JSON object
  * call read('map.feature@testMultiple') testJSONArray

From what I understand, calling a feature and passing in a N-sized JSON array will make the feature run for N number of times.

The question here is, is this run multi-threaded? Because from what I am observing, the run time seems to be longer than expected.

For example, when I run with 10 threads:

When N = 1, the run time is about 20 seconds.
When N = 5, the run time is about 90 seconds.
When N = 10, the run time is about 150 seconds.

So I just want to make sure the scenario here is fulling utilizing multithread feature of Karate.

Edit: Adding in the runner

Results results = Runner.path(features)
  .tags(tags)
  .parallel(10);

Thank you in advance.

user_143475 · Answer

A call is never multi-threaded. You have to get proper Scenario-s into the picture.

This answer may give you the hint you are looking for: https://stackoverflow.com/a/60387907/143475

Also refer the documentation: https://github.com/karatelabs/karate#dynamic-scenario-outline - and pay attention to the option of using a JS function to generate data. But note that the "Dynamic Scenario Outline" will undergo a bit of a refactor.

Multithreading in Dynamic Data Driven Testing

1 Answers