Testing Environment Laptop is 4 core CPU, 16GB RAM and 512GB SSD.
Elasticsearch is running single node on windows server 2012 VM. This VM has 16GB RAM and 4 core CPU.
The Elasticsearch have 18 GB data. I am using NEST version: 7.6.1 for indexing and query Same query with Postman is taking only 1221 ms but with NEST client it is taking about 10-13 seconds for 10000 size. When I decrease the page size query time is decreased for 3000 it is taking almost 5 seconds.
Benchmarks of same query for REST Client and NEST Client. Nest client took: 12384 ms Nest client documents: 10000 Rest client took: 1966 ms
I have build Elasticsearch repository in asp.net core 3.1.
Settings of Client
var settings = new ConnectionSettings(new Uri(url))
.EnableDebugMode()
.DefaultIndex(defaultIndex);
var client = new ElasticClient(settings);
services.AddSingleton<IElasticClient>(client);`
Get List Method
public List GetList(SearchDescriptor descriptor, string index = null) where T : class, IElasticBaseModel
{
descriptor.From(0)
.Take(scrollPageSize).Scroll("5m");
index = index ?? _options.DefaultIndex;
descriptor = descriptor.Index(index);
var searchResponse = _client.Search<T>(descriptor);
var results = new List<T>();
while (true)
{
if (!searchResponse.IsValid || string.IsNullOrEmpty(searchResponse.ScrollId))
throw new Exception($"Search error: {searchResponse.ServerError.Error.Reason}");
if (!searchResponse.Documents.Any())
break;
results.AddRange(searchResponse.Documents);
searchResponse = _client.Scroll<T>("5m", searchResponse.ScrollId);
}
_client.ClearScroll(new ClearScrollRequest(searchResponse.ScrollId));
return results;
}
This query took 11.5 seconds.
Below is Debug Info from NEST client.
Valid NEST response built from a successful (200) low level call on POST: /position/_search?pretty=true&error_trace=true&typed_keys=true&scroll=5m
Audit trail of this API call: [1] HealthyResponse: Node: http://myip:9200/ Took: 00:00:08.9185311 Request:
{"from":0,"query":{"bool":{"filter":[{"bool":{"must":[{"geo_bounding_box":{"boost":1.1,"validation_method":"coerce","type":"indexed","location":{"bottom_right":{"lat":47.6125,"lon":39.575},"top_left":{"lat":64.8875,"lon":-17.075}}}}],"should":[{"bool":{"filter":[{"bool":{"should":[{"bool":{"must":[{"terms":{"machine":["6F1401","6G1519"]}},{"range":{"time":{"gte":"2019-09-01T00:00:00Z","lte":"2019-09-30T23:59:59.9999999Z"}}}]}}]}}]}}]}}]}},"size":10000}`
Response:
{
"_scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAHY_0WVlFRQlpTazJSUjZKbjJ4OGplQlJ3UQ==",
"took" : 2876,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : "59128374",
"relation" : "eq"
},
"max_score" : 0.0,
"hits" : [
{
"_index" : "position",
"_type" : "_doc",
"_id" : "6H1561_1541153618.03",
"_score" : 0.0,
"_source" : {
"id" : "6H1561_1541153618.03",
"time" : "2018-11-02T10:13:38.0300000Z",
"machine" : "6H1561",
"orderid" : 56,
"bunkerunloading" : false,
"operatingmode" : 1,
"depthguidanceon" : true,
"engineon" : true,
"istimelygap" : false,
"identifier" : 6852,
"location" : {
"lat" : 50.613713333,
"lon" : 4.014146667
}
}
}