When we are trying to query a datasource , the query runs for 5+ mins and throws network error in console. We are trying to fetch some huge result in millions. Is this some limitation in druid, where we can't fetch huge records? Other aggregated queries are running fine and producing results.
SELECT * FROM "datasource"
WHERE "__time" >= TIMESTAMP '2021-06-21' and "__time" <= TIMESTAMP '2021-06-23' and consumerid=1234
Segment Granularity: DAY
Query Granularity : DAY
Segments Created: 736
Avg Segment Size: 462 MB
Total Datasource Size: 340.28 GB
Replicated Size: 680.55 GB
secondary partition: single_dim (consumerid)
Is there some way to overcome this Issue?
I've tried this via API also, after 5.30 seconds it throws error.
curl --location --request POST 'https://druiddev-druid.int.org/druid/v2/sql' --header 'Authorization: Basic Username/p' --header 'Content-Type: application/json' --data-raw '{
"query": "SELECT * FROM datasource WHERE consumerid=1234 and buydate='\''01/01/2021'\''",
"resultFormat" : "csv",
"batchSize":20480
}' > output.dat
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 17.9M 0 17.9M 0 165 58699 0 --:--:-- 0:05:20 --:--:-- 86008
curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)