I use AWS SDK for C++ on Windows and Linux environments. The task is to get an object's body:
const auto objectsReq = Aws::S3::Model::GetObjectRequest()
.WithBucket(bucket)
.WithKey(key);
// get the object from the bucket
auto objectsResp = s3Client->GetObject(objectsReq); // the error curlCode 28 is here
if (!objectsResp.IsSuccess())
return objectsResp.GetError().GetExceptionName();
On Windows it works fine with any number of threads.
Linux I have on a virtual machine (WMware Workstation player 16), with one thread it works but with thread count > 1, after some time it fails. The fail time correlates with thread numbers and applied timeouts:
Aws::Client::ClientConfiguration clientCfg;
clientCfg.httpRequestTimeoutMs = 120'000l;
clientCfg.connectTimeoutMs = 120'000l;
clientCfg.requestTimeoutMs = 120'000l;
Also I've tried to change retry strategy:
auto retryStrategy = std::make_shared<Aws::Client::StandardRetryStrategy>(5l); // strategy with custom max retries
clientCfg.retryStrategy = retryStrategy;
To clarify: the data set is huge and extending timeouts does not help. And I've checked other posts and AWS S3 doc and forums, and the only answer I got - was timeouts - which do not help.
UPDATE: aws log:
[DEBUG] 2022-08-31 14:42:57.589 AWSClient [140673533605632] Request Successfully signed
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] Making request to https://s3.us-west-1.amazonaws.com/dev-maksim/test_data/DevOps%20Data%20set/DevOps%20Data/qafiles/New/24Hourtest/nested_set/1/test_doc%20%2857%29.docx
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] Including headers:
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] amz-sdk-invocation-id: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] amz-sdk-request: attempt=1
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20220831/us-west-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date, Signature=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] content-type: application/xml
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] host: s3.us-west-1.amazonaws.com
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] user-agent: aws-sdk-cpp/1.9.160 Linux/5.15.0-46-generic x86_64 Clang/9.0.1
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] x-amz-api-version: 2006-03-01
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] x-amz-content-sha256: UNSIGNED-PAYLOAD
[TRACE] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] x-amz-date: 20220831T144257Z
[DEBUG] 2022-08-31 14:42:57.589 CurlHandleContainer [140673533605632] Attempting to acquire curl connection.
[INFO] 2022-08-31 14:42:57.589 CurlHandleContainer [140673533605632] Connection has been released. Continuing.
[DEBUG] 2022-08-31 14:42:57.589 CurlHandleContainer [140673533605632] Returning connection handle 0x7ff10c04b7b8
[DEBUG] 2022-08-31 14:42:57.589 CurlHttpClient [140673533605632] Obtained connection handle 0x7ff10c04b7b8
[DEBUG] 2022-08-31 14:42:57.589 CURL [140673533605632] (Text) STATE: INIT => CONNECT handle 0x7ff10c04b7b8; line 1834 (connection #-5000)
[DEBUG] 2022-08-31 14:42:57.589 CURL [140673533605632] (Text) Added connection 6. The cache now contains 1 members
[DEBUG] 2022-08-31 14:42:57.589 CURL [140673533605632] (Text) STATE: CONNECT => RESOLVING handle 0x7ff10c04b7b8; line 1880 (connection #6)
[DEBUG] 2022-08-31 14:42:57.647 CURL [140673533605632] (Text) family0 == v4, family1 == v6
[DEBUG] 2022-08-31 14:42:57.647 CURL [140673533605632] (Text) Trying 52.219.193.0:443...
[DEBUG] 2022-08-31 14:42:57.647 CURL [140673533605632] (Text) STATE: RESOLVING => CONNECTING handle 0x7ff10c04b7b8; line 1964 (connection #6)
[DEBUG] 2022-08-31 14:43:19.481 CURL [140673617475328] (Text) Connection timeout after 30000 ms
[DEBUG] 2022-08-31 14:43:19.481 CURL [140673617475328] (Text) multi_done: status: 28 prem: 1 done: 0
[DEBUG] 2022-08-31 14:43:19.481 CURL [140673617475328] (Text) The cache now contains 0 members
[DEBUG] 2022-08-31 14:43:19.481 CURL [140673617475328] (Text) Closing connection 5
[DEBUG] 2022-08-31 14:43:19.481 CURL [140673617475328] (Text) Expire cleared (transfer 0x7ff11404c848)
[ERROR] 2022-08-31 14:43:19.481 CurlHttpClient [140673617475328] Curl returned error code 28 - Timeout was reached
[DEBUG] 2022-08-31 14:43:19.482 CurlHandleContainer [140673617475328] Destroy curl handle: 0x7ff11404c848
[DEBUG] 2022-08-31 14:43:19.482 CurlHandleContainer [140673617475328] Created replacement handle and released to pool: 0x7ff1140af1d8
[DEBUG] 2022-08-31 14:43:19.482 AWSClient [140673617475328] Request returned error. Attempting to generate appropriate error codes from response
[ERROR] 2022-08-31 14:43:19.482 AWSClient [140673617475328] HTTP response code: -1
Resolved remote host IP address:
Request ID:
Exception name:
Error message: curlCode: 28, Timeout was reached
0 response headers:
[WARN] 2022-08-31 14:43:19.482 AWSClient [140673617475328] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
[DEBUG] 2022-08-31 14:43:19.482 AWSClient [140673617475328] Date header was not found in the response, can't attempt to detect clock skew
[WARN] 2022-08-31 14:43:19.482 AWSClient [140673617475328] Request failed, now waiting 0 ms before attempting again.
[TRACE] 2022-08-31 14:43:19.482 AWSClient [140673617475328] No content body, content-length headers
[DEBUG] 2022-08-31 14:43:19.482 AWSAuthV4Signer [140673617475328] Note: Http payloads are not being signed. signPayloads=0 http scheme=https
[DEBUG] 2022-08-31 14:43:19.482 AWSAuthV4Signer [140673617475328] Canonical Header String: amz-sdk-invocation-id:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
amz-sdk-request:attempt=2; max=11
content-type:application/xml
host:s3.us-west-1.amazonaws.com
x-amz-api-version:2006-03-01
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20220831T144319Z
[DEBUG] 2022-08-31 14:43:19.482 AWSAuthV4Signer [140673617475328] Signed Headers value:amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date
[DEBUG] 2022-08-31 14:43:19.482 AWSAuthV4Signer [140673617475328] Canonical Request String: GET
/dev-maksim/test_data/DevOps%20Data%20set/DevOps%20Data/qafiles/New/24Hourtest/nested_set/1/test_doc%20%28312%29.docx
amz-sdk-invocation-id:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
amz-sdk-request:attempt=2; max=11
content-type:application/xml
host:s3.us-west-1.amazonaws.com
x-amz-api-version:2006-03-01
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20220831T144319Z
amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
[DEBUG] 2022-08-31 14:43:19.482 AWSAuthV4Signer [140673617475328] Final String to sign: AWS4-HMAC-SHA256
20220831T144319Z
20220831/us-west-1/s3/aws4_request
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx