Amazon DynamoDB Async client http metrics

user_1667977 0
Viewed 1507

I am using Amazon SDK (Java) DynamoDB Async client v2.10.14 with custom configuration:

DynamoDbAsyncClientBuilder = DynamoDbAsyncClient
            .builder()
            .region(region)
            .credentialsProvider(credentialsProvider)
            .httpClientBuilder(
                NettyNioAsyncHttpClient.builder()
                    .readTimeout(props.readTimeout)
                    .writeTimeout(props.writeTimeout)
                    .connectionTimeout(props.connectionTimeout)
            )

I often run into a connect timeout:

io.netty.channel.ConnectTimeoutException: connection timed out: dynamodb.region.amazonaws.com/1.2.3.4:443

I expect this is due to my settings, but I need aggressive timeouts. I used to run into the same issues with the defaults anyway (it just took longer). I would like to know why I am getting into this situation. My gut feel is that it's related to connection pool exhaustion, or other issues with the pool.

Are there any metrics I can turn on to monitor this?

amazon-dynamodb metrics
1 Answers

It seems like your application is a "latency-aware DynamoDB client application". The underlying HTTP client behavior needs to be tuned for its retry strategy. Luckily, the AWS Java SDK provides full control over the HTTP client behavior and retry strategies, this documentation helps to explain how to tune AWS Java SDK HTTP request settings and parameters for the underlying HTTP client:

https://aws.amazon.com/blogs/database/tuning-aws-java-sdk-http-request-settings-for-latency-aware-amazon-dynamodb-applications/

In the document it provides an example on how to tune the five HTTP client configuration parameters that are set during the ClientConfiguration object creation and discusses very comprehensively about each of the parameters:

  • ConnectionTimeout
  • ClientExecutionTimeout
  • RequestTimeout
  • SocketTimeout
  • The DynamoDB default retry policy for HTTP API calls with a custom maximum error retry count

Tuning this latency-aware DynamoDB application "requires an understanding of the average latency requirements and record characteristics (such as the number of items and their average size) for your application, interdependencies between different application modules or microservices, and the deployment platform. Careful application API design, proper timeout values, and a retry strategy can prepare your application for unavoidable network and server-side issues"

Quoted from: https://aws.amazon.com/blogs/database/tuning-aws-java-sdk-http-request-settings-for-latency-aware-amazon-dynamodb-applications/

user_44259640
Related