Using Testcontainers lib, my CosmosDB container will not start inside Azure ADO pipeline using Ubuntu build agent. Running on my local machine, it works perfectly. Is there a way to debug this problem?
I refactored my pipeline to start CosmosDB in a separate build step BEFORE the tests run and in that case it worked.
When I try to start the container using the Testcontainers lib, the CosmosDB says it's starting but the container hangs and 55 seconds later the test setup fails.
How do you debug this problem? I have tried all sorts of things, including ramping up logging settings from my logback-test.xml so they log everything from testcontainers package.
I tried this and it gave me no additional info:
<logger name="org.testcontainers" level="DEBUG" additivity="false">
<appender-ref ref="STDOUT"/>
</logger>
<logger name="com.github.dockerjava" level="INFO" additivity="false">
<appender-ref ref="STDOUT"/>
</logger>
Here is the log output:
18:11:58.367 [main] INFO ?.m.com/cosmosdb/linux/azure-cosmos-emulator:latest] - Creating container for image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
...
18:11:59.230 [main] INFO ?.m.com/cosmosdb/linux/azure-cosmos-emulator:latest] - Container mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest is starting: 60154433735b0b8f9281e94910a1fb7c675918163d29e2788ec8864e2fcf9b81
...
18:11:59.402 [main] INFO o.t.c.wait.strategy.HttpWaitStrategy - /competent_tharp: Waiting for 300 seconds for URL: http://localhost:49154/_explorer/index.html (where port 49154 maps to container port 8081)
/agent/_work/_temp/160a9d74-cd5c-4b46-b79c-80cc9d2bb814.sh: line 10: 12511 Killed sudo mvn clean package
##[debug]Exit code 137 received from tool '/bin/bash'
##[debug]STDIO streams have closed for tool '/bin/bash'
##[error]Bash exited with code '137'.
##[debug]Processed: ##vso[task.issue type=error;]Bash exited with code '137'.
##[debug]task result: Failed
##[debug]Processed: ##vso[task.complete result=Failed;done=true;]
Another variation I tried gives me this exception:
java.lang.IllegalStateException: Container exited with code 255
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:497)
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:331)
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:329)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317)
Inside the docker container, it fails to reach "Started" state, and hangs right after it says "Starting". It never prints out the messages that the partitions are getting created:
cosmosdb
mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
RUNNING
This is an evaluation version. There are [145] days left in the evaluation period.
Starting