I am developing a simple micro-service based application using Spring Boot, Kafka & MongoDB.
There are 4 Spring Boot based REST services that performs CRUD on MongoDB(shared between all services for now) and publish/poll messages to/from Kafka cluster.
Currently I have them all deployed on hand crafted AWS EC2 instances. Kafka/Zookeeper has 3 brokers/instances. MongoDb with no replication/sharding.
What do I want to achieve ?
- Run the micro-services in docker containers managed by ECS.
- Ability to scale out these services.
- Ability to create dedicated environment(DEV, QA, UAT, PROD) on ECS.
- Run Kafka and MongoDB cluster with replication and sharding as containers managed by ECS.
I am fairly new to Docker & ECS concepts as such and not able to find exactly how is it done. I am able to create Dockerfile, image and push to docker hub and use one of the service ( that has no external dependencies ) on ECS. One of the Dockerfile looks like this:-
FROM openjdk:8-jre-alpine
EXPOSE 443 443
LABEL "author":"Kumar Sambhav Jain"
COPY build/distributions/app.tar /opt/app/
RUN tar -xvf /opt/app/admin.tar -C /opt/app/
WORKDIR /opt/app/admin/bin
ENTRYPOINT ["./admin"]
Questions:-
- What is ideal or right way to deploy the whole application ? Do I need ECS cluster per environment (DEV, QA, UAT) or ECS cluster per service type - two services need to be publicly exposed on port 443/80.
- How do the service connect to MongoDB and Kafka running in containers ? Currently Kafka broker and mongo connection are maintained in application.yml files like this:-
spring: data: mongodb: host: 172.31.18.25 database: abcd username: test password: abcd@123 cloud: stream: kafka: binder: zk-nodes: - 172.31.18.58:2181 - 172.31.28.201:2181 - 172.31.25.60:2181 brokers: - 172.31.18.58:9092 - 172.31.28.201:9092 - 172.31.25.60:9092
How does it change once everything is containerized ?
Reference to any study material on this exact topic.