Large amount of concurrent POST requests in Symfony & AWS RDS

user_1176717 0
Viewed 186

I work as a junior backend developer for a product that has about 100.000 users on average per month. Our senior backend developer suddenly quit and now I'm on my own with still a lot to learn. Hence this beginner's question.

We have a Symfony 3 application connected to a MySQL database (hosted on AWS RDS) using Doctrine and Sonata Admin bundles, running on about 2-4 AWS EC2 instances. Most requests we need to handle are GET requests which return the same data for every user, thus very cacheable, which is why we can serve a large number of users with only 2-4 EC2 instances.

Now I have been tasked with creating a very lightweight "analytics" system, that stores some data for each user every time they use the application. Meaning a POST request that writes to the database for each user every time they use the app.

We sometimes have 20.000 concurrent users in the application. How can I achieve this without our EC2 autoscaling blowing up, or blocking the database? This user data is not time sensitive, so it doesn't have to be written to the database right away. A delay of up to an hour or so is acceptable.

Thank you.

symfony amazon-rds amazon-web-services mysql php
1 Answers

I would recommend using SNS + SQS + Lambda to achieve this. One common design pattern is called “fanout.” In this pattern, a message published to an SNS topic is distributed to a number of SQS queues in parallel. By using this pattern, you can build applications that take advantage parallel, asynchronous processing. For example, you could publish a message to a topic every time the user uses the application. Independent processes (using AWS Lambda), each reading from a separate SQS queue and pushing them into the database. enter image description here Image shared from : https://www.jeremydaly.com/how-to-use-sns-and-sqs-to-distribute-and-throttle-events/

Advantages of this design pattern:

  1. This design pattern is highly scalable, even to million concurrent requests.
  2. Implementation is very easy since the components are integration ready
  3. This pattern is reusable and can be extended to any new future requirements.
  4. There is no need to plan for a capital investment of servers, all components are serverless and you can pay for only the usage.
  5. Using Cloud Formation Templates or SAM, integration can be automated.
  6. Code change needed in existing system is very less.
user_54040240
Related