#Issue 25 - Message Queues For Scalable System
Message queues in a distributed system allow different parts to communicate and transfer data. It allows different parts to work independently and makes the overall system more reliable and scalable.
The basic structure of a message queue is like this: producer applications create messages and send them to the message queue. Then, there is a consumer application that connects to the queue and retrieves the messages for processing. The messages stay in the queue until the consumer picks them up.
Suppose, we have an e-commerce site, where after user confirms an order, an email and invoice is sent. So, we have an email service and an invoice generator service in backend.
If we keep our system like this, there are some issues we will face-
The request is processed synchronously, leading to a longer wait time for the user.
The system would struggle to handle bursts of requests efficiently. During peak times, this could lead to dropped requests or degraded performance.
Any server failure during processing would result in a lost request, requiring the user to resubmit their data.
Now if we introduce a message queue in the system, after each order we can push the data in message queue and send the success response to frontend without waiting for the email and invoice services. The email and invoice service can consume information from message queue when their resource become available and process their tasks accordingly.
Here, user requests are the produces, orders are being produced and sent to message queues. The email and invoice services are the consumers, as they are reading data from message queue to process requests.
Now, email and invoice service can process data sequentially and send data to user once their task is done. Requests are handled independently of user interactions, improving system responsiveness and scalability.
Books I am Reading
I am currently reading Miyazakiworld: A Life in Art By Susan Napier. I am a huge fan Hayao Miyazaki’s movies and this book dives into his mind, ideas and experiences that inspired him to make some of the most famous animated films. Very delightful read until now!
I recently finished Your Utopia By Bora Chung, it contains a series of short stories of science fiction genre. I liked some of them but it wasn’t the best of Bora Chung, overall it was an average read.
That’s it folks. Thanks for reading!
Informative read!!