HARDCODE

 There are two keywords that you must understand when publishing a message: latency and throughput.  Throughput is a measure of how much data can be processed in a given amount of time. It's usually measured in bits per second (bit/s), or data packets per second. High throughput means the system can process a large amount of data quickly, which is often desirable in high-load scenarios. For example, if a Kafka producer can send 1000 messages per second to a broker, the throughput is 1000 messages per second. Latency, on the other hand, is a measure of time delay experienced in a system, the time it takes for a bit of data to travel from one point to another in a network. It is usually measured in milliseconds. Low latency means that data can be transferred quickly from source to destination. For example, if a message takes 10 milliseconds from the time it's sent by a Kafka producer until it's received by a broker, the latency is 10 milliseconds. In all systems, there's ...

 People say that Kafka is a dumb broker. It just holds data under some defined topics and forward messages from producers to consumers. It doesn't do much processing on the messages, doesn't route them based on content, and doesn't transform them. But that actually isn't entirely true. Kafka does much more than just storing messages. It keeps count of which consumer group consumed which message by tracking its offset. The consumer group is the group Id you give your consumer once you initialize it. Having more than one consumer helps you avoid consumer failures and scale better. If a message is consumed by one consumer in a group, no other consumers in the same group will receive it. However, all other consumers with other group Ids will receive the message. It's important to note that the offset of each message is basically the id of the message related to the group. So, if "message A" was published on a brand-new group its offset will be 0. But if anoth...

 Message brokers come in all colors. Each broker has its own edge. There are some brokers which are aimed to be simple and direct such as Amazon SQS (which is literally called Simple Queue Service). Some of them can be used in a simple fashion, but can also be used to implement complex patterns like RabbitMQ which is a topic I talked about in my previous posts . And then there is a broker that was designed to: handle heavy-duty streaming, maximize efficiency, allow you to scale up, serialize messages to bytes, partition messages and much more. That broker is indeed Apache Kafka . Due to its wide array of features, Kafka can be overwhelming. And sometimes it feels too exhausting to start comprehending its principles and what the broker can provide. So, why not strip down the broker from all of its extra shining features and just start from its core features like publishing and consuming simple JSON serialized string messages. What we're going to explore isn't just Apache Kafk...

    Exceptions can happen anytime and if you don't know how to handle them, then you have got a problem. Or maybe you just haven't checked out my post about exception handling. NET Global Exception Handling: 3 Techniques Beyond Try/Catch Blocks . Anyway, because of that you need to plan what to do if an exception happens during processing a message that you received from a  RabbitMQ queue. You have two options: Negatively Acknowledge (Nack) Using BasicNack:  This method is more flexible. It allows you to negatively acknowledge one or more messages. It takes three parameters: the delivery tag of the message to nack, a boolean indicating whether to nack multiple messages, and a boolean indicating whether to requeue the message.  If the multiple parameter is true, all messages up to and including the one with the specified delivery tag are nacked. If the requeue parameter is true, the nacked messages will be requeued. If it's false, the messages will be dis...

  RabbitMQ is often compared to as a post office . And this post office has multiple postmen (Exchanges) and each exchange is responsible for a number of mailboxes (Queues) and each postman has his own style of delivery (Exchange Types). Well, the postmen of RabbitMQ are by default careless.  So, if they get an address that they can't find, they just throw away the message . So, you have to set your ground rules and show them what to do if they get a non-existing address. In my last RabbitMQ post  Publish and Consume Messages Using RabbitMQ as a Message Broker in 4 Simple Steps 🐰  I showcased this problem when I deliberately sent a message to a non-existing queue name and as you can see in the post there was no exception thrown or anything. So basically, the message was a victim of fire and forget . So, let's use the same setup as the previous post and let's try two techniques that could help us avoid this problem. Our aim is to at least log any message that did not...

  SOLID principles seem to be the foundation that most engineers tend to build their solutions on. Using message brokers can help you achieve these core principles. Message brokers allow services and components to communicate directly, even if they were written in different languages or implemented on different platforms so it serves the Single-Responsibility Principle . Also, Message brokers act as intermediaries, allowing applications to communicate without direct dependencies. They provide a common abstraction for message handling which helps serve another principle which is  Dependency Inversion Principle . Of course, not to mention that most microservices, domain driven and event driven approaches are heavily dependent on message brokers. And message brokers come in all shapes and sizes and a lot of popular tools can help you set up a nice intermediatory space for your messages. Some of these tools are PubSub+ , Redis , Kafka , AWS SQS and more. Some of these are in...