← Back to Articles         
Idempotent Consumer

Idempotent Consumer

Published on 29 Sep 2026     9 min read Microservices
Messaging & Event-Driven

What Is an Idempotent Consumer?

An Idempotent Consumer is a message consumer designed so that processing the same message multiple times does not cause the business operation to happen multiple times.

In simple words:

Same message received many times → business effect happens only once.

This is extremely important in microservices because message brokers can redeliver messages.

1. Simple Example

Suppose Inventory Service receives:

OrderCreated
EventId  = EVT-1001
Product  = Laptop
Quantity = 2

Current stock:

Stock = 100

First delivery:

EVT-1001
   ↓
Inventory Consumer
   ↓
100 - 2
   ↓
Stock = 98 ✓

Now the broker accidentally/redelivers the same event:

EVT-1001
   ↓
Inventory Consumer

Without idempotency:

98 - 2
  ↓
96 ❌

With an idempotent consumer:

EVT-1001
   ↓
Already processed?
   ↓
YES
   ↓
Ignore duplicate

Stock remains 98 ✓

2. Why Do We Need It?

Messaging systems commonly use delivery semantics where a message may be delivered more than once.

For example:

Broker
   │
   │ EVT-1001
   ▼
Consumer
   │
   ├── Update database ✓
   │
   ├── ACK message
   │
   └── Network failure ❌

The database update succeeded, but the broker didn't receive the acknowledgement.

The broker may think:

"I don't know whether EVT-1001
was successfully processed."

So it redelivers:

Broker
   │
   │ EVT-1001 again
   ▼
Consumer

The consumer must safely handle this.

3. What Does "Idempotent" Mean?

An operation is idempotent when executing it multiple times produces the same intended result as executing it once.

For example:

UPDATE Orders
SET Status = 'Paid'
WHERE Id = 1001;

Running it repeatedly:

Paid
 ↓
Paid
 ↓
Paid

The final state remains:

Paid

This operation is naturally close to idempotent.

But:

UPDATE Products
SET Stock = Stock - 5
WHERE Id = 1001;

is not.

Run once:

100 → 95

Run twice:

100 → 95 → 90 ❌

4. Idempotent Consumer Pattern

A common implementation gives every event a unique ID:

{
  "eventId": "EVT-1001",
  "orderId": 5001,
  "productId": 100,
  "quantity": 2
}

The consumer stores processed message IDs:

ProcessedMessages

MessageId
----------------
EVT-1001
EVT-1002
EVT-1003

When a message arrives:

Receive Message
      ↓
Check MessageId
      ↓
Already processed?
      │
   ┌──┴──┐
  YES    NO
   │      │
   ▼      ▼
 Ignore  Process
          ↓
      Save MessageId

5. Simple C# Example

Event:

public class OrderCreatedEvent
{
    public Guid EventId { get; set; }

    public int OrderId { get; set; }

    public int ProductId { get; set; }

    public int Quantity { get; set; }
}

Processed-message entity:

public class ProcessedMessage
{
    public Guid MessageId { get; set; }

    public DateTime ProcessedAtUtc { get; set; }
}

Consumer:

public async Task HandleAsync(
    OrderCreatedEvent message)
{
    bool alreadyProcessed =
        await _dbContext.ProcessedMessages
            .AnyAsync(x =>
                x.MessageId == message.EventId);

    if (alreadyProcessed)
    {
        Console.WriteLine(
            "Duplicate message ignored.");

        return;
    }

    var product =
        await _dbContext.Products
            .SingleAsync(x =>
                x.Id == message.ProductId);

    product.Stock -= message.Quantity;

    _dbContext.ProcessedMessages.Add(
        new ProcessedMessage
        {
            MessageId = message.EventId,
            ProcessedAtUtc = DateTime.UtcNow
        });

    await _dbContext.SaveChangesAsync();
}

6. Important: Use a Unique Constraint

Checking:

AnyAsync(...)

alone is not sufficient.

Imagine two consumer instances process the same message simultaneously:

Consumer A                   Consumer B

Check EVT-1001               Check EVT-1001
     ↓                            ↓
Not Found                    Not Found
     ↓                            ↓
Process                       Process

Both could process it.

Therefore, the database should enforce uniqueness.

For example:

CREATE TABLE ProcessedMessages
(
    MessageId UNIQUEIDENTIFIER
        NOT NULL PRIMARY KEY,

    ProcessedAtUtc DATETIME2
        NOT NULL
);

Now the database provides a final protection against inserting the same MessageId twice.

7. Most Important Part — Transaction

Consider this:

Update Inventory ✓
       ↓
APPLICATION CRASH
       ↓
Save ProcessedMessage ✗

The broker redelivers the event.

The consumer sees no processed record and reduces inventory again.

Therefore, ideally:

Business Operation
       +
ProcessedMessage
       ↓
Same Database Transaction

For example:

BEGIN TRANSACTION
       │
       ├── Reduce Inventory
       │
       └── Insert ProcessedMessage
       │
       ▼
     COMMIT

Either both succeed:

Inventory updated ✓
Message recorded  ✓

or both fail:

Inventory updated ✗
Message recorded  ✗

8. Transaction Example

public async Task HandleAsync(
    OrderCreatedEvent message)
{
    await using var transaction =
        await _dbContext.Database
            .BeginTransactionAsync();

    try
    {
        bool alreadyProcessed =
            await _dbContext.ProcessedMessages
                .AnyAsync(x =>
                    x.MessageId == message.EventId);

        if (alreadyProcessed)
        {
            await transaction.RollbackAsync();
            return;
        }

        var product =
            await _dbContext.Products
                .SingleAsync(x =>
                    x.Id == message.ProductId);

        product.Stock -= message.Quantity;

        _dbContext.ProcessedMessages.Add(
            new ProcessedMessage
            {
                MessageId = message.EventId,
                ProcessedAtUtc = DateTime.UtcNow
            });

        await _dbContext.SaveChangesAsync();

        await transaction.CommitAsync();
    }
    catch
    {
        await transaction.RollbackAsync();
        throw;
    }
}

The essential design is:

┌─────────────────────────────┐
│ Database Transaction        │
│                             │
│  Update Business Data       │
│           +                 │
│  Record MessageId           │
│                             │
│  COMMIT                     │
└─────────────────────────────┘

9. When Do We ACK the Message?

The message should normally be acknowledged after successful processing.

For Azure Service Bus:

await handler.HandleAsync(message);

await args.CompleteMessageAsync(
    args.Message);

Flow:

Receive Message
      ↓
Check duplicate
      ↓
Process business logic
      ↓
Save ProcessedMessage
      ↓
Commit transaction
      ↓
ACK / Complete

10. What If ACK Fails?

Suppose:

Process business logic ✓
Database commit        ✓
ACK                    ❌

The broker may redeliver:

EVT-1001
    ↓
Consumer
    ↓
ProcessedMessages
    ↓
EVT-1001 already exists
    ↓
Ignore business operation
    ↓
ACK

That's exactly what an idempotent consumer protects against.

11. Naturally Idempotent Operations

Sometimes the business operation itself is naturally idempotent.

Example:

UPDATE Orders
SET Status = 'Shipped'
WHERE Id = 1001;

Running twice may still result in:

Status = Shipped

But operations such as these are dangerous:

Stock = Stock - 5

Balance = Balance - 1000

LoyaltyPoints = LoyaltyPoints + 100

ChargeCreditCard(5000)

Repeating them changes the result.

For those cases, explicit idempotency protection is especially important.

12. Business Idempotency Key

For critical operations, you may also have a business identifier.

For example:

PaymentId = PAY-10001

Create a unique constraint:

CREATE UNIQUE INDEX UX_Payments_PaymentId
ON Payments(PaymentId);

Then:

PAY-10001
    ↓
First request
    ↓
Process Payment ✓


PAY-10001
    ↓
Duplicate
    ↓
Payment already exists
    ↓
Don't charge again ✓

This is particularly important for:

Payments
Bank transfers
Orders
Invoices
Refunds

13. Idempotent Consumer vs Inbox Pattern

They are closely related.

Idempotent Consumer

The behavior/principle:

Don't produce duplicate business effects when the same message is processed repeatedly.

Inbox Pattern

A common implementation technique:

Message Broker
      ↓
Consumer
      ↓
Inbox / ProcessedMessages
      ↓
Business Processing

The Inbox stores or tracks incoming messages so duplicates can be detected.

14. Inbox + Outbox

These two patterns solve different problems.

                  PRODUCER
                     │
              Business Change
                     +
               Outbox Message
                     │
                     ▼
              Message Broker
                     │
                     ▼
                  CONSUMER
                     │
              Inbox / MessageId
                     +
               Business Change

Outbox

Protects the producer side:

"My database was updated, but what if publishing fails?"

Inbox / Idempotent Consumer

Protects the consumer side:

"What if the same message is delivered again?"

15. Example End-to-End Architecture

Order Service
     │
     ▼
┌─────────────────────┐
│ SQL Transaction     │
│                     │
│ Insert Order        │
│ Insert Outbox       │
└──────────┬──────────┘
           │
           ▼
     Outbox Worker
           │
           ▼
     Message Broker
           │
           │ OrderCreated
           ▼
   Inventory Service
           │
           ▼
┌────────────────────────┐
│ SQL Transaction        │
│                        │
│ Check/record MessageId │
│ Reduce Inventory       │
└────────────────────────┘

This combination is commonly used for reliable event-driven microservices.

16. Idempotent Consumer Does Not Mean "No Duplicates"

This distinction is important.

An idempotent consumer does not necessarily prevent the broker from delivering duplicates.

Instead:

Broker delivers:

EVT-1001
EVT-1001
EVT-1001

The consumer ensures:

Business effect:

EVT-1001 → Process ✓
EVT-1001 → No duplicate effect
EVT-1001 → No duplicate effect

So:

We tolerate duplicate delivery while preventing duplicate business effects.

17. At-Least-Once + Idempotency

Many messaging architectures effectively work around this principle:

At-least-once delivery
        +
Idempotent Consumer
        ↓
Safe duplicate handling

Rather than assuming:

Message will always arrive
exactly once

we design the consumer to safely handle retries and redelivery.

18. Advantages

An idempotent consumer provides:

  • Protection from duplicate processing
  • Safe message redelivery
  • Better retry handling
  • Better crash recovery
  • More reliable event-driven systems
  • Protection for inventory/payment operations
  • Better compatibility with at-least-once delivery

19. Disadvantages

It adds complexity:

  • Processed-message storage
  • Additional database operations
  • Unique constraints
  • Transaction handling
  • Concurrency handling
  • Cleanup/retention strategy
  • Additional application logic

For important side effects, this is usually worthwhile.

20. Key Points

For interviews, remember:

  • Idempotent Consumer prevents duplicate business effects.
  • The same message may be delivered multiple times.
  • Give messages a unique MessageId / EventId.
  • Track processed message IDs when necessary.
  • Use a database unique constraint to handle concurrency.
  • Business change + processed-message record should ideally be in the same local transaction.
  • ACK the broker only after successful processing/commit.
  • If ACK fails, redelivery can occur.
  • The idempotency check makes that redelivery safe.
  • Some operations are naturally idempotent.
  • Critical operations may additionally use a business idempotency key.
  • Inbox Pattern is a common implementation of consumer-side deduplication.
  • Transactional Outbox solves the corresponding producer-side reliability problem.

21. Interview Questions and Answers

Q1. What is an idempotent consumer?

Answer:

An idempotent consumer is a message consumer designed so that processing the same message multiple times does not create duplicate business effects.

Q2. Why do we need an idempotent consumer?

Answer:

Because a message broker may redeliver a message when acknowledgement fails, the consumer crashes, or processing is retried.

Q3. How do you implement an idempotent consumer?

Answer:

Give each message a unique ID, store processed IDs, check whether the message has already been processed, and avoid repeating the business operation.

For critical operations, use a unique constraint and transaction.

Q4. Why should the business operation and MessageId be in the same transaction?

Answer:

Otherwise the business operation could succeed and the application could crash before recording the MessageId. On redelivery, the consumer could perform the business operation again.

Q5. When should the consumer ACK the message?

Answer:

After the business operation and idempotency record have successfully committed.

Process
   ↓
Commit
   ↓
ACK

Q6. What happens if ACK fails after commit?

Answer:

The message may be redelivered. The consumer detects the already-processed MessageId and avoids repeating the business operation.

Q7. What is the difference between Inbox and Outbox?

Inbox Outbox
Consumer side Producer side
Handles received messages Handles outgoing events
Helps prevent duplicate effects Helps prevent lost publications
Supports idempotency Supports reliable publishing

Interview-Friendly Answer

An idempotent consumer is a message consumer that can safely receive the same message multiple times without repeating the business effect. It typically uses a unique MessageId or EventId and records processed messages. When the same message is redelivered, the consumer detects that it has already been processed and skips the duplicate operation. For critical processing, the business update and processed-message record should be committed in the same database transaction, and the broker message should be acknowledged only after that transaction succeeds. This is especially important in at-least-once messaging systems.

Keep Learning - More From This Category

← Back to Articles