Is a transactional outbox needed for event-sourced systems?

user_2946580 0
Viewed 217

I'm talking about the transactional outbox patterns, as described here: https://microservices.io/patterns/data/transactional-outbox.html

As far as I understand, a transactional outbox is needed if you update state and publish a corresponding event in one transaction.

But what about event-sourced systems, where there is no state update, just publishing events? Do you even need a transactional outbox pattern?

For example if I use AWS EventBridge as an event bus. Sending events to the bus either succeed or fail. Once events are on the bus, they can be handled in a resilient, fault-tolerant way. So would a transactional outbox needed in this case?

event-sourcing cqrs domain-driven-design
2 Answers

Most event sourcing frameworks work effectively as outboxes. When an entity publishes an event, that event gets saved in the DB (or event store) and the event processor picks that whenever is needed.

So answering your question: I would say no, as the event sourcing framework is a sort of very advanced outbox.

Just in case, the above works only when the services persist only to one storage engine (DB , Kafka, whatever) in a transactional way. If you have to interact with multiple storage services, things can become a lot more complicated depending on the semantics of the messaging system. If the messaging system can deal with a send-at-least-once semantics, there's no need for an outbox, but any consumer will need to be able to deal with duplicated messages.

user_6088200

But what about event-sourced systems, where there is no state update, just publishing events? Do you even need a transactional outbox pattern?

In the context of CQRS, "event sourcing" usually means that you have some persisted representation of history (aka, there is a durable copy of the "event stream" in some storage appliance).

So there is still a "state update" - because when we flush everything out of transient memory we still want to have a copy of the events to rebuild from (we need that, because the events we have already emitted are an input that we use when computing new events).

If "saving events to use later" and "publishing events" are two different transactions, then you need to consider failure modes: what happens if you crash between the two transactions?

The motivation for transactional outbox is something like "if we can't guarantee that the publish will eventually happen, we don't want to save the state either." Save and publish becomes (sort of) an all or nothing operation.

Note: as a rule, event buses don't support the kinds of first-writer-wins guarantees to be used as an event store; it's not a good mechanism for "save events to use later" because you lose the ability to ensure that events in the stream don't contradict each other. Makes it difficult to design a robust system.

The happy path isn't the problem.

user_547340
Related