Event-Driven Distribution

Summary

Event-Driven Distribution is the routing layer that connects event ingestion to subscriber delivery in Integration Hub. When an event is ingested, the Event Processor resolves all active subscriptions, creates one OutboxEvent envelope per subscriber, and routes each envelope to a domain-specific outbox topic. This produces a fan-out effect: a single ingested event can generate multiple outbox messages across different domain topics, each targeting a different subscriber endpoint. Four domain outbox topics provide isolation for user lifecycle, access/session, policy/client, and organization event categories.

Why It Exists

Event distribution in Integration Hub faces two competing requirements:

1. Fan-out — A single platform event may be relevant to multiple subscribers, each needing its own delivery with unique endpoint and authentication
2. Domain isolation — Different event categories have different traffic patterns, failure modes, and SLA requirements

Domain-based topic routing satisfies both: the fan-out happens during envelope creation (one OutboxEvent per subscription), and domain isolation happens during topic routing (envelopes are placed on separate topics based on event category).

Where It Fits in Keymate

Event-Driven Distribution connects the Producer Model (which categorizes events and creates envelopes) with the Connector & Delivery Model (which consumes envelopes and delivers via HTTP). It is the "middle mile" of the Integration Hub pipeline.

Boundaries

In scope:

• Domain-based outbox topic structure
• Fan-out behavior (one OutboxEvent per subscription)
• DLT topic structure
• Topic isolation and independent scaling

Out of scope:

• Event categorization logic → Producer Model
• Subscription resolution → Consumer Model
• HTTP delivery → Connector & Delivery Model
• DLT processing → Connector Runtime

How It Works

Domain Topic Architecture

Integration Hub uses four domain outbox topics, each with a corresponding dead letter topic (DLT):

Domain	Purpose	Example Events
User Lifecycle	User CRUD, registration, password, role assignments	User created, role assigned, email verified
Access / Session	Authentication, token operations	Login, logout, token exchange, token refresh
Policy / Client	Client management, consent, grants	Client registered, consent granted, grant revoked
Organization	Organization membership, invitations	Org created, member invited, member unassigned

Fan-Out Mechanism

When an event is ingested, the Event Processor performs subscription-aware fan-out:

Key characteristics:

• One envelope per subscription — Each OutboxEvent contains a specific subscriber's delivery endpoint and authentication. If 3 subscribers match, 3 messages are published.
• Same domain topic — All envelopes for the same event category go to the same domain topic, regardless of which subscriber they target.
• Parallel creation — Envelopes are created in parallel for efficiency.
• Independent delivery — Each OutboxEvent is consumed and delivered independently by the Outbox Processor. A delivery failure to one subscriber does not affect delivery to others.

Topic Isolation Benefits

Domain-based topics provide several operational advantages:

Benefit	Description
Independent scaling	The Outbox Processor can allocate different consumer thread pools per topic based on traffic volume
Failure isolation	If all subscribers for one domain fail, only that domain's DLT fills up; other domains continue normally
Independent monitoring	Each topic/DLT pair can have its own alerts, dashboards, and SLAs
Selective processing	In degraded conditions, operators can pause consumption from one domain without affecting others

DLT Pairing

Each outbox topic has a paired DLT topic with the .DLT suffix. When the Outbox Processor fails to deliver a message after all retries, the failure strategy automatically routes the message to the corresponding DLT topic. The DLT Processor consumes all four DLT topics, persists failed events, and manages retry and replay.

Diagram

Example Scenario

Scenario

A policy change event triggers fan-out to two subscribers on the policy domain topic. One delivery succeeds; the other fails and routes to DLT.

Input

• Actor: Identity provider (policy change event)
• Resource: EventHubEvent with policy client update
• Action: Fan-out and delivery
• Context:
  • Event categorized as Policy / Client domain
  • Two active subscriptions: Subscriber A (OAuth2) and Subscriber B (API Key)

Expected Outcome

1. Event Processor resolves 2 subscriptions via gRPC
2. Two OutboxEvent envelopes are created and published to the Policy outbox topic
3. Outbox Processor delivers to Subscriber A → 200 OK → broker ACK
4. Outbox Processor delivers to Subscriber B → 503 → retries exhausted → broker NACK → DLT
5. DLT Processor persists the failed event for Subscriber B and schedules retry

Common Misunderstandings

• "One event goes to all four domain topics" — Each event is routed to exactly one domain topic based on its operation/resource type. The fan-out is at the subscription level (multiple envelopes on the same topic), not at the topic level.

• "Subscriber A's failure blocks Subscriber B" — Each OutboxEvent is an independent event bus message. Delivery failures for one subscriber do not affect other subscribers, even for the same original event.

• "DLT topics are shared across domains" — Each domain has its own DLT topic. Failed user lifecycle events go to the user lifecycle DLT, failed policy events go to the policy DLT, etc.

warning

The fan-out factor (number of OutboxEvent messages per ingested event) is determined by the number of matching subscriptions. If many subscribers register for the same event key, each ingested event generates a proportional number of outbox messages. Monitor subscription counts per event key to prevent unexpected topic volume.

Design Notes / Best Practices

• Monitor per-domain throughput — Track message rates and delivery latencies for each domain topic independently. This helps identify which domain is under stress and enables targeted scaling.

• Use DLT counts as health signals — A rising DLT message count for a specific domain often indicates a subscriber-side issue (endpoint down, auth expired). Set up alerts on DLT topic lag.

tip

The four-domain structure is defined by the event categorization configuration in the Event Processor. Adding a new domain requires creating a new outbox topic, a new DLT topic, a new delivery channel in the Event Processor and Outbox Processor, and updating the categorization logic.

Next Step

Continue with Connector & Delivery Model to learn how OutboxEvents are consumed and delivered to subscriber HTTP endpoints.

Related Docs

Producer Model

How events are categorized and envelopes are created.

Consumer Model

How subscriptions determine the fan-out factor.

Connector & Delivery Model

How OutboxEvents are consumed and delivered via HTTP.

Connector Runtime

DLT processing, retry scheduling, and operator replay.