Delivery and Subscription Model

Summary

Event Hub uses a scheduler-based delivery model to forward events from the outbox table to event bus topics. Event Hub maps each event type to its own dedicated event bus topic through a delivery channel. Event Hub serializes events in Protobuf format and ensures schema compatibility through Schema Registry. This structure allows consumers to subscribe only to the event types they care about and manage schema evolution safely.

Why It Exists

Sending all events to a single event bus topic creates two fundamental problems:

1. Consumer complexity — Consumers must filter out event types they are not interested in, creating unnecessary network and CPU overhead.
2. Schema management — Different event types use different Protobuf schemas. Managing multiple schemas in a single topic complicates compatibility checks in Schema Registry.

Event Hub solves these problems by routing each event type to its own topic. Consumers subscribe only to the topics they need, and each topic manages its own schema evolution rules independently.

Where It Fits in Keymate

The Delivery & Subscription model is the bridge between Event Hub's outbox table (write side) and downstream consumers (read side). It represents the detailed internal structure of Stage 3 (Forwarding to the Event Bus) described in the Overview page.

Boundaries

In scope:

• Event bus topic mapping structure and channel configuration
• Scheduler-based polling and delivery mechanism
• Protobuf serialization and Schema Registry integration
• Producer acknowledgment (acks) behavior

Out of scope:

• Outbox table data model and SQL queries → Outbox Replay & DLQ
• Consumer-side schema expectations and contract rules → Consumer Contracts
• Event acceptance flow via gRPC → Overview

How It Works

Topic Mapping Structure

Event Hub maps each event type to a dedicated event bus topic through a delivery channel. Event Hub follows a consistent one-to-one pattern: one event type, one channel, one event bus topic, and one scheduler entry.

When teams introduce a new event type, they replicate this same pattern — adding a new channel configuration, a new delivery pipeline, and a new scheduler entry.

note

Each event type maps to a dedicated event bus topic. Topic names follow a platform-defined naming convention.

Channel Configuration

Every event bus channel includes the following properties:

• Serialization: The channel serializes messages in Protobuf format with Schema Registry integration
• Durability: The producer waits for acknowledgment from all in-sync replicas, providing the strongest durability guarantee
• Schema Registry: Each channel connects to Schema Registry for schema registration and compatibility checks

Scheduler-Based Polling

An independent scheduler job runs for each event type at configurable intervals. Each job polls the outbox table for unprocessed records of its event type and triggers the delivery pipeline.

• Each event type has its own configurable scheduler interval
• A separate cleanup scheduler handles processed event deletion
• All scheduler jobs start with a short delay after application startup to allow initialization

Delivery Flow

Each event type's delivery pipeline follows the same flow:

1. Poll — The scheduler queries unprocessed records from the outbox table. A concurrency-safe mechanism ensures only one instance processes each record.
2. Deserialize — Event Hub converts each record's binary event data to a Protobuf message.
3. Send — Event Hub sends the event to the corresponding event bus topic. The eventId field serves as the message key.
4. ACK/NACK — The event bus broker acknowledges or rejects the message.
5. Status Update — Based on the result, Event Hub marks the record as PROCESSED (success) or FAILED (failure). Event Hub updates all records in the batch in a single transaction.

Protobuf Serialization

Event Hub serializes event bus message values in Protobuf format:

• Serializer: Protobuf serializer
• Schema Registry: The serializer registers each topic's schema in Schema Registry and automatically validates compatibility with the existing schema during send.
• Message key: The event's eventId (String) — the unique event identifier serves as the message key.
• Message value: The EventHubEvent Protobuf message — a single Protobuf message carries all event data (top-level fields + sub-event details).

Teams manage proto definitions through versioned external artifacts, published to an internal artifact registry.

warning

Schema changes require publishing new versions of the proto artifacts and updating dependency versions across Event Hub and all consumers.

Acknowledgment Behavior

Event Hub tracks message send results through an asynchronous ack/nack mechanism:

• ACK — When the event bus broker accepts the message, Event Hub marks the event as PROCESSED. Event Hub can optionally send additional audit logs on success.
• NACK — When the event bus broker rejects the message, Event Hub marks the event as FAILED. Event Hub logs the error but does not interrupt the flow.
• Connection error — If Event Hub cannot establish a connection to the event bus, it silently recovers the error. Event Hub marks the event as FAILED in the outbox and retries it in the next scheduler cycle.

The default configuration provides the strongest durability guarantee by requiring acknowledgment from all in-sync replicas. Teams can adjust this setting for scenarios requiring lower latency.

Diagram

Example Scenario

Scenario

An external system integration produces a change event, and Event Hub delivers it to the corresponding event bus topic.

Input

• Actor: Scheduler job for the corresponding event type
• Resource: Unprocessed outbox record
• Action: Poll, deserialize, and send to the event bus
• Context:
  • Channel: the mapped delivery channel
  • Topic: the corresponding event bus topic
  • Serialization: Protobuf with Schema Registry
  • Message key: the event's eventId value

Expected Outcome

1. The scheduler queries the outbox for unprocessed records of the event type
2. Event Hub deserializes the binary event data to a Protobuf message
3. Event Hub sends the message to the event bus topic with the event ID as the message key
4. The serializer registers the schema with Schema Registry
5. The event bus broker acknowledges the message
6. Event Hub updates the outbox record to PROCESSED

Common Misunderstandings

• "Event Hub sends each event type to a different event bus cluster" — All channels share the same event bus cluster. Separation is at the topic level, not the cluster level.

• "Scheduler period guarantees end-to-end latency" — The scheduler period is the polling interval, not an end-to-end latency guarantee. Actual latency depends on outbox query time, event bus send time, and acknowledgment wait time.

• "It works without Schema Registry" — The Protobuf serializer depends on Schema Registry for schema registration. If Schema Registry is unreachable, serialization fails and the event falls into FAILED status.

warning

Configure event bus broker and Schema Registry addresses per channel. In production environments, set these values via environment variables or Kubernetes ConfigMap/Secret — hardcoded values are for development environments only.

Design Notes / Best Practices

• Keep the default durability configuration — Event Hub's outbox pattern already provides at-least-once delivery. Combined with full replica acknowledgment, this minimizes data loss risk on the event bus side. Only consider lowering this for specific performance requirements.

• Manage schema evolution through proto artifacts — Teams centralize proto definitions in versioned artifacts and manage schema changes through versioning, guaranteeing schema compatibility between Event Hub and consumers.

• Monitor per channel — Each channel is an independent event bus producer instance. Monitor message send metrics, ack latencies, and error rates on a per-channel basis.

tip

The batch size configuration determines the maximum number of records pulled from the outbox per scheduler cycle. Tune this value to your workload — reducing it in low-volume environments decreases unnecessary query load, while increasing it in high-volume environments raises throughput.

Related Use Cases

• Adding a new event type: requires a new enum value, a new delivery pipeline, a new channel configuration, and a new event bus topic definition
• Differentiating scheduler periods per event type (more frequent polling for high-priority events)
• Configuring Schema Registry compatibility rules on a per-topic basis

Next Step

Continue with Outbox Replay & DLQ to understand the event lifecycle in the outbox table, retry mechanism, and cleanup strategy.

Related Docs

Overview

Event Hub's overall architecture and the Transactional Outbox pattern.

Outbox Replay & DLQ

Event statuses in the outbox table, FAILED record retries, and cleanup strategy.

Consumer Contracts

Protobuf schema rules and event format for event bus consumers.

Audit & Observability

Event Hub audit log integration and OpenTelemetry observability.

How many event bus topics does Event Hub use?

Event Hub uses a dedicated event bus topic for each event type. Adding new event types creates new topics.

What format are event bus messages serialized in?

Event Hub serializes messages in Protobuf format using a Protobuf serializer. The serializer registers schemas in Schema Registry and performs compatibility checks automatically. The message key is the event's unique ID (String), and the value is the EventHubEvent Protobuf message.

How do I change the scheduler polling interval?

Each event type has its own configurable scheduler period. Teams can adjust these values per event type via configuration properties or environment variables.