Connector & Delivery Model

Summary

The Connector & Delivery Model defines how Integration Hub's Outbox Processor delivers events to subscriber HTTP endpoints. The Outbox Processor consumes OutboxEvent messages from the four domain outbox topics, extracts the delivery endpoint and authentication credentials from the envelope, and makes authenticated HTTP POST calls to each subscriber. The delivery client supports multiple authentication types (Basic, OAuth2, JWT, API Key) and payload formats (JSON, binary). Successful deliveries are acknowledged in the event bus; failures trigger retries and, ultimately, dead letter routing.

Why It Exists

Raw event bus messages cannot reach subscribers that expect HTTP callbacks. The Connector & Delivery Model bridges this gap by:

1. Abstracting event bus complexity — Subscribers do not need event bus clients, Schema Registry access, or Protobuf deserialization
2. Handling authentication — Each subscriber can have different authentication requirements; the delivery model resolves credentials per subscription
3. Providing delivery feedback — HTTP responses (2xx, 4xx, 5xx, timeouts) provide immediate delivery status, enabling retry and DLT routing decisions

Where It Fits in Keymate

The Connector & Delivery Model is the downstream execution layer of the Integration Hub pipeline. It consumes OutboxEvent envelopes produced by the Producer Model and delivers them to subscriber endpoints registered via the Consumer Model. Failed deliveries flow into the Connector Runtime for retry and DLT processing.

Boundaries

In scope:

• OutboxEvent consumption from domain outbox topics
• HTTP delivery with authentication
• Payload format handling (JSON, binary)
• Delivery result classification and broker ACK/NACK
• Error code mapping for connection failures

Out of scope:

• Subscription resolution and topic routing → Producer Model
• Retry scheduling and DLT persistence → Connector Runtime
• Subscription registration → Consumer Model

How It Works

Outbox Consumption

The Outbox Processor merges four domain outbox topic streams into a single processing pipeline:

Consumer Configuration	Value
Deserializer	Protobuf deserializer
Protobuf type	OutboxEvent
Offset strategy	earliest
Commit mode	Manual ACK/NACK
Failure strategy	Dead letter queue routing

Each consumed message is parsed as an OutboxEvent and passed to the delivery pipeline.

HTTP Delivery Pipeline

1. Parse — The OutboxEvent envelope is deserialized. The event, endpoint, protocol, and auth fields are extracted.
2. Build — A delivery request is constructed from the envelope, containing the target URL, authentication model, and payload.
3. Authenticate — The delivery client resolves the authentication type and prepares the HTTP request accordingly.
4. Send — An asynchronous HTTP POST is made to the subscriber endpoint with the configured timeout.
5. Result — The HTTP response status determines the outcome:
   • 2xx → Success: message is ACKed, EXECUTED audit log recorded
   • 4xx/5xx/timeout → Failure: retry up to the configured maximum (default: 3), then NACK

Authentication Handling

The delivery client supports the following authentication models:

Auth Type	HTTP Behavior
BASIC_AUTH	Sets Authorization: Basic <base64(username:password)> header
OAUTH	Performs OAuth2 token exchange (supports client_credentials, password flows), then sets Authorization: Bearer <token>
HEADER_TOKEN / JWT_HEADER	Sets Authorization: Bearer <token> with a pre-shared token
QUERY_PARAMETER	Appends the token as a URL query parameter

Payload Formats

The delivery payload format is determined by the subscription's payload_type:

PayloadType	Behavior
PAYLOAD_TYPE_JSON	Event is delivered as a JSON body with Content-Type: application/json
Binary / Protobuf	Event is delivered as raw bytes

Error Code Mapping

The delivery client maps connection-level errors to HTTP status codes for consistent error handling:

Error Type	Mapped HTTP Status	Description
DNS resolution failure	502 (Bad Gateway)	Subscriber hostname cannot be resolved
Connection refused	503 (Service Unavailable)	Subscriber endpoint is not accepting connections
Request timeout	504 (Gateway Timeout)	Subscriber did not respond within the configured timeout
SSL/TLS error	525 (SSL Handshake Failed)	TLS negotiation failed

Configuration

Property	Default	Description
Delivery timeout	1s (Outbox Processor) / 5s (DLT Processor)	HTTP delivery timeout
Retry count	3	Max delivery retries before DLT routing

Diagram

Example Scenario

Scenario

An OutboxEvent containing a user creation event is consumed from the user lifecycle outbox topic and delivered to a subscriber with OAuth2 authentication.

Input

• Actor: Outbox Processor
• Resource: OutboxEvent from the user lifecycle outbox topic
• Action: HTTP POST to subscriber endpoint
• Context:
  • Endpoint: https://my-service.example.com/webhooks/users
  • Auth type: OAuth2 (client_id, client_secret, token_url)
  • Payload type: JSON
  • Timeout: 1s

Expected Outcome

1. Outbox Processor parses the OutboxEvent envelope
2. OAuth2 token exchange: POST to token_url with client credentials → receives access token
3. HTTP POST to https://my-service.example.com/webhooks/users with Authorization: Bearer <token> and JSON body
4. Subscriber responds with 200 OK
5. Message ACKed, EXECUTED audit log recorded

Common Misunderstandings

• "The Outbox Processor transforms the event payload" — The Outbox Processor delivers the event payload as-is from the OutboxEvent envelope. No content transformation occurs.

• "OAuth2 tokens are cached indefinitely" — Token exchange occurs per delivery attempt. For high-volume scenarios, consider subscriber-side token caching strategies.

• "All subscribers receive the same auth type" — Each subscription has its own auth configuration. Different subscribers can use different authentication types for the same event type.

warning

The delivery timeout defaults to 1 second in the Outbox Processor. Subscriber endpoints must respond within this window, or the delivery is treated as failed. Increase this value if subscribers perform significant processing before responding.

Design Notes / Best Practices

• Respond quickly, process asynchronously — Subscriber endpoints should acknowledge receipt (return 2xx) immediately and process the event asynchronously. Long processing times risk delivery timeouts and unnecessary retries.

• Return appropriate HTTP status codes — Return 4xx for permanent errors (bad request, unauthorized) and 5xx for transient errors. The Outbox Processor retries on all non-2xx responses, so accurate status codes help operators diagnose issues.

tip

The four domain outbox topics provide natural isolation for delivery processing. If one domain's subscribers are slow or failing, it does not affect delivery to subscribers of other domains.

Next Step

Continue with Connector Runtime to understand the retry strategy, dead letter processing, and operator replay API for failed deliveries.

Related Docs

Overview

The four-service architecture and end-to-end event flow.

Producer Model

How OutboxEvent envelopes are created and routed to domain topics.

Consumer Model

How subscriptions and auth credentials are registered.

Connector Runtime

Retry strategy, DLT processing, and operator replay API.