Skip to main content

gRPC Ingestion Surface

Summary​

Integration Hub's Event Processor service accepts platform events through three ingestion channels: event bus consumption (primary), gRPC ingest, and REST ingest. The event bus channel consumes EventHubEvent messages from the upstream event topic. The gRPC endpoint accepts EventMessage payloads for direct event submission. The REST endpoint provides an HTTP-based alternative. Regardless of the ingestion channel, all events follow the same downstream processing path: subscription resolution, OutboxEvent envelope creation, and domain topic routing.

Why It Exists​

Platform events originate from multiple sources with different integration capabilities:

  1. Event-bus-native sources — Internal platform services that produce events to event bus topics (primary ingestion path via Event Hub)
  2. gRPC-capable services — Services that can make direct gRPC calls for real-time event submission with strong typing and schema validation
  3. HTTP-only services — External systems or legacy services that can only communicate via REST/HTTP

By supporting three ingestion channels, Integration Hub accommodates all producer types without requiring intermediary adapters.

Where It Fits in Keymate​

The ingestion surface is the entry point of the Integration Hub pipeline. It feeds events into the Event-Driven Distribution layer, which routes them to domain-specific outbox topics for delivery.

Boundaries​

In scope:

  • Event bus consumer configuration for upstream events
  • gRPC ingest endpoint and EventMessage proto
  • REST POST /api/events endpoint
  • Audit logging on event receipt

Out of scope:

How It Works​

Event Bus Ingestion (Primary Channel)​

The Event Processor consumes EventHubEvent Protobuf messages from the upstream event bus topic:

ConfigurationValue
ConnectorEvent bus connector
DeserializerProtobuf deserializer
Protobuf typeEventHubEvent
Offset strategyearliest

At startup, the Event Processor subscribes to the event bus stream and processes each event through the full pipeline: subscription resolution → envelope creation → topic routing.

gRPC Ingestion​

The Event Processor exposes a gRPC endpoint for direct event submission:

EventMessage proto
message EventMessage {
  string id = 1;              // Unique event identifier
  string source = 2;          // Source service name
  string type = 3;            // Event type identifier
  google.protobuf.Timestamp timestamp = 4;
  bytes raw_payload = 5;      // Raw event payload
  string trace_id = 6;        // OpenTelemetry trace ID
  string span_id = 7;         // OpenTelemetry span ID
}

The gRPC ingest supports OpenTelemetry context propagation via trace_id and span_id fields, enabling end-to-end distributed tracing from the event source through the Integration Hub pipeline.

REST Ingestion​

The Event Processor also exposes a REST endpoint for HTTP-based event submission:

POST /api/events
Content-Type: application/json
Body: EventHubEvent (JSON representation)

This endpoint accepts EventHubEvent messages in JSON format and feeds them into the same processing pipeline as event bus and gRPC ingested events.

Audit on Ingestion​

Every ingested event triggers an audit log entry via the Audit Collector gRPC service:

Audit ActionTrigger
EVENT_RECEIVEDEvent successfully ingested from any channel
EVENT_VALIDATION_ERROREvent failed validation
PAYLOAD_PARSE_ERROREvent deserialization failed

Diagram​

Example Scenario​

Scenario​

An external service submits an event directly via gRPC because it does not have event bus access.

Input​

  • Actor: External service with gRPC client
  • Resource: EventMessage with user update payload
  • Action: gRPC ingest call
  • Context:
    • id: "event-uuid-123"
    • source: "external-service"
    • type: "USER_UPDATE"
    • trace_id: OpenTelemetry trace context

Expected Outcome​

  1. Event Processor receives the EventMessage via gRPC
  2. An EVENT_RECEIVED audit log is sent to Audit Collector
  3. The event enters the standard processing pipeline (subscription resolution, envelope creation, topic routing)
  4. Active subscriptions matching the event key receive delivery via the downstream pipeline

Common Misunderstandings​

  • "gRPC ingest bypasses subscription resolution" — All three ingestion channels feed into the same processing pipeline. Subscription resolution, envelope creation, and topic routing apply regardless of the ingestion method.

  • "REST ingest is the recommended production path" — Event bus consumption is the primary and recommended ingestion channel. gRPC and REST ingest are alternative paths for sources that cannot produce directly to the event bus.

warning

The gRPC and REST ingest endpoints accept events directly into the processing pipeline without outbox persistence. If the Event Processor crashes between ingestion and topic routing, events received via gRPC or REST may be lost. For guaranteed delivery, use the event bus ingestion path, which benefits from built-in durability.

Design Notes / Best Practices​

  • Prefer event bus ingestion for reliability — The event bus provides built-in message persistence and consumer offset management. Use gRPC or REST ingest only when event bus access is not available to the event source.

  • Include OpenTelemetry context — When using gRPC ingest, populate trace_id and span_id to enable end-to-end distributed tracing through the Integration Hub pipeline.

tip

The Event Processor's HTTP and gRPC ports are configurable via application properties. In Kubernetes deployments, expose these ports via separate Service resources for independent scaling and access control.

Next Step​

Continue with Producer Model to learn how ingested events are categorized and routed to domain-specific outbox topics.

Overview

The four-service architecture and end-to-end event flow.

Producer Model

How events are mapped to domain topics after ingestion.

Event-Driven Distribution

Topic routing and fan-out mechanism.

Event Hub

The upstream component that produces events to the event bus.