Splitter

Tags

"How can we process a message if it contains multiple elements, each of which may have to be processed in a different way?" — Hohpe & Woolf, Enterprise Integration Patterns, 2003

Intent

The Splitter decomposes a composite message into a sequence of individual messages, each containing one element of the original. This enables parallel processing of message parts by independent downstream consumers — each part flows through the pipeline as an autonomous unit.

The Splitter is the inverse of the Aggregator: the Splitter fans out one composite message into N individual messages; the Aggregator reassembles N related messages into one. Together they form the canonical Scatter-Gather workflow. The Splitter initiates the scatter; the Aggregator closes the gather.

A critical requirement: every split message must carry a correlationId (set to the parent message's ID), a splitIndex (the element's 0-based position), and totalParts (the total number of split messages). Without correlationId, downstream consumers cannot associate parts with their parent, and an Aggregator cannot detect completeness. Without totalParts, the Aggregator has no size-based completeness condition and cannot know when to release. Omitting these headers is the primary Splitter failure mode.

When NOT to Use

  • When the composite message must be processed atomically — splitting breaks transactional atomicity across the parts; use a batch processor that handles all elements in a single transaction instead.
  • When split parts have no shared processing step and the fan-out serves no coordination purpose — splitting adds routing overhead without benefit; iterate the collection in-process with an Iterator-Pattern.
  • When the number of split parts is unbounded and could overwhelm downstream consumers — apply back-pressure, rate limiting, or a bounded work queue before splitting; unbounded fan-out is a denial-of-service risk.

When to Use

  • Order line processing: one order with N line items → N individual line messages, each processed in parallel by an inventory or pricing service.
  • Bulk event replay: a batch payload of M domain events → M individual event messages, each routed to its appropriate handler.
  • Scatter-Gather: split a composite request to N services, process in parallel, then reassemble all N responses with a downstream Aggregator.

How It Works

The Splitter receives a composite message, iterates over its elements, and publishes one output message per element to the downstream channel. Each output message carries three required metadata fields:

  1. correlationId — set to the parent message's ID. Required for the downstream Aggregator to group parts into the same bucket.
  2. splitIndex — the element's position (0-based) within the original composite. Enables ordering in the assembled result.
  3. totalParts — the total number of split messages produced. Enables size-based completeness detection in the Aggregator: when receivedCount == totalParts, the bucket is complete.

The Splitter itself is stateless — all correlation tracking is encoded in the message headers. Downstream consumers can process split messages in any order; the Aggregator handles sequencing and reassembly. The statelessness of the Splitter makes it horizontally scalable: multiple Splitter instances can run in parallel without coordination.

Flow Diagram

flowchart TD
    IN["Composite Message<br/>[Item A, Item B, Item C]"]

    subgraph Splitter
        SP["Split by element"]
    end

    IN --> SP

    SP --> M1["Message 1<br/>Item A<br/>correlationId=parent<br/>splitIndex=0<br/>totalParts=3"]
    SP --> M2["Message 2<br/>Item B<br/>correlationId=parent<br/>splitIndex=1<br/>totalParts=3"]
    SP --> M3["Message 3<br/>Item C<br/>correlationId=parent<br/>splitIndex=2<br/>totalParts=3"]

    M1 --> CH["Downstream Channel"]
    M2 --> CH
    M3 --> CH

    style IN fill:#e1f5fe,stroke:#0288d1
    style SP fill:#fff3e0,stroke:#f57c00

TypeScript Example

// Splitter — TypeScript (split composite message into individual messages)
// Source: Hohpe & Woolf, Enterprise Integration Patterns, 2003
interface OrderMessage { orderId: string; lines: { sku: string; qty: number }[]; }
interface LineMessage  { correlationId: string; splitIndex: number; totalParts: number; sku: string; qty: number; }
 
function split(order: OrderMessage): LineMessage[] {
  return order.lines.map((line, i) => ({
    correlationId: order.orderId,   // carry parent ID for downstream Aggregator
    splitIndex: i,
    totalParts: order.lines.length,
    ...line
  }));
}

Java Example

// Splitter — Apache Camel Java DSL
// Source: camel.apache.org/components/4.10.x/eips/split-eip.html
from("direct:order")
    .split(body().tokenize(","))     // split CSV body into individual messages
    // .parallelProcessing()         // optional: process split messages concurrently
    .to("direct:processLine");
// Each sub-message is routed independently
// Use the Aggregator pattern (aggregate-eip) to reassemble results

Note: .parallelProcessing() is shown as a comment variant — the default is serial processing, which is simpler to reason about for a first exposition. Parallel processing requires thread-safe consumers downstream.

Lineage Backward

  • Iterator-Pattern — Iterator traverses elements of a collection in-process within a single execution context. The Splitter extends this concept to message-passing: it publishes each element as an independent message to a channel, enabling parallel out-of-process processing by independent consumers.

Lineage Forward

  • Aggregator — Splitter and Aggregator are the canonical inverse pair. The Scatter-Gather workflow: Splitter fans out one composite message into N individual messages → parallel processing by N consumers → Aggregator reassembles N results into one combined message. A Splitter without a downstream Aggregator produces messages that cannot be coordinated back into a unified result.
PatternRelationship
AggregatorInverse — Scatter-Gather pair
Iterator-PatternIn-process iteration ancestor
Pipes-and-FiltersSerial processing chain vs parallel fan-out
Message-RouterRoutes to one channel vs publishes N messages

Sources

Linked mentions