Scatter Gather in Mule 4

Published on

May 7, 2021

Author

MuleSoft Integration Team

Scatter Gather Pattern helps MuleSoft developers distribute different tasks to achieve parallel processing of tasks/messages/events & then finally aggregate the responses as a single response.

The Scatter-Gather component receives a Mule event and sends a reference of this Mule event to each processing route.
Each of the processing routes starts executing in parallel then returns a Mule event. This is because the Scatter-Gather component executes each route in parallel, not sequentially.
After all processing routes have finished execution, the Scatter-Gather component creates a new Mule event that combines all resulting Mule events from each route. It then passes the new Mule event to the next component in the flow.

Scatter Gather Example:

In Scatter-Gather flow, we have three routes: Flow Reference Component(Flow Reference routes the Mule event to another flow or sub-flow). One set Payload Transformer(The Set Payload component lets you update the payload of the message).

Both sub-flow contains a Database select query operation which gives us data from the Database. This transform message component converts the data from the Database into JSON format and a logger which will track the logs.

1) Route 1 contains a Flow reference of a sub-flow (db-flow1)

Configuration of select query operation –

Configuration XML:

<sub-flow name="db-flow1" doc:id="bf4acf02-bd0d-4156-9980-6aad5b1e908b" >
		<db:select doc:name="Select" doc:id="38eff10a-1141-4b16-aeb4-c3ed78eeae66" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="d3590e43-a0c1-42e9-b6d5-562db1bec17d" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="342df3a5-b836-4a4c-a775-06bde82712d4" message="#[payload]"/>
	</sub-flow>

2) Route 2 contains a Flow reference of a sub flow (db-flow2)

Configuration of Select Query :

Configuration XML:

<sub-flow name="db-flow2" doc:id="f88987da-ea4c-44f5-bc39-0096d78eaa72" >
		<db:select doc:name="Select" doc:id="930efe51-8e73-40f2-9184-8174be3e9d01" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product1]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="ec45dbb2-92eb-43e2-836b-84029b0f4290" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="b100cf5e-809e-4d09-bfba-05f8114b748d" message="#[payload]"/>
	</sub-flow>

3) Route 3 contains a Set Payload component

Configuration of Set Payload:

Configuration XML:

<route >
				<set-payload value="scatter-gather flow payload" doc:name="Set Payload" doc:id="f8c0d364-044e-4e59-b31d-3fb4e8805466" />
				<logger level="INFO" doc:name="Logger" doc:id="62563f04-a79c-4cef-9ab9-5eea9e17637a" message="#[payload]"/>
			</route>

Tables in Database:

Product Table

2. Product1 Table

Flow XML:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core" xmlns:db="http://www.mulesoft.org/schema/mule/db"
	xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="dc87c934-3f63-4e54-a7a6-57b9852c6e2a" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>
	<db:config name="Database_Config1" doc:name="Database Config" doc:id="99af9304-0aa1-484d-986c-be9022aa3d21" >
		<db:my-sql-connection host="localhost" port="3306" user="root" password="Caelius@123" database="muletrainingdb" />
	</db:config>
	<flow name="scatter-gather" doc:id="b7ea3fed-f402-4359-8a0d-c7a952903010" >
		<http:listener doc:name="Listener" doc:id="8e71879a-1711-4c3d-b4a1-1effa23cdec0" config-ref="HTTP_Listener_config" path="/test"/>
		<scatter-gather doc:name="Scatter-Gather" doc:id="f79ea679-6ffc-432c-a8e5-c56c9a6ff025" >
			<route >
				<flow-ref doc:name="db-flow1" doc:id="da378a2d-6584-4682-be11-746cb1b443a6" name="db-flow1"/>
			</route>
			<route >
				<flow-ref doc:name="db-flow2" doc:id="765de27e-c4ac-4c0b-bced-58f49f2bc460" name="db-flow2"/>
			</route>
			<route >
				<set-payload value="scatter-gather flow payload" doc:name="Set Payload" doc:id="f8c0d364-044e-4e59-b31d-3fb4e8805466" />
				<logger level="INFO" doc:name="Logger" doc:id="62563f04-a79c-4cef-9ab9-5eea9e17637a" message="#[payload]"/>
			</route>
		</scatter-gather>
		<ee:transform doc:name="Transform Message" doc:id="df9303a4-c1dd-4eec-a731-a9358859ed18">
					<ee:message>
						<ee:set-payload><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
					</ee:message>
				</ee:transform>
	</flow>
	<sub-flow name="db-flow1" doc:id="bf4acf02-bd0d-4156-9980-6aad5b1e908b" >
		<db:select doc:name="Select" doc:id="38eff10a-1141-4b16-aeb4-c3ed78eeae66" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="d3590e43-a0c1-42e9-b6d5-562db1bec17d" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="342df3a5-b836-4a4c-a775-06bde82712d4" message="#[payload]"/>
	</sub-flow>
	<sub-flow name="db-flow2" doc:id="f88987da-ea4c-44f5-bc39-0096d78eaa72" >
		<db:select doc:name="Select" doc:id="930efe51-8e73-40f2-9184-8174be3e9d01" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product1]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="ec45dbb2-92eb-43e2-836b-84029b0f4290" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="b100cf5e-809e-4d09-bfba-05f8114b748d" message="#[payload]"/>
	</sub-flow>
</mule>

Output Combining all routes:

  {
    "0": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": [
            {
                "original_price": 1000.0,
                "product_id": 1,
                "name": "Hp Pavilion laptop",
                "offer_valid_until": "2016-06-27T10:45:56",
                "description": "Hp Laptop ",
                "brand_name": "HP",
                "offer_price": 1000.0
            },
            {
                "original_price": 3000.0,
                "product_id": 2,
                "name": "Macbook Pro laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Apple Laptop ",
                "brand_name": "Apple",
                "offer_price": 3000.0
            },
            {
                "original_price": 2000.0,
                "product_id": 3,
                "name": "Mac Book  laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Apple Laptop ",
                "brand_name": "Apple",
                "offer_price": 2000.0
            },
            {
                "original_price": 4000.0,
                "product_id": 4,
                "name": "IBM laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "IBM Laptop ",
                "brand_name": "IBM",
                "offer_price": 4000.0
            },
            {
                "original_price": 1000.0,
                "product_id": 5,
                "name": "MotoX Mobile",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Motorola Mobile ",
                "brand_name": "Motorola",
                "offer_price": 1000.0
            },
            {
                "original_price": 5000.0,
                "product_id": 6,
                "name": "Samsung Note 5",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Samsung Mobile ",
                "brand_name": "Samsung",
                "offer_price": 5000.0
            }
        ],
        "outboundPropertyNames": [],
        "attributes": null
    },
    "1": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": [
            {
                "Name": "hp",
                "id": 1001
            },
            {
                "Name": "asus",
                "id": 1002
            },
            {
                "Name": "dell",
                "id": 1003
            },
            {
                "Name": "apple",
                "id": 1004
            }
        ],
        "outboundPropertyNames": [],
        "attributes": null
    },
    "2": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": "scatter-gather flow payload",
        "outboundPropertyNames": [],
        "attributes": {
            "headers": {
                "user-agent": "PostmanRuntime/7.28.0",
                "accept": "*/*",
                "postman-token": "01f2bd2d-5eaa-4d9f-bd6a-164ab590be81",
                "host": "localhost:8081",
                "accept-encoding": "gzip, deflate, br",
                "connection": "keep-alive"
            },
            "clientCertificate": null,
            "method": "GET",
            "scheme": "http",
            "queryParams": {},
            "requestUri": "/test",
            "queryString": "",
            "version": "HTTP/1.1",
            "maskedRequestPath": null,
            "listenerPath": "/test",
            "localAddress": "/127.0.0.1:8081",
            "relativePath": "/test",
            "uriParams": {},
            "rawRequestUri": "/test",
            "rawRequestPath": "/test",
            "remoteAddress": "/127.0.0.1:54523",
            "requestPath": "/test"
        }
    }
}

All the MuleSoft Anypoint platform users can use this guide to implement scatter-gather pattern in their Mule projects.

Find more MuleSoft technical guides at Caelius Consulting Resource Centre.

Recent Blogs

BlogFeb 11, 2026

Designing for Reality: Integrating 837 Claims When X12 Meets Production

Designing 837 Claim Integration for Real-World Healthcare Systems When it comes to 837 claim integration, most architects assume the X12 specification guarantees predictability. On paper, the 837 Professional, Institutional, and Dental transactions look clean and orderly. In production? Not even close. Real-world 837 files behave differently across trading partners. Loops appear conditionally. Repeatable segments shift… Continue reading Designing for Reality: Integrating 837 Claims When X12 Meets Production

Blog

5 min read

Designing for Reality: Integrating 837 Claims When X12 Meets Production

BlogDec 4, 2025

AI-Driven PDF Parsing in Salesforce

Introduction For the current digital ecosystem, data is an important aspect for decision-making. Yet, for many organizations, a significant portion of this valuable data remains locked away in unstructured formats. Organizations handle thousands of PDF documents daily — ranging from contracts and invoices to lab reports, quotations, and service agreements. Traditionally, extracting structured data from… Continue reading AI-Driven PDF Parsing in Salesforce

Blog

6 min read

AI-Driven PDF Parsing in Salesforce

BlogNov 5, 2025

Compression Namespace in Apex: A Powerful New Salesforce Feature

Introduction Working with documents inside Salesforce has always challenged developers because of the platform’s multitenant constraints. Previously, packaging and sending files in a compact form required external services, like an AWS Lambda function, that retrieved files via API and then compressed them. With the introduction of the Compression Namespace and the powerful pre-defined Apex functions,… Continue reading Compression Namespace in Apex: A Powerful New Salesforce Feature

Blog

5 min read

Compression Namespace in Apex: A Powerful New Salesforce Feature

BlogSep 18, 2025

Boost LWC Performance with Debouncing

Introduction Lightning Web Components (LWC) is a modern framework for building fast and dynamic user interfaces on the Salesforce platform. However, one common challenge in web development, including LWC, is efficiently handling user input, especially when dealing with rapid or repetitive events, such as typing in a search field. This is where debouncing becomes an… Continue reading Boost LWC Performance with Debouncing

Blog

7 min read