Scatter Gather in Mule 4

Published on

May 7, 2021

Author

MuleSoft Integration Team

Scatter Gather Pattern helps MuleSoft developers distribute different tasks to achieve parallel processing of tasks/messages/events & then finally aggregate the responses as a single response.

Scatter Gather Component

The Scatter-Gather component receives a Mule event and sends a reference of this Mule event to each processing route.
Each of the processing routes starts executing in parallel then returns a Mule event. This is because the Scatter-Gather component executes each route in parallel, not sequentially.
After all processing routes have finished execution, the Scatter-Gather component creates a new Mule event that combines all resulting Mule events from each route. It then passes the new Mule event to the next component in the flow.

Scatter Gather Example:

In Scatter-Gather flow, we have three routes: Flow Reference Component(Flow Reference routes the Mule event to another flow or sub-flow). One set Payload Transformer(The Set Payload component lets you update the payload of the message).

Scatter-Gather Flow

Both sub-flow contains a Database select query operation which gives us data from the Database. This transform message component converts the data from the Database into JSON format and a logger which will track the logs.

1) Route 1 contains a Flow reference of a sub-flow (db-flow1)

Flow refrence (db-flow1)

Configuration of select query operation –

Configuration XML:

<sub-flow name="db-flow1" doc:id="bf4acf02-bd0d-4156-9980-6aad5b1e908b" >
		<db:select doc:name="Select" doc:id="38eff10a-1141-4b16-aeb4-c3ed78eeae66" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="d3590e43-a0c1-42e9-b6d5-562db1bec17d" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="342df3a5-b836-4a4c-a775-06bde82712d4" message="#[payload]"/>
	</sub-flow>

2) Route 2 contains a Flow reference of a sub flow (db-flow2)

Flow refrence (db-flow2)

Configuration of Select Query :

Configuration of select query for db-flow2

Configuration XML:

<sub-flow name="db-flow2" doc:id="f88987da-ea4c-44f5-bc39-0096d78eaa72" >
		<db:select doc:name="Select" doc:id="930efe51-8e73-40f2-9184-8174be3e9d01" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product1]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="ec45dbb2-92eb-43e2-836b-84029b0f4290" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="b100cf5e-809e-4d09-bfba-05f8114b748d" message="#[payload]"/>
	</sub-flow>

3) Route 3 contains a Set Payload component

Route 3 (Set Payload)

Configuration of Set Payload:

Configuration of Set Payload

Configuration XML:

<route >
				<set-payload value="scatter-gather flow payload" doc:name="Set Payload" doc:id="f8c0d364-044e-4e59-b31d-3fb4e8805466" />
				<logger level="INFO" doc:name="Logger" doc:id="62563f04-a79c-4cef-9ab9-5eea9e17637a" message="#[payload]"/>
			</route>

Tables in Database:

Product Table

2. Product1 Table

Product1 Table

Flow XML:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core" xmlns:db="http://www.mulesoft.org/schema/mule/db"
	xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="dc87c934-3f63-4e54-a7a6-57b9852c6e2a" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>
	<db:config name="Database_Config1" doc:name="Database Config" doc:id="99af9304-0aa1-484d-986c-be9022aa3d21" >
		<db:my-sql-connection host="localhost" port="3306" user="root" password="Caelius@123" database="muletrainingdb" />
	</db:config>
	<flow name="scatter-gather" doc:id="b7ea3fed-f402-4359-8a0d-c7a952903010" >
		<http:listener doc:name="Listener" doc:id="8e71879a-1711-4c3d-b4a1-1effa23cdec0" config-ref="HTTP_Listener_config" path="/test"/>
		<scatter-gather doc:name="Scatter-Gather" doc:id="f79ea679-6ffc-432c-a8e5-c56c9a6ff025" >
			<route >
				<flow-ref doc:name="db-flow1" doc:id="da378a2d-6584-4682-be11-746cb1b443a6" name="db-flow1"/>
			</route>
			<route >
				<flow-ref doc:name="db-flow2" doc:id="765de27e-c4ac-4c0b-bced-58f49f2bc460" name="db-flow2"/>
			</route>
			<route >
				<set-payload value="scatter-gather flow payload" doc:name="Set Payload" doc:id="f8c0d364-044e-4e59-b31d-3fb4e8805466" />
				<logger level="INFO" doc:name="Logger" doc:id="62563f04-a79c-4cef-9ab9-5eea9e17637a" message="#[payload]"/>
			</route>
		</scatter-gather>
		<ee:transform doc:name="Transform Message" doc:id="df9303a4-c1dd-4eec-a731-a9358859ed18">
					<ee:message>
						<ee:set-payload><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
					</ee:message>
				</ee:transform>
	</flow>
	<sub-flow name="db-flow1" doc:id="bf4acf02-bd0d-4156-9980-6aad5b1e908b" >
		<db:select doc:name="Select" doc:id="38eff10a-1141-4b16-aeb4-c3ed78eeae66" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="d3590e43-a0c1-42e9-b6d5-562db1bec17d" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="342df3a5-b836-4a4c-a775-06bde82712d4" message="#[payload]"/>
	</sub-flow>
	<sub-flow name="db-flow2" doc:id="f88987da-ea4c-44f5-bc39-0096d78eaa72" >
		<db:select doc:name="Select" doc:id="930efe51-8e73-40f2-9184-8174be3e9d01" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product1]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="ec45dbb2-92eb-43e2-836b-84029b0f4290" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="b100cf5e-809e-4d09-bfba-05f8114b748d" message="#[payload]"/>
	</sub-flow>
</mule>

Output Combining all routes:

  {
    "0": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": [
            {
                "original_price": 1000.0,
                "product_id": 1,
                "name": "Hp Pavilion laptop",
                "offer_valid_until": "2016-06-27T10:45:56",
                "description": "Hp Laptop ",
                "brand_name": "HP",
                "offer_price": 1000.0
            },
            {
                "original_price": 3000.0,
                "product_id": 2,
                "name": "Macbook Pro laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Apple Laptop ",
                "brand_name": "Apple",
                "offer_price": 3000.0
            },
            {
                "original_price": 2000.0,
                "product_id": 3,
                "name": "Mac Book  laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Apple Laptop ",
                "brand_name": "Apple",
                "offer_price": 2000.0
            },
            {
                "original_price": 4000.0,
                "product_id": 4,
                "name": "IBM laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "IBM Laptop ",
                "brand_name": "IBM",
                "offer_price": 4000.0
            },
            {
                "original_price": 1000.0,
                "product_id": 5,
                "name": "MotoX Mobile",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Motorola Mobile ",
                "brand_name": "Motorola",
                "offer_price": 1000.0
            },
            {
                "original_price": 5000.0,
                "product_id": 6,
                "name": "Samsung Note 5",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Samsung Mobile ",
                "brand_name": "Samsung",
                "offer_price": 5000.0
            }
        ],
        "outboundPropertyNames": [],
        "attributes": null
    },
    "1": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": [
            {
                "Name": "hp",
                "id": 1001
            },
            {
                "Name": "asus",
                "id": 1002
            },
            {
                "Name": "dell",
                "id": 1003
            },
            {
                "Name": "apple",
                "id": 1004
            }
        ],
        "outboundPropertyNames": [],
        "attributes": null
    },
    "2": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": "scatter-gather flow payload",
        "outboundPropertyNames": [],
        "attributes": {
            "headers": {
                "user-agent": "PostmanRuntime/7.28.0",
                "accept": "*/*",
                "postman-token": "01f2bd2d-5eaa-4d9f-bd6a-164ab590be81",
                "host": "localhost:8081",
                "accept-encoding": "gzip, deflate, br",
                "connection": "keep-alive"
            },
            "clientCertificate": null,
            "method": "GET",
            "scheme": "http",
            "queryParams": {},
            "requestUri": "/test",
            "queryString": "",
            "version": "HTTP/1.1",
            "maskedRequestPath": null,
            "listenerPath": "/test",
            "localAddress": "/127.0.0.1:8081",
            "relativePath": "/test",
            "uriParams": {},
            "rawRequestUri": "/test",
            "rawRequestPath": "/test",
            "remoteAddress": "/127.0.0.1:54523",
            "requestPath": "/test"
        }
    }
}

All the MuleSoft Anypoint platform users can use this guide to implement scatter-gather pattern in their Mule projects.

Find more MuleSoft technical guides at Caelius Consulting Resource Centre.

Recent Blogs

BlogSep 9, 2025

Salesforce Pricing Automation: Boost Efficiency And Accuracy with Apex Triggers

Introduction In order to succeed in today’s fast-paced business landscape, precision and speed define competitive advantage. For businesses, especially those managing complex product catalogs, ensuring accurate pricing on sales orders or custom lines can be a time-consuming and error-prone task. To overcome this challenge, Salesforce trigger handlers offer a powerful solution to automate the entire… Continue reading Salesforce Pricing Automation: Boost Efficiency And Accuracy with Apex Triggers

Blog

6 min read

Salesforce Pricing Automation: Boost Efficiency And Accuracy with Apex Triggers

BlogJul 14, 2025

Connecting MuleSoft and Azure SQL with Entra ID

Introduction Establishing a secure connection between MuleSoft and Azure SQL Database can be challenging, especially if you are using Entra ID (formerly known as Azure Active Directory) for authentication. This blog walks through a fully working configuration for connecting to Azure SQL using ActiveDirectoryServicePrincipal in Mule runtime 4.7.4 with Java 8 — addressing driver setup,… Continue reading Connecting MuleSoft and Azure SQL with Entra ID

Blog

2 min read

Connecting MuleSoft and Azure SQL with Entra ID

BlogJun 30, 2025

Understanding Salesforce Flow Approval Processes

Introduction: Salesforce introduced Flow Approval Processes in the Spring '25 release. This is an evolved version of the classic approval process model, powered by Flow Orchestrator. The new approach brings unprecedented flexibility, enabling the creation of dynamic, multi-level, and logic-driven approval workflows that are entirely declarative. Continue reading the blog to get a deeper understanding… Continue reading Understanding Salesforce Flow Approval Processes

Blog

5 min read

Understanding Salesforce Flow Approval Processes

BlogMay 14, 2025

Capturing Real-time Record Updation Using LWC

Introduction In modern CRM ecosystems, real-time Salesforce integration and seamless user experiences are no longer optional but fundamental for driving operational efficiency. Imagine your sales reps making important Opportunity changes, but the ERP remains out of sync, leading to confusion and data errors. We understood the necessity to bridge this data gap and implemented a… Continue reading Capturing Real-time Record Updation Using LWC

Blog

5 min read