Object Store Connector In Mule 4

What is an Object Store?

Object Store in Mule 4 allows you to save data as a key-value pair and access it later for any transformation or condition logic. The Object Store is primarily used to store watermarks, access tokens, and user information as a key-value pair. We can utilise the pre-built Object Store connector to access the Object Store within the same project.

Interesting, right? Well, there’s more!
The Object Store connector supports seven different types of operations. They are

1. Clear: Deletes/Removes all the contents in the Object Store.
2. Contains: Returns a Boolean value whether the key is present or not.
3. Remove: This operation removes the value associated with a given key
4. Retrieve: Returns the data stored for the given key.
5. Retrieve all: Returns all the key-value pairs from the Object Store.
6. Retrieve all keys: Retrieves a list containing all keys currently held by the Object Store.
7. Store: Uses the given key to store the given value.

Configuring Global Object Store:

Step 1: Search for Object Store under component configuration by navigating to Global Element. 

Step 2: Configure the connector as follows:

Didn’t understand the image? Let’s dig into the above configuration

The first thing you notice is that Persistent is checked, which implies that your data will be kept in the Disk and will be persistent – you will not lose your data if the program stops or crashes! If you do not select Persistent, your data will be saved in-memory and you will lose it if the app stops or crashes. Persistent storage should always be used.

Moving on! 

• Max Entries: The number of entries that can be stored. For example, imagine you are watching your favourite movie in a theatre. You will be unable to book more seats after all of the available seats have been filled. In other words, if the store grows to more than 10 entries, the exceeding items will be discarded when the expiration thread runs.
• Entrance TTL: TTL stands for TIME TO LIVE. As the name suggests it’s entry time out. Referring to the above config., TTL of 8 hours means every entry lives for 8 hours.
• Expiration Interval: This is the frequency at which the expiration thread shall run. An expiration interval of 1 hour means the thread runs every hour and discards the elements that exceed their time to live (TTL).
• Units: These basically refer to the units of time such as seconds, hours, etc. Note: Be mindful of the units used for each attribute.
• Configuration Reference: To extend a default object store, such as Redis, you must configure a Private object store.

Concluding the configuration the object store can store a max of 10 entries, and the time to live is 8 hours. As the Expiration interval is 1 hour, all the values inside the object store expire, in other words are discarded after 1 hour after they exceed their TTL. Possible operations now include;

1. Store

Store operation is used to store data in key-value pair. We create a sample flow using a HTTP listener, a store component to store the given key as well as value & a logger to show us the logs.

The Store configuration is as follows,

As mentioned earlier, the Object Store stores the data in the form of key-value pair. Here, the ‘key’ is the ‘song-details’ & the ‘value’ is the array of objects (payload). The payload is now stored in the object store via postman

Let’s send a request as follows,

Our data has been stored in the object store in the form of a key-value pair!

2. Retrieve

The Retrieve operation is used to return the value of the specified key from the object store. Enough theory, let’s do hands-on! Let’s get the data saved by the store operation. Creating a sample flow using HTTP listeners, fetch operations, and loggers as follows:

Note: The red exclamation over Transform message is Anypoint Studio metadata errors. You will often see them in your program, these have no effect over it & are to be ignored!

Here is what the configuration of ‘retrieve’ operation looks like –

• Default Value – If the key is present then mule returns the value of the key. If the key is not present then the default value specified in the configuration is returned.

Transform message component can be used here to mould the output from the retrieve operation according to our needs. Our transform message looks something like this since we want the output in the form of “song”-“artist”.

The retrieved value of the key looks like the image below as we required!

*An exercise for you guys* – Try putting a key that is not present in your object store, & see what output you will receive & leave a comment down below what you get!

3. Retrieve all

‘Retrieve All’ operation is used to retrieve all the keys and values from the object store.

However, using this flow, we get an output in a non-JSON format

As a result, we need to use a Transform Message component which has the following DataWeave –

%dw 2.0
output application/json 
---
payload mapObject (value,key)->
 {
     '$(key)': read(value,'application/json')
 }

The ‘Retrieve All’ can be configured easily just selecting the object store component that we are storing our data in, as follows –

Finally, our key-value pair is retrieved as follows

4. Contains

Contains operation is a quite simple one! It is used to check if there is any value associated with the specified key & then returns a Boolean value. It returns true if the key is present else returns false if the key is not present.

The flow for contains is as follows

The configuration of contains is as follows

The Transform Message is to transform the output to 

Let’s check if the key ‘employee-details’ is present in our object store or not by hitting the response on

Voila! We got ‘true’ since our key was present. But what if we tried putting some other key (here – chelsea) which is not present

5. Retrieve All Keys:

As the name of the operation suggests, Retrieve All Keys operation is used to retrieve all keys from the object store. The flow is as follows

As we created only one key, we get the following output –

6. Remove:

Remove operation is used to remove the values associated with the given key. If the key is not present then mule throws ‘OS:KEY_NOT_FOUND’ error. The flow is as follows;

The configuration for ‘Remove’ is as follows;

After hitting the remove response on Postman one time we get a blank body of output & hitting it one more time gives us the following output:

7. Clear:

Clear operation clears/deletes all the key-value pairs present in the object store. The flow looks as follows;

Configuration for ‘Clear’ component looks as follows:

Configuration for ‘Retrieve’ operation looks as follows;

After performing the clear operation, all the contents inside the Object Store were cleared which can be seen via the details logged under the Logger component;

Payload Inside the Object Store:

 {
        "song": "Fly Me To The Moon",
        "artist": "Frank Sinatra"
    },
    {
        "song": "Bohemian Rhapsody",
        "artist": "Queen"
    },
    {
        "song": "Never Gonna Give You Up",
        "artist": "Rick Astley"
    },
    {
        "song": "Nights",
        "artist": "Frank Ocean"
    },
    {
        "song": "God's Plan",
        "artist": "Drake"
    }
]

INFO  2022-04-12 00:05:24,063 [[MuleRuntime].uber.01: [employeedetails].employee-details-Flow6.CPU_LITE @5c4e6d09] [processor: employee-details-Flow6/processors/0; event: 2666feb1-b9c6-11ec-b602-983b8fb709d6] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: Before clear operation!

INFO  2022-04-12 00:05:24,068 [[MuleRuntime].uber.20: [employeedetails].employee-details-Flow6.BLOCKING @6c828605] [processor: employee-details-Flow6/processors/2; event: 2666feb1-b9c6-11ec-b602-983b8fb709d6] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: After Clear operation!

INFO  2022-04-12 00:05:24,071 [[MuleRuntime].uber.01: [employeedetails].employee-details-Flow6.BLOCKING @6c828605] [processor: employee-details-Flow6/processors/4; event: 2666feb1-b9c6-11ec-b602-983b8fb709d6] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: Nothing to Retrieve!

The retrieve operation comes to play at the end as per the above flow, however, as all the contents are cleared there is nothing to retrieve and thus, we get the following output “Empty” on Postman (refer to the default value of the retrieve component configuration in ‘Clear’).

One major question arises what is the difference between ‘Clear’ & ‘Remove’ operation. ‘Remove’ removes the value associated with the given key from the object-store. ‘Clear’ clears all the contents in the store. It removes all the content that is both keys and values from the object-store.

Our application seems good & ready to be deployed!

Have fun practicing, fellow Muley!