EIPinCPI – Pipes-and-Filters

December 15, 2019 Bhalchandra Wadekar

This week we begin studying Message Routing patterns. First Message Routing pattern is Pipes-and-Filters.

When do I use this pattern?

Use this pattern when the steps for the integration are reusable. The simplest example is reusing a flow that connects to the receiver system. For example, consider integrating a CRM system with ECC. The CRM system sends Customer and Orders to the ECC system. Here, the flow for both Customer and Order consists of 3 steps:

  1. Read from CRM System
  2. Map to IDoc
  3. Send IDoc to ECC

Reading the orders and customers would require separate flows. Mapping will also require separate flows as the source and target messages are different. However, sending IDoc to ECC can be designed to be a reusable step.

Pipes-and-Filters in CPI

In CPI, I’ll use JMS Adapters to demonstrate the Pipes-and-Filters pattern.

Demo of Pipes-and-Filters – Customer Integration

Let’s consider Northwind as the CRM system that creates/updates Customers. The flow will read customers from Northwind, map them to DEBMAS IDoc, and send the DEBMAS IDoc to ECC.

Step 1: Read Customer

Here, OData Adapter is used to read Customers from Northwind and JMS Receiver Adapter is used to send the read customers into ‘Customers’ queue.

Other Patterns

Here, the Read Customer demonstrates Request-Reply pattern.

Step 2: Map Customer

Here, customers are read from the ‘Customers’ queue using JMS Sender Adapter, mapped to DEBMAS IDoc and put into ‘ECC’ queue.

Step 3: Send to ECC

This flow reads from ‘ECC’ queue and posts the IDoc to ECC.

Demo of Reusability in Pipes-and-Filters – Order Integration

When sending orders from the CRM system into ECC, the same 3 steps are applicable. Read orders from Northwind, map them to ORDERS IDoc, and send the ORDERS Idoc to ECC. However, the 3rd step of sending the IDoc to ECC is a reusable component.

Step 1: Read Order

Similar to Customer Integration, this flow reads orders and puts them into the ‘Orders’ queue.

Step 2: Map Order

Similar to Customer Integration, this flow reads orders from the ‘Orders’ queue, maps them to ORDERS IDoc, and puts them in the ‘ECC’ queue.

Step 3: Send to ECC

This is a common step between Customer integration and Order integration. It reads from ‘ECC’ queue and sends the IDocs to ECC.

Monitoring tip for Pipes-and-Filter pattern

Dividing the flow of a message into several components like this does not make message monitoring difficult. Use the SAP_ApplicationID header to effectively understand the flow of the message. Check the 5th tip in Eng Swee Yeoh‘s tips for CPI development for more information. For official documentation, search for SAP_ApplicationID in Headers and Exchange Properties provided by the Integration Framework.

Conclusion

Pipes-and-Filters is a powerful pattern that promotes creating small components that are reusable. The dependency between these components is reduced through asynchronous processing. The Asynchronous processing is implemented by the use of JMS queues in CPI.

References/Further Readings

Hope this helps,
Bala

About the Author

Bhalchandra Wadekar

Bhalchandra Wadekar is an Integration Enthusiast who authored the Enterprise Integration Patterns in SAP Cloud Platform Integration (#EIPinCPI) series on SAP Blog Community. He is a fan of integration from API Management and Open Connectors to Message Based Integration and Data Integration.

More Content by Bhalchandra Wadekar
Previous Article
EIPinCPI – Format Indicator
EIPinCPI – Format Indicator

Next Article
EIPinCPI – Content-based Router
EIPinCPI – Content-based Router