Before we examine why you should be looking at SOA, let’s back track and cover the basics of SOA. An SOA infrastructure is an ecosystem of products that work together to enable service endpoints to communicate in a managed and coordinated way. For example:
- An ecosystem of products could include; order management, inventory control, warehouse management, invoicing, and delivery management.
- An example of a set of service endpoints could include a web-interface that allows customers to order products, check for availability, check their credit limit and check for when the product will be delivered.
The SOA toolset would allow each of these applications to communicate so that once a product has been ordered, it would then be removed from inventory, if stocks reached a certain level, then further supplies would be ordered and organised.
Once the order is confirmed by the customer pressing the confirm button on a web-interface, it would be automatically scheduled for delivery and a message sent to the shipping department for when the goods are to leave and arrive.
In addition an invoice could automatically be sent to the customer via email. All of this without any intervention from an operator to produce invoices, check for stock, check for delivery times or re-order further stock.
SOA architecture will typically include:
- A toolset to handle messaging and communication between the applications in the ecosystem
- A toolset to handle business process management that will sort out the workflow and sort out what each application is to do next.
For example when an order is received, it needs to be invoiced and then sent to Warehousing Application to arrange delivery, the Warehousing Application would automatically allocate a carrier.
In addition the Warehousing or Order Management Applications would also talk to the Inventory Control Application to remove the item from stock. The Inventory Control Application would also automatically order further product once supplies have reached a specified limit.
- A toolset to handle business rules
For example, some business rules might include give a 10% discount when a hundred items are ordered, charge extra for delivery on Sundays and bank holidays, give a complimentary ticket when a certain promotional code is entered.
- A toolset to handle data to ensure each application can understand the data from a variety of data sources and to aggregate and produce reports that you could not produce by using one application on its own.
Without SOA, substantial programming would be involved for example to:
- Produce a consistent method for each of these applications to communicate with every other application in the ecosystem in order to share their data.
- Produce an interface that co-ordinates and manages the outputs (or service endpoints) produced by the integration of the applications.
The SOA infrastructure includes several product categories that allow you to integrate these products and produced the desired outputs quickly and consistently without the need for substantial programmer intervention.
You can read up more about SOA in the special report here.
In this article we will focus on Data Services as they can be considered the first step in an SOA initiative.
Introducing Data Services
Data Services are still a relative newcomer to the field of IT and are an evolution of various attempts to have one interface for multiple databases. A Data Service Platform is a set of tools used for creating, publishing and operating Data Services.
All the enterprise IT companies (Red Hat, Microsoft, IBM and so on) have a set of Data Services tools that they provide to access any data source whether it be the latest version of Oracle, an old legacy DOS database or a web-based site. These set of tools should also be able to work with any operating system platform be it Linux, Windows, UNIX or MAC.
This doesn’t seem to be particularly exciting until you realise that every small company has at least three or four disparate databases of information to the large enterprises which will have at least fifty separate silos of information.
We can quickly count them as the HR employee database, the accounting system, the network directory database (eg active directory), asset management database, any customer relationship software, our email system, purchase ledger, stock control and so on.
Now in these environments a Data Service initiative has many benefits including lower data management costs and compliance of financial and security information (Credit Crunch and Security). You can see how the Data Services tools work by considering Diagram 1
In Diagram 1 (see below), we have the top row which consists of a set of data consumers.
These data consumers include:
- The Business Intelligence Data Consumer can drill down into all the data produced by our ecosystem of applications to give us information on for example our top 20% of customers and how much they’ve ordered
- The Spreadsheet Data Consumer – can allow us to take the data and produce sophisticated what if analyses
- The Portal Data Consumer – can enable us to have an on-line delivery verification system based on what products are ordered
- The Data Dashboard Data Consumer – could quickly give us graphical representations of market segments by product or market segments by location at a more high level view than our Business Intelligence Data Consumer
Decoupling of Data
The Bottom Row of Diagram 1 represents a variety of data sources. For example the van delivery data source could be from an access database, inventory management data source could be in Oracle database format and we may have SAP data source from our invoicing application.
Note the Data Layer above these data sources. This Data Layer is important as it wraps data in a format of your choice. Essentially the layer decouples data from their data sources to allow the data consumers to understand the various data sources.
Lower Data Management Costs
As an IT consultant, I get to manage and work with many different projects. All of them have some data stored in some form and often require either internal database programmers or are outsourced (ie to the software vendor) for customisation as no two companies ever seem to be the same. This can be incredibly costly if you have five or six different silos of information that require
a. Individual customisation.
b. A single perspective view
c. Integration of some components
d. The same information to be input into each database (Eg employee information would need to be input into at least HR database, Network directory database, email database and the payroll database)
With a Data Services Platform you have:
- The ability to create reports on every data source in a consistent format.
- The ability to extract data without having to learn each application that created the data source. This alone greatly reduces customisation and consultancy costs.
- The ability to combine information extracted from a variety of data sources without having to have further programming and re-engineering done.
- The ability to use a single interface to input all data
- The ability to have a uniformity of view (ie you don’t have to use the Oracle report view and then for example adjust to a web-based report view for a web-based database or a legacy input screen for an old Dbase III system).
- The ability to work with future implementations of applications and databases
Another problem that occurs is when changes and upgrades are made to the database. Many of these changes can affect the reports that we receive on the data and cost further consultancy to correct.
By using a Data Services approach, you are actually decoupling your applications and reports from your data sources so that your reports can remain unaffected no matter how much the database application or data source format changes. As a result ‘correction consultancy’ is no longer needed when upgrades are made.
Credit Crunch and Security Compliance
So what has the credit crunch and security compliance have to do with SOA and Data Services? Well actually everything. Since Sarbanes Oxley, the 9/11 terrorist attack, the Dot Com Boom and Bust and the current Credit Crunch crisis (caused by a variety of malpractices), corporations are under more and more pressure to prove financial and security compliance and allow greater scrutiny from governments.
This pressure is likely to get worse as more regulation comes in to prevent and to locate answers to either a corporations financial state or vulnerability to attack. Reports and access to data from disparate data sources are required and sometimes within impossible development timescales and budgetary restraints.
Data Services Agility and Flexibility
Data Services toolsets have been shown to reduce development times and give you the ability to leverage your existing data assets, for instance by producing complex reports linking multiple data sources in near real time. It is this agility which is one of the most important benefits of Data Services especially within the last decade’s economic turbulence.
Companies are increasingly having to show compliance and change the way they view their data sources in shorter and shorter timelines. Data Services flexibility allows corporations to create modular components for their different data sources to create an agile architecture.
The Best Data Services Suite
I have deliberately avoided getting involved in the technicalities of SOA and Data Services as they are different for each software vendor and really wouldn’t like to comment on who has the best suite as it is still too early to say. However it is certainly worth you examining the various offerings in greater depth.
Some vendors like Microsoft even have free webcasts on the subject. One vendor, Red Hat who have also sponsored this report have taken a slightly different approach as they now provide a Data Services Platform based on Open Source technology.
This can only be good news in an industry increasingly under attack from organised cyber criminals in that it is relatively easier for your programmers to detect Trojans and other malware within any code compared to non-open source code.
You can see the Red Hat Data Services Designer tool in action in Diagram 2 (below) aggregating information from different data sources.
You can download a variety of whitepapers from Redhat’s Data Services division here.
Ben Chai is a IT and Security consultant, analyst and writer. He currently runs incomingthought.com.