Datastage is a data warehousing tool that has the capability and flexibility of handling the most demanding integration requirements. If you want to consider the origins of the current IBM product we will have to start with Ascential (Earlier called Informix when they had both the DB and integration divisions)who were waged in a competition of sorts to bringing forward tools that would take care of integration between systems seamlessly coupled with a wide range of transformation functions. The potential of pushing Datastage was realised especially in the last couple of decades where corporations were seeing huge growth in their data warehouse and data stores along with the need for better and faster business intelligence. The growths of such data stores were huge and exponential that it made sense in going for multiple servers instead of the old one server store.
Ascential had initially started off with server jobs which did not have the flexibility of parallel runs. They achieved some degree of parallelism by introducing the concept of multiple instance jobs. Having trained its eyes on the integration market and realising the need to faster better integration solutions, Ascential acquired Torrent Systems for sole intention of using their parallel engine. If you look at the Datastage installation folder you can see that there are 2 engines. The DSEngine, which was and still being used for server jobs , and the PXEngine, which was acquired from Torrent Systems, which is used for parallel job runs. In addition to the above acquisition Ascential also acquired Vality, Metagenix and Mercator for its various different functions.
After these acquisitions Ascential rolled out its first major breakthrough Datastage product. Datastage 6. They had introduced the parallel job type with a new set of parallel stages. They also introduced Server job shared container for parallel jobs. Ascential then went into overdrive to release Datastage 7 which introduced a whole lot of functionalities as compared to its predecessor. This release gave a huge amount of focus to parallel jobs as they realised parallelism was the road to the future. Datastage 7.5 x 2 was released in December 2004 and this was the first release of parallel jobs that could run on Windows. While the Server runs on all the same Unix and Linux platforms as 7.5.1 it adds the additional platform of Windows 2003 Standard or Enterprise on the Intel x86 Processor Family. There were no changes to parallel jobs in this release apart from the capability to compile and run them on Windows.
In 2005 IBM acquired Ascential Software and moved the products into the WebSphere Information Integration suite. Datastage 8 was released in October 2006 for Windows and April 2007 for Unix this is the first version to run on theIBM Information Server. There are a number of parallel job improvements in this release:
Lookup stage now supports two new lookup types: range lookup and caseless lookup.
New Slowly Changing Dimension stage
New QualityStage stages for parallel jobs.
Java stages are available by default as compared to its earlier state as plugins
Webservice stages
Parameter sets