Using SAP Data Services to Replicate Data From SAP Applications to BigQuery

If you want to query data from tables in the virtual environment, use the hyper-scaler storage to integrate data like Google BigQuery with business data in SAP Data Warehouse Cloud. The advantage is that it is possible to query Google BigQuery data from SAP Data Warehouse Cloud, leading to businesses being able to make an analysis of their critical operational data.

When you replicate data from SAP to BigQuery you can, in one place, analyze third-party and SAP data thereby maximizing the Rate of Return in both SAP and Google services. Further, you can ensure live connectivity to query BigQuery by virtual tables as well as get real-time analytics without data replication.

However, data replication can be used to combine the data from SAP systems with data from other systems in BigQuery as well as back up data in SAP. Replication from SAP to BigQuery is an automated process and can be executed by anybody conversant with the configuration of SAP Basis, SAP DS, and Google Cloud.

Data Replication from SAP to BigQuery

How does data replication from SAP to BigQuery work?

Data is first extracted from the SAP application or an underlying database by SAP Data Services and then transformed or formatted to match the architecture of BigQuery. Finally, the data is moved to BigQuery by the load job. Once this step is completed, the data is available in BigQuery for analysis.

When the export of data is to be done, it is possible to decide when the SAP Data Service should start the process. During replication, all data that is present in the target BigQuery table is overwritten by the newly exported data. After the process is completed, it is not necessary to keep the data in BigQuery in sync with the data in the source system.

An advantage here is that the CDC (Change Data Capture) feature of the SAP Data Services is leveraged by the SAP Replication Server for the provisioning of data and delta capabilities for all source tables in real time.

Let’s examine the flowchart or how data moves from SAP to BigQuery.

• First, SAP applications in the source system update the data

• he changes to the data are then replicated by the SAP LT Replication Server and stored in the Operational Delta Queue

• The queue for data changes is tracked at pre-set intervals by a subscriber of the Operational Delta Queue, SAP DS.

• The SAP DS then extracts the data from the delta queue and formats the data to make it compatible with the BigQuery structure. This starts the process of loading data from SAP to BigQuery.

• Once completed, data is now available in BigQuery for analysis.

The source database can be SAP HANA or any other but the rider is that it should be supported by SAP.