This article describes the capabilities of Oracle GoldenGate replication technology. Oracle GoldenGate enables the exchange of changed data between heterogeneous databases and provides the capability to migrate databases to another platform or database with zero or minimal downtime. It is very light weight replication software as compared to other replication technologies available in the market, it captures the transaction level committed data from source database and move, apply that committed transactions over to the target database server near real time.
It is widely used replication for Oracle Snapshot Standby databases for MIS reporting purpose, specially in banking domains, financial sectors. I have worked for India's leading banking customer where we use Oracle GoldenGate to replicate crucial business reporting tables from production database servers to MIS (Snapshot Standby) reporting databases for real-time reporting for the management and Branch users to offload the workload from Core-Banking production server to the Snapshot-Standby databases.
Below is the diagram which depicts the type of replication possible with Oracle GoldenGate.
With the flexibility, and the filtering, transformation, and custom processing features of Oracle GoldenGate, you can support numerous business requirements:
¦ Business continuance and high availability.
¦ Initial load and database migration.
¦ Data integration.
¦ Decision support and data warehousing.
Oracle GoldenGate can be configured for the following purposes:
- A static extraction of data records from one database and the loading of those records to another database.
- Continuous extraction and replication of transnational DML operations and DDL changes (for supported databases) to keep source and target data consistent.
- Extraction from a database and replication to a file outside the database.
Oracle GoldenGate is composed of the following components:
- Manager
- Extract
- Data pump
- Replicat
- Trails or extract files
- Checkpoints
- Collector
Overview of Manager
---------------------------
Manager is the control process of Oracle GoldenGate. Manager must be running on each system in the Oracle GoldenGate configuration before Extract or Replicat can be started, and Manager must remain running while those processes are running so that resource management functions are performed.
Manager performs the following functions:
■ Start Oracle GoldenGate processes
■ Start dynamic processes
■ Maintain port numbers for processes
■ Perform trail management
■ Create event, error, and threshold reports
One Manager process can control many Extract or Replicat processes. On Windows systems, Manager can run as a service.
Overview of Extract Process:
------------------------------------
The Extract process is the extraction (capture) mechanism of Oracle GoldenGate. Extract runs on a source database system or on a downstream database, or both, depending on the database and the implementation requirements.
You can configure Extract in one of the following ways:
■ Initial loads: For initial data loads, Extract process extracts (captures) a current, static set of data directly from their source objects.
■ Change synchronisation: To keep source data synchronised with another set of data, Extract captures DML and DDL operations after the initial synchronisation has taken place.
Multiple Extract processes can operate on different objects at the same time. For example, two Extract processes can extract and transmit in parallel to two Replicat processes (with two persistence trails) to minimise target latency when the databases are large. To differentiate among different Extract processes, you assign each one a group name.
Overview of Data Pumps
------------------------------
A data pump is a secondary Extract group within the source Oracle GoldenGate configuration. If a data pump is not used, Extract must send the captured data operations to a remote trail on the target. In a typical configuration with a data pump, however, the primary Extract group writes to a trail on the source system. The data pump reads this trail and sends the data operations over the network to a remote trail
Note: Extract ignores operations on objects that are not in the Extract configuration, even though the same transaction may also include operations on objects that are in the Extract configuration. on the target. The data pump adds storage flexibility and also serves to isolate the primary Extract process from TCP/IP activity.
In general, a data pump can perform data filtering, mapping, and conversion, or it can
be configured in pass-through mode, where data is passively transferred as-is, without
manipulation. Pass-through mode increases the throughput of the data pump, because
all of the functionality that looks up object definitions is bypassed.
Overview of Replicat
--------------------------
The Replicat process runs on the target system, reads the trail on that system, and then reconstructs the DML or DDL operations and applies them to the target database. Replicat uses dynamic SQL to compile a SQL statement once, and then execute it many times with different bind variables.
You can configure Replicat in one of the following ways:
■ Initial loads: For initial data loads, Replicat can apply a static data copy to target
objects or route it to a high-speed bulk-load utility.
■ Change synchronization: When configured for change synchronization, Replicat
applies the replicated source operations to the target objects using a native
database interface or ODBC, depending on the database type.
You can use multiple Replicat processes with one or more Extract processes and data
pumps in parallel to increase throughput. To preserve data integrity, each set of
processes handles a different set of objects. To differentiate among Replicat processes,
you assign each one a group name
Overview of Trails
-----------------------
To support the continuous extraction and replication of database changes, Oracle GoldenGate stores records of the captured changes temporarily on disk in a series of files called a trail. A trail can exist on the source system, an intermediary system, the target system, or any combination of those systems, depending on how you configure Oracle GoldenGate. On the local system it is known as an extract trail (or local trail). On a remote system it is known as a remote trail.
Overview of Extract Files
-------------------------------
In some configurations, Oracle GoldenGate stores extracted data in an extract file instead of a trail. The extract file can be a single file, or it can be configured to roll over into multiple files in anticipation of limitations on file size that are imposed by the operating system. In this sense, it is similar to a trail, except that checkpoints are not recorded. The file or files are created automatically during the run. The same versioning features that apply to trails also apply to extract files.
Overview of Checkpoints
-------------------------------
Checkpoints store the current read and write positions of a process to disk for recovery purposes. Checkpoints ensure that data changes that are marked for synchronization actually are captured by Extract and applied to the target by Replicat, and they prevent redundant processing. They provide fault tolerance by preventing the loss of data should the system, the network, or an Oracle GoldenGate process need to be restarted. For complex synchronization configurations, checkpoints enable multiple
Extract or Replicat processes to read from the same set of trails. Checkpoints work with inter-process acknowledgments to prevent messages from being lost in the network. Oracle GoldenGate has a proprietary guaranteed-message delivery technology.
Overview of Collector
---------------------------
Collector is a process that runs in the background on the target system when continuous, online change synchronization is active.
Collector does the following:
■ Upon a connection request from a remote Extract to Manger, scan and bind to an available port and then send the port number to Manager for assignment to the requesting Extract process.
■ Receive extracted database changes that are sent by Extract and write them to a trail file. Manager starts Collector automatically when a network connection is required, so Oracle GoldenGate users do not interact with it. Collector can receive information from only one Extract process, so there is one Collector for each Extract that you use. Collector terminates when the associated Extract process
terminates.
