Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Oracle Goldengate 12c Implementers Guide

You're reading from  Oracle Goldengate 12c Implementers Guide

Product type Book
Published in Jul 2015
Publisher
ISBN-13 9781785280474
Pages 422 pages
Edition 1st Edition
Languages

Table of Contents (21) Chapters

Oracle GoldenGate 12c Implementer's Guide
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Getting Started Installing and Preparing GoldenGate Design Considerations Configuring Oracle GoldenGate Configuration Options Configuring GoldenGate for HA Advanced Configuration Managing Oracle GoldenGate Performance Tuning Troubleshooting GoldenGate The Future of GoldenGate GGSCI Commands
GoldenGate Installed Components
Acronyms
Index

Chapter 10. Troubleshooting GoldenGate

Troubleshooting is quite often the final chapter in most technical books, and for good reason, we need to understand how a product works before we can fix it. Having said that, readers with a sound knowledge of GoldenGate can dive straight in to find the key point or action that they seek.

In this chapter, we will discuss the following topics in detail:

  • Investigating Oracle errors

  • Investigating and resolving network issues

  • Handling exceptions to avoid a process failure

  • Recovering from a process failure

  • Conflict detection and resolution

  • Analyzing the GoldenGate health check report

  • Handling Oracle sequences

  • Using the LOGDUMP utility to diagnose data-related issues

  • Upgrading Oracle GoldenGate

  • Performing Dynamic Rollback

Starting with Troubleshooting tips, you will learn how to investigate and resolve some of the common issues faced by the GoldenGate administrator.

Troubleshooting tips


Although you have configured your GoldenGate environment, maybe in its simplest form, with one source and one target database, but despite starting the Manager, Extract, and Replicat processes, data is not being replicated. So, what can be done?

The easiest way to determine that data replication is not working is to execute row count SQL against a given source table and compare results with its target table. Then, drill into the GoldenGate processes to obtain their status and statistics.

The following sections provide the necessary troubleshooting tips to enable a quick resolution to the most common issues.

Troubleshooting process startup failures

Start your troubleshooting at the source. We need to determine that Extracts are processing data before looking at all the other components in the data stream. Once confirmed, we can move on to troubleshoot the data pump and Replicat processes.

If the Extract process will not start, the SEND command will fail. However, GoldenGate...

GoldenGate health check


Since Oracle GoldenGate 11g Release 2, a health check script has been made available on the MOS portal at https://support.oracle.com.

The SQL script comes in three flavors for each database version and provides a health check report for integrated capture and apply processes. The supported database versions are:

  1. Oracle database 11.2.0.3 (integrated capture only)

  2. Oracle database 11.2.0.4

  3. Oracle database 12.1.0.1

Once logged in to MOS, search for Doc ID: 1448324.1 and download the required version. Copy the script to your database server and execute it as follows:

sqlplus /nolog
SQL> spool /tmp/icrhc_12101.html
SQL> @icrhc_12101.sql

When the script completes, the HTML file can be opened and viewed using a web browser.

The report provides a wealth of information that comprises the following areas:

  1. Summary

    • The overview information

    • Advice and warnings of potential issues with configuration

  2. Configuration

    • The generic database information

    • Integrated Extract/capture process configuration...

The GoldenGate dynamic performance views


Like all database components, Oracle provides a set of dynamic performance views (known as the v$ views) that allow the DBA to investigate and diagnose performance bottlenecks on the fly. The v$ views may be queried with minimal overhead during heavy processing to determine the following information:

Investigating network issues


GoldenGate is heavily dependent on a fast and reliable network between a source and the target database; without this, data replication cannot take place. Furthermore, if your network fails, the Extract, data pump, and Replicat processes may not necessarily ABEND, so you could be running blind.

TCP/IP

By default, the GoldenGate Manager uses the TCP port 7809. This starts a server collector process on the port 7819. As the load increases, the server collector dynamically chooses a port in the default range or from that specified in the range configured by the Manager process' DYNAMICPORTLIST parameter. The Manager can dynamically spawn up to 256 additional processes running on dedicated ports, which may or may not be open and available to the operating system.

One obvious check to confirm connectivity between hosts is to use the Linux ping utility. However, ping uses the Internet Control Message Protocol (ICMP), which can also be blocked by firewalls. So, to confirm...

Investigating Oracle errors


It is impossible to discuss every eventuality. However, should an Oracle error cause the Manager, Extract, or Replicat process to ABEND, their corresponding report files will contain the Oracle error number and message.

The following log message shows an ORA-01466 error and the corresponding error description from a failed Extract process:

2015-04-11 10:14:24  ERROR   OGG-00663  Oracle GoldenGate Capture for Oracle, eload01.prm:  OCI Error ORA-01466: unable to read data - table definition has changed (status = 1466), SQL <SELECT x."DEPTNO",x."DNAME",x."LOC" FROM "SRC"."DEPT"  AS OF SCN 141984  x>. 

This is useful in determining the root cause, but in some cases, it doesn't provide sufficient detail on how to progress the issue. Luckily, help is at hand; every Oracle Home contains an error code lookup utility, which nine times out of ten provides additional information about the error and instruction on how to fix it. The utility is oerr, which is located...

Exception handling


Oracle GoldenGate 12c now supports Conflict Detection and Resolution (CDR). However, out-of-the-box, GoldenGate takes a catch all approach to exception handling. For example, by default, should any operational failure occur, a Replicat process will ABEND and roll back the transaction to the last known checkpoint. This may not be ideal in a production environment.

The HANDLECOLLISIONS and NOHANDLECOLLISIONS parameters can be used to control whether or not a Replicat process tries to resolve the duplicate record error and the missing record error. The way to determine what error occurred and on which Replicat is to create an exceptions handler.

Exception handling differs from CDR by trapping and reporting Oracle errors suffered by the data replication (DML and DDL). On the other hand, CDR detects and resolves inconsistencies in the replicated data, such as mismatches with before and after images.

Exceptions can always be trapped by the Oracle error they produce. GoldenGate...

Handling Oracle Sequences


Although GoldenGate supports the replication of Oracle Sequence values, there are a number of issues we need to be aware of. For example, GoldenGate will support Oracle Sequence replication for CDC if checkpoints are not maintained. This is dependent on the SPECIALRUN parameter that exists in the Replicat parameter file. However, they are not supported for initial loads in which the source data is derived from the source tables and not the redo logs. So, the sequence values are not extracted.

For bidirectional (active-active) environments, the database sequences must generate values on the target database independent to the source. Note that in a cascade environment, GETAPPLOPS must be enabled on the Extract to capture sequence values replicated by the Replicat process.

Sequence gaps often occur in an Oracle database, depending on the associated sequence, cache size, and number of instances in the RAC environment. However, unless the NOCHECKSEQUENCEVALUE parameter...

Using LOGDUMP


LOGDUMP is a great utility and a real bonus to the Oracle GoldenGate software bundle. Without LOGDUMP, we will not be able to read a trail file, which would make us blind to troubleshooting data-related issues.

LOGDUMP has a command-line interface that allows you to open files, format the display, and navigate through a file, including filtering data. To invoke the utility, navigate to the GoldenGate Home directory and enter logdump, as shown in the following command:

[oracle@db12c ogg]$ ./logdump
Oracle GoldenGate Log File Dump Utility for Oracle
Version 12.1.2.1.0 OGGCORE_12.1.2.1.0_PLATFORMS_140727.2135.1
Copyright (C) 1995, 2014, Oracle and/or its affiliates. All rights reserved.

Logdump 49 >ENV
Version             : Linux, x64, 64bit (optimized) on Aug  7 2014 02:57:05
Current Directory   : /u01/app/oracle/product/12.1.2/ogg
LogTrail            : *Not Open*
Display RecLen      : 140
Logtrail Filter     : On
Trans History       : 0 Transactions, Records 100, Bytes...

Upgrading GoldenGate


Should you be unfortunate enough to hit a bug in your GoldenGate environment, Oracle Support may suggest an upgrade to the latest release. Upgrading the Oracle database can be both complex and risky. It will also require careful planning and testing. GoldenGate upgrades are simple in comparison.

Testing is an essential part of any upgrade. When instructed by Oracle Support to upgrade to a newer version, always ensure that the issue is resolved (with no side effects) in a test environment before upgrading production systems.

To upgrade the GoldenGate Home to a later release, we can follow these simple steps that are the same for a fresh installation, as described in Chapter 2, Installing and Preparing GoldenGate:

  1. Download the required GoldenGate version media pack from the Oracle Software Delivery Cloud by entering the following URL into your web browser: http://edelivery.oracle.com

    Note

    You will need to register and accept the license agreement before you can download any...

Dynamic Rollback


Sometimes, transactions are performed on the source system in error. Ideally, you would not want them replicated to the target database. So, what can be done?

We can't feasibly shut down our production database and perform a point-in-time recovery, nor can we use flashback on a specific set of transactions. We could set up Event Actions to detect and ignore erroneous transactions, such as DELETE operations without a WHERE clause, but it is impossible to trap every eventuality. Fortunately, since GoldenGate 11g Release 2, a reverse data processing utility is bundled with the software that can back out transactions (once identified) based on the period of time in which they occurred.

The reverse utility is typically employed to undo accidental deletes that have occurred on a source table. Let's see how it works.

Creating process groups and trails for reverse processing

Before we can use the reverse utility, we must create dedicated process groups for input and output trail files...

Summary


Whether you are a novice or an experienced DBA, you will be drawn to these pages time and again. Troubleshooting is an everyday task for many IT professionals, but no one can master each and every eventuality. This chapter has captured some of the most common failure scenarios, offering help and guidance to a successful resolution. From using LOGDUMP to drilling into the GoldenGate trail files to automatic exception handling, you learned the importance of a methodical approach to troubleshooting.

We also discovered how to upgrade our GoldenGate environment with ease using OUI, providing the mechanism to keep us abreast of the constant demand for enhanced performance and new features.

In the next chapter, we will take a look at the technologies that are evolving from the Cloud, including the Oracle Integration Cloud Service and GoldenGate's role in the Big Data space.

lock icon The rest of the chapter is locked
You have been reading a chapter from
Oracle Goldengate 12c Implementers Guide
Published in: Jul 2015 Publisher: ISBN-13: 9781785280474
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime}

View name

Description

V_$GOLDENGATE_CAPABILITIES

This lists the enabled capabilities of GoldenGate in the database, such as

DBLOGREADER,

TRIGGERSUPPRESSION,

TRANSIENTDUPLICATE, and

DDLTRIGGEROPTIMIZATION.

V_$GOLDENGATE_CAPTURE

This is similar to V_$STREAMS_CAPTURE that provides the integrated capture process information.

V_$GOLDENGATE_TABLE_STATS

This lists the collision, detection, and resolution (CRD) statistics for the integrated apply process.

V_$GOLDENGATE_TRANSACTION

This lists the in-flight transaction statistics.

V_$GOLDENGATE_MESSAGE_TRACKING

This is available in Oracle database future Release 12.2...