What is a duplicate?

Introduction

In Integration Fox, duplicates are identified when multiple records share the same value for a specified Cross Entity Relationship (CER). The CER acts as a unique identifier to determine whether records represent the same record between the two platforms being integrated.

Example:

Suppose two contact records exist in your system:

• Contact A has a Contact ID of 1234 and an email address sales@company.com.

• Contact B has a Contact ID of 7890 and the same email address sales@company.com.

If the CER has been set to use the email field, the platform will flag these records as duplicates because their CER values (email addresses) match. This ensures data integrity by highlighting potential overlaps for review or resolution.

Why Duplicates Are Not Synced

Preventing Data Corruption

Integration Fox prioritises data integrity above all else. When duplicate records share the same Cross Entity Relationship (CER) value, the platform cannot safely determine which record should receive updates from a connected system. Syncing changes to multiple duplicates risks overwriting or corrupting critical data, as conflicting information may exist across records.

Example Scenario:

Using the earlier example where two contacts share the email sales@company.com (CER = email):

• Contact A (ID 1234) has the first name "J" and last name "Smith".

• Contact B (ID 7890) has the first name "J" and last name "Doe".

If an operator updates the first name to "John" in the external platform, Integration Fox cannot automatically apply this change. By pausing the sync for this record, the platform prevents ambiguous or destructive changes.

Key Takeaway:

Duplicates create uncertainty in data ownership. Integration Fox avoids assumptions to safeguard your systems from unintended data loss or corruption. Resolving duplicates (e.g., merging records or adjusting the CER configuration) restores reliable syncing.

Resolving Duplicates

1. Prevention: Enforce Data Integrity at the Source

The most effective way to manage duplicates is to prevent them from occurring. Proactively refine your business processes and system configurations to enforce uniqueness for critical fields. For example:

Review and adjust your Cross Entity Relationship (CER) configuration to ensure it aligns with a truly unique identifier (e.g., email, customer ID).

Implement validation rules in source systems to block duplicate entries during data creation.

2. Remediation: Address Existing Duplicates

If duplicates already exist, follow these steps to resolve them:

Request a Duplicate Report: This will identify all records flagged as duplicates based on your CER configuration. This report provides visibility into overlapping data and helps prioritize cleanup efforts.

Manual Resolution: Rename one of the duplicate records you no longer want to keep. EG: Contact B - OLD. Once you have renamed it you can then either merge the record into the record you wish to keep or archive the duplicate record.

3. Integration Reset: Restore Reliable Syncing

Once duplicates are resolved:

Schedule an integration reset to ensure the platform syncs only clean, authoritative data moving forward. This step prevents outdated or conflicting records from resurfacing in connected systems.

Key Takeaway:

Resolving duplicates requires a combination of proactive prevention and systematic cleanup. By addressing root causes (e.g., CER configuration) and leveraging tools like Duplicate Reports, you can maintain seamless integration workflows and ensure trustworthy data across platforms.