Embedded Files
Integrating with HubSpot's API is generally seamless, but under certain conditions, users may notice delays in data synchronisation. Integration Fox prioritises reliability and efficiency, but API performance can be influenced by HubSpot's design limits and specific data scenarios. Below, we explain the root causes of these slowdowns and how to mitigate them.
1. HubSpot's Delta Sync Threshold and Backfill Process
By default, Integration Fox uses HubSpot's search feature to fetch only records modified since the last successful sync. This efficient process runs frequently (e.g., multiple times per hour) and works efficiently for routine updates.
However, if more than 10,000 records are modified or created in HubSpot within a short timeframe, HubSpot forces a full backfill. Instead of retrieving recent changes, the API forces a complete re-sync of all records in the affected object (e.g., contacts, companies). This process can take hours or even days, depending on:
Total record count in the object.
HubSpot's API rate limits and performance.
Complexity of data (e.g., custom properties, associations).
Common Triggers for Backfill Scenarios:
Large Data Imports: Bulk CSV uploads or third-party migrations.
Mass Email Campaigns: Sending emails to large segments (e.g., 10,000+ contacts).
Workflows/Automation: Actions that update thousands of records (e.g., bulk property changes, list enrollments).
Third-Party Integrations: Multiple apps modifying the same HubSpot object concurrently.
2. API Rate Limits and Pagination Overhead
HubSpot enforces strict API rate limits to ensure fair usage across all customers. For example:
10 requests per second (for most endpoints).
250,000 daily requests (varies by tier).
When Integration Fox processes a backfill, it must paginate through all records (e.g., 100 records per page for contacts). For large datasets, this results in thousands of sequential requests, which are slowed further by:
Retry Delays: Integration Fox automatically retries failed requests (e.g., due to rate limits), adding latency.
Data Parsing: Transforming nested HubSpot data (e.g., custom properties, associations) into your system's format.
3. Impact of HubSpot's Architecture
HubSpot's API is optimised for transactional use cases, not bulk operations. Key limitations include:
No Bulk Update Endpoints: Changes to individual records require separate API calls.
Search Limitations: Complex queries (e.g., filtering by multiple custom properties) perform slower.
Associations Overhead: Fetching linked objects (e.g., a contact's company) adds extra calls.
Mitigation Strategies
To minimise delays and avoid triggering backfills:
Prevent Backfill Scenarios
Avoid Bulk Updates During Peak Hours: Schedule large imports, workflows, or campaigns during off-peak times.
Batch Large Imports: Split CSV uploads into smaller chunks (e.g., 5,000 records per file).
Optimise Workflows: Avoid workflows that update all records in a list. Use segmentation to target smaller groups.
How Integration Fox Handles Delays
While backfills are unavoidable in some cases, Integration Fox ensures reliability by:
Queuing Requests: Automatically managing retries and rate limits.
Providing Status Alerts: Notifying you of backfill progress or errors.
Ensuring data integrity: Syncing the oldest modifications first, to maintain correct workflow behaviour in the destination platform.
For more information on HubSpot API usage guidelines, see the link below:
Page updated
Report abuse