Fast. Simple. No meetings. No delays.
Home / Blog / Software Development / CRM and ERP Integration Patterns for Web Applications

CRM and ERP Integration Patterns for Web Applications

Reliable integration patterns: queues, retries, idempotency keys, field mapping, reconciliation jobs and monitoring.

Cover graphic for the article: CRM and ERP Integration Patterns for Web Applications

Reliable CRM ERP integration patterns share the same core: decide which system owns each field, move changes through a queue, retry safely with idempotency keys, map data explicitly, run reconciliation jobs to catch drift, and monitor everything. Assume every call can fail, repeat or arrive late, and design for that.

This is the technical companion to the business-facing overview of integrating ERP, CRM and web systems with APIs, which covers scope and planning. Here the focus is on how the pieces behave under failure. For the shape of the APIs you expose yourself, see designing scalable, maintainable APIs.

Failure modes behind CRM ERP integration patterns

An integration that works in a demo can still corrupt data in production. List the failures before drawing the architecture:

  • Timeouts with unknown outcome. You sent a request and got no answer. Did the other side process it?
  • Duplicates. Webhooks and retries deliver the same message more than once.
  • Out-of-order events. An update arrives before the create, or an old update overwrites a newer one.
  • Partial failure. The customer was created, but the order referencing it was not.
  • Rate limits and downtime at either system, sometimes for hours.
  • Schema drift. A field is renamed, a picklist value is added, or a required field appears.
  • Bad data. Missing emails, duplicated customers, text in numeric fields.

Every pattern below answers one or more of these.

Sync direction and ownership

For every entity and field, decide which system is the source of truth. Typical splits: the CRM owns leads, contacts and sales stage; the ERP owns products, prices, stock, invoices and payment status; your web application owns its own user accounts and behaviour.

Prefer one-way flows per field. Two-way sync of the same field invites conflicts, since both systems change it. If you must have it, define a rule such as "last update wins by timestamp from the owner system" or "ERP wins for anything financial", and log every conflict. Document the result as a simple table: entity, field, owner, direction, trigger and frequency.

Webhooks or polling?

ApproachStrengthsWeaknesses
WebhooksNear-real-time, efficient, little wasted trafficCan be missed, duplicated or reordered; you need a public endpoint and signature checks
PollingYou control timing and load; easy to resume after downtimeDelay between change and detection; wasted calls; needs a reliable "changed since" filter
BothWebhook for speed, periodic polling as a safety netMore code, but the most robust for important data

Many older ERPs offer only polling, file exports or a database view, so your choices depend on what the system supports.

Queues and retries

Never call a slow or unreliable external system inside a user's web request. Put the work on a queue and process it in a worker, so the user sees a fast response and the integration can be paused and resumed.

  1. Receive and store first. When a webhook arrives, verify it, save the raw payload, return success, and process later.
  2. Retry with back-off. Retry transient errors (timeouts, server errors, rate limits) with increasing delays and some random jitter. Do not retry validation errors; they will fail every time.
  3. Set a maximum. After a number of attempts, move the message to a dead-letter queue or failed-jobs table for human review, with the error attached.
  4. Preserve order where it matters. Process messages for the same record in sequence, for example by partitioning on the record ID.

How do I handle rate limits?

Read the provider's documented limits and any headers that report remaining quota or a wait time. Throttle your workers to stay under the limit, honour "retry after" instructions, batch requests where the API supports it, and prefer incremental syncs over full reloads. A queue makes this natural: slow the consumers rather than failing the producers.

Idempotency keys

Idempotency means doing the same operation twice has the same effect as doing it once. It is what makes retries safe.

  • When the API supports an idempotency key header, generate a stable key per business operation (for example, a hash of the order ID and action) and reuse it on every retry.
  • When it does not, make your own writes idempotent: look up by an external ID before creating, or use upserts keyed on a unique reference.
  • On the receiving side, store the IDs of processed messages in a table with a unique constraint and skip repeats.
  • Store the other system's ID against your record the moment it is returned, so a retry updates instead of creating a second one.

Without this, a timeout followed by a retry can create duplicate invoices or customers.

Mapping and transformation

Field mapping is where integrations quietly go wrong. Treat it as code with tests, not as scattered assignments.

  • Keep mappings in one module or config per integration, and write them as explicit input-to-output functions.
  • Normalise early: trim whitespace, standardise phone, country and currency formats, and make time zones explicit.
  • Map picklists and status codes with lookup tables, and decide what happens with an unknown value (reject, default or park for review) instead of passing it through.
  • Handle required fields that one system has and the other lacks.
  • Version your mapping, and add tests using realistic sample payloads, including nasty ones.
  • Keep a canonical internal shape if more than two systems are involved, so you map each system once rather than to every other.

Reconciliation jobs

What is a reconciliation job?

A reconciliation job is a scheduled process that compares the data in two systems and reports or fixes differences. It exists because no event-driven sync is perfect: messages get lost, people edit records directly, and bugs happen.

A practical design:

  1. Choose a window (recent changes) and a full comparison less often.
  2. Compare keys and a few fields or a hash of them, such as status, totals and updated timestamps.
  3. Classify mismatches: missing on one side, different values, or duplicates.
  4. Auto-repair only safe categories according to the ownership rules, and send the rest to a review list.
  5. Record the counts and trend them over time. A rising mismatch rate is an early warning that something changed.

Observability

You cannot support an integration you cannot see. Build the following in from the start:

  • A correlation ID that follows a record through webhook, queue, mapping and outbound call, included in every log line.
  • Structured logs of each attempt with the system, operation, status and duration, while keeping personal data out of logs where possible.
  • Metrics: queue depth, age of the oldest message, success and failure counts, retry counts, and latency per external call.
  • Alerts on things that need a human: dead-letter items, a stuck queue, authentication failures, a sudden drop to zero traffic.
  • An admin screen to inspect a record's sync history and re-queue a failed item safely.

Putting it together

Start with the ownership table, then add a queue, idempotent writes, a tested mapping layer and a nightly reconciliation job. Expand to more entities once the first flow is boring. For the API side of the web application, REST API design for PHP applications is a useful companion. If you want a second opinion on an integration design, see our software development overview or get in touch.

Frequently asked questions

What is a reconciliation job?

It is a scheduled process that compares records in two systems and reports or repairs differences. It catches what event-driven sync misses: lost messages, direct edits and bugs. Safe categories can be auto-repaired using your ownership rules, while the rest go to a review list, with mismatch counts tracked over time.

Webhooks or polling?

Webhooks are fast and efficient but can be missed, duplicated or reordered. Polling is simpler to control and resume but adds delay and wasted calls. For important data, use webhooks for speed with periodic polling or reconciliation as a safety net. Some older systems only support polling or exports.

How do I handle rate limits?

Read the provider's documented limits and rate-limit headers, throttle your queue workers to stay under them, and honour retry-after instructions. Batch where the API allows, prefer incremental syncs to full reloads, and back off with jitter on limit errors. A queue lets you slow consumers instead of failing.

Related articles

🚀 Start Now