Back to Insights
IntegrationTodayJustin Pennington

Design the Failure Path: What Your Integration Does When the Sync Breaks

Design the Failure Path: What Your Integration Does When the Sync Breaks

Every integration demo goes the same way. An order is placed on the website, someone refreshes the ERP, and there it is — customer created, order logged, inventory decremented. Everyone nods. The project gets approved.

That demo covers maybe two percent of what the integration will actually do over its life. The rest is the bad day: the API that times out, the address field with an emoji in it, the credit hold that rejects a record the website already accepted payment for. The real question isn't whether your integration works. It's what it does when it doesn't.

Four Ways a Sync Actually Fails

Most teams treat "the integration broke" as one event. It isn't. There are four distinct failure modes, and they need four different responses.

Transient failures. The network hiccuped. The ERP was mid-restart. The API rate limit kicked in. Nothing is wrong with the data — the attempt just landed at a bad moment. These should be retried automatically, and in most cases nobody should ever hear about them.

Bad data. A required field is empty, a date is malformed, a currency code doesn't exist. Retrying this a thousand times will fail a thousand times. It needs a human, or a cleanup rule, not persistence.

Business rule rejections. The record is technically valid but the receiving system said no: customer over credit limit, item discontinued, warehouse closed. This isn't a technical error at all. It's a business decision that arrived through a technical channel, and it usually needs a person with authority, not an IT ticket.

Silent partial success. The most dangerous one. The customer record was created, the order failed. Or the order posted but the inventory adjustment didn't. Nothing alarms, because from each system's perspective everything it was asked to do worked. You find out at month-end.

If your integration handles all four the same way, it's handling none of them well.

Retry Without Creating Duplicates

The instinct to retry is correct. The risk is that you retry something that actually succeeded — the record posted, but the confirmation never made it back — and now you have two of them.

The fix is making each operation safe to repeat. Practically, that means every record carries a unique identifier from the sending system, and the receiving system checks for it before creating anything. Send the same order twice, and the second attempt updates or is ignored rather than creating a twin. This is unglamorous plumbing, and it is the difference between an integration you can trust and one your team quietly stops trusting after the first duplicate invoice.

It's also why "just add a retry" is rarely a five-minute change. Retries are only safe if the receiving end was designed to expect them.

Nothing Gets Dropped, Ever

When a record can't be processed, it has to go somewhere. Not a log file nobody reads — somewhere with a name, a count, and an owner. Call it a holding queue, an error bin, whatever your team will actually say out loud.

The rule is simple: a record is either processed or it's sitting in the queue. There is no third state. If you can't produce a number for "how many records are stuck right now," your integration doesn't have error handling. It has hope.

The queue also needs a replay button. Once the underlying problem is fixed — the customer record cleaned up, the credit hold released — someone should be able to reprocess the stuck items without re-keying them or calling a developer.

Errors Need a Human Owner, Not an Inbox

Technical alerts sent to a shared IT address get filtered within a month. Alerts that work look different. They say what happened in business language — "3 orders from Tuesday haven't reached the ERP" — they go to a named person whose job includes fixing it, and they escalate if the count grows instead of shrinking.

Ask yourself one question: if the integration stopped completely tonight, how long before someone noticed? If the answer is "when a customer calls," the monitoring is decorative.

Reconciliation Catches What Alerts Miss

Alerts catch loud failures. Reconciliation catches quiet ones. A daily job that compares counts and totals across both systems — orders created versus orders received, invoice totals on each side — will surface the silent partial successes that no error handler ever fired on.

It doesn't need to be sophisticated. Two numbers that should match, checked every morning, and a flag when they don't. Most of the ugliest integration surprises we see would have been caught in week one by a check that simple.

What to Ask Before You Sign

When you're evaluating an integration build — internal or vendor — the happy path will look fine in every proposal. Press on the rest:

  • What happens to a record that fails, and who sees it?
  • Can the same record be sent twice without creating a duplicate?
  • Who is notified, in what language, and after how long?
  • How do we reprocess stuck records without a developer?
  • What daily check would tell us the two systems still agree?

Answers to those five questions predict how the integration will feel in year two far better than any feature list.

If your systems are connected but you've quietly lost confidence that the numbers match — or you're scoping a new integration and want the failure path designed in from the start — that's exactly the kind of work we do. Reach out to Infraxio and let's look at it together.