Connecting Existing Software: APIs, Data Sync and Failure Recovery

A useful integration includes recovery, not just connectivity

Connecting two applications means deciding which records they share, which system owns each fact and how the connection recovers when something fails. An API request that succeeds once demonstrates connectivity. A dependable integration also handles duplicate messages, interrupted requests, conflicting updates and missing data.

For a business owner, the practical objective is a clear operating result. An approved order reaches fulfilment. A recorded payment updates the appropriate balance. A customer correction appears in the places that need it. The engineering should make those outcomes understandable even when the systems are temporarily out of step.

Start with one data flow and its business consequence. Avoid beginning with a vague requirement to “sync everything both ways.”

Identify the system of record for each fact

The system of record is the authority for a particular piece of information. Your ERP may own stock, the commerce platform may own product descriptions and the finance system may own issued invoice numbers. One application does not necessarily own the entire shared customer or product record.

Document ownership at field or event level where needed. If staff can edit the same address in two systems, decide how conflicting changes are resolved. “Latest update wins” is a policy, not a neutral default, and may be inappropriate when an older event arrives late.

Also define what an event means. “Order created,” “order approved” and “order dispatched” are different business moments. Sending all three as a generic order update can cause the receiving system to take the wrong action.

Choose API, webhook or batch according to the job

Approach	Useful when	Questions to resolve
Direct API request	One system needs a result during a workflow	Timeout, authorization and uncertain outcomes
Webhook or event	A change should notify another system	Authenticity, duplicates, order and durable receipt
Scheduled polling	The source has no suitable event delivery	Cursor, rate limits and acceptable delay
Batch import/export	Data moves in larger periodic sets	Snapshot boundaries, reconciliation and rejected rows

A webhook is a message sent when something happens. It can reduce polling, but it is not a guarantee of exactly one delivery in perfect order. Stripe's official documentation explicitly discusses signature verification, retries, duplicate events and ordering behavior. Those are useful examples of why an integration must follow the particular provider's delivery contract. Source: Stripe webhook documentation.

This is an architectural example, not a claim that Stripe has been deployed in a particular AivoraLabs client project.

Map identities explicitly

The same customer or product often has different identifiers in different applications. Store the mapping instead of matching repeatedly by a name that people can edit. Decide how new records are created, how duplicates are detected and what happens when a source record is archived.

For historical imports, preserve the original identifier alongside the new one. That gives operators a way to investigate a rejected row or reconcile a document. If the integration silently creates a new customer each time the name changes, the apparent sync success hides a growing data problem.

Validate required fields before sending them. A receiving system may require a currency, tax code or organization reference that the source does not contain. Missing mappings should produce an actionable queue, not arbitrary guessed values.

Make repeated attempts safe

Consider an order-creation request that times out. The receiving system may have created the order even though the caller never received the response. Retrying blindly can create a duplicate. Giving up immediately can leave the caller believing the order does not exist.

An idempotency key identifies one intended operation across repeated attempts. The receiving side can recognize the key and return the previous outcome. The key should distinguish a retry from a genuinely new request; using only the request's visible values can conflate two legitimate similar orders.

AWS explains this distinction in its discussion of retry-safe APIs and caller-provided request identifiers. Source: Making retries safe with idempotent APIs.

Define how long deduplication records remain valid and what happens if the same key is reused with different data. These details belong in the integration contract and its tests.

Separate durable work from network delivery

When an application commits a business change, it can also record the notification that must be delivered. A worker then sends the notification and tracks the result. This pattern is often called an outbox.

The benefit is practical: a temporary network failure does not require repeating the original business transaction. An operator can retry delivery while the original order or payment keeps its identity and timestamp.

Retries should have limits and delays. A permanently invalid record will not improve because it is submitted every second. Separate temporary failures from validation failures, retain a useful error and provide a review path for records that need a person's decision.

Reconciliation catches what retries cannot

A queue can report that messages were processed while the resulting business records still disagree. Reconciliation compares the state that should exist across systems. It might check source orders against destination records, expected quantities against accepted quantities or payment references against recorded balances.

Choose the checks according to consequence. A missing marketing preference update and a duplicated invoice need different urgency. Keep the source reference, destination reference and last attempted operation available to authorized operators.

Do not fill logs with secrets or entire customer payloads when identifiers and error categories are sufficient. Operational visibility should help staff investigate without creating an unnecessary copy of sensitive data.

Plan a controlled cutover

Test with representative records and a provider sandbox when available. Include duplicate delivery, events arriving out of order, a timeout after success, invalid credentials, rate limiting and a record that requires manual repair.

For rollout, identify the starting snapshot or cursor and how changes during migration are captured. Begin with a bounded flow, compare expected results and expand after reconciliation is understood. Keep rollback instructions clear about data: reverting application code does not automatically reverse writes already made in another system.

Questions for an integration brief

Name the systems, owners and accessible environments. List the events and fields that must move, their authority, acceptable delay and expected volume. Add the failure cases that would interrupt the business, the person responsible for reviewing them and the evidence needed to accept the connection.

Our software integration, backend development and software migration services address these connected responsibilities. Discuss one data flow you need to make dependable.