Recover an unknown external outcome

A timeout tells the caller that it did not receive a usable answer within a limit. It does not prove that the remote system did nothing. Treating every timeout as a failure can duplicate charges. Treating every timeout as success can grant access without payment. The useful state is unknown, with a defined method to resolve it.

Store the local operation before making the external request, but do not hold a database transaction open across a slow network call. The operation needs a status, a stable provider key, expected amount and currency, attempt metadata, and the provider identifier when known. A state machine makes recovery reviewable. Pending can become confirmed, rejected, or unknown. Unknown can return to confirmed or rejected through authoritative evidence. Time passing alone is not evidence of rejection.

Provider responses, signed webhooks, and reconciliation queries can race. All of them must update the same operation using rules that prevent stale evidence from undoing a valid terminal result. A webhook that says an event occurred may require fetching the current object state, especially if delivery order is not guaranteed. Confirm that the identifier, account, amount, and currency match the intended operation before granting the business effect.

Retries must follow the provider's contract. Some APIs safely reuse an operation key. Some require a lookup before another attempt. A key may expire. Bound the retry budget and use backoff with randomness to avoid synchronized retry bursts. A recovery worker should surface unresolved operations for review rather than silently create a new key after every timeout.

A fictional customer buys a 1,200-unit subscription in currency INR. Local operation P5 uses provider key K5. The provider commits charge C8, but the reply is lost. The application marks P5 unknown and displays payment confirmation pending. A webhook later identifies C8. The handler verifies its signature, fetches the authoritative object if needed, checks the amount and currency, and changes P5 to confirmed. It creates entitlement T4 once using P5 as the business identity.

A delayed retry response also reports C8. It reaches the same confirmed state and does not create a second entitlement. The useful audit trail records both pieces of evidence without counting two purchases.

This teaching table describes one purchase workflow. The exact provider contract must be checked before implementation. In particular, key retention, lookup consistency, webhook semantics, and terminal-state meanings can differ between providers.

The table deliberately excludes a rule that unknown becomes rejected after a fixed number of minutes. A deadline can escalate investigation or change the user message, but it cannot establish that the remote charge never happened. If a provider offers a documented final absence guarantee, the application can use it. Without such a guarantee, absence from one eventually consistent lookup may be insufficient.

1P5 expected: account=A2, amount=1200, currency=INR, key=K52attempt R1: provider call starts3provider:   charge C8 committed4attempt R1: timeout5local P5:   unknown6webhook W1: C8 confirmed, signature verified7lookup L1:  C8 amount=1200 currency=INR account=A28local P5:   confirmed, entitlement T49attempt R2: returns C8 for K510local P5:   remains confirmed, T4 unchanged

The webhook is evidence, not a command to skip validation. Verify authenticity using the provider's documented method and raw payload requirements where applicable. Then connect the event to the intended operation. A validly signed event for another account is still not confirmation of P5. Signature verification and business matching answer different questions.

A late event can describe an older state. If the provider's event order is not guaranteed, a cancellation or refund event may arrive around a success event. The operation model must distinguish the purchase confirmation from later lifecycle events. Do not compress charge, refund, entitlement, and cancellation into one boolean paid. A refund may require a separate business decision about access rather than rewriting history to pretend the charge never occurred.

Suppose K5 is no longer retained by the provider. A new request with K5 might now be treated as a new operation, depending on the provider contract. The local application must not assume that the familiar string still carries its old guarantee. Retain enough local evidence to identify the original operation and use the documented reconciliation route.

The operator's recovery record should contain safe identifiers, expected amount and currency, known provider objects, attempts, and evidence timestamps. It should not require copying full payment details into a chat. A manual investigation can be valid when rare unresolved cases cannot be decided safely by automation. The key is to make the uncertainty visible and prevent an automatic second charge while the case waits.

The first misconception is that a timeout is equivalent to a rollback. The caller's waiting period and the provider's transaction are independent. The remote system may have finished after the caller stopped listening, or the reply may have disappeared after completion.

The second misconception is that a signed webhook proves every business field is correct for the local purchase. The signature establishes origin and integrity under the provider's scheme. The application must still match account, object identity, amount, currency, and relevant lifecycle state.

A third common failure is holding a database lock while waiting for an external provider. The lock can remain occupied during a slow response and create contention for unrelated operations. A durable state machine lets the application release local transactions between bounded steps while retaining the information needed for recovery.

Change L1 to return C8 in USD while P5 expects INR. The correct next state is review-required, not confirmed. Now make L1 temporarily return no object while W1 names C8. Preserve both observations and investigate through the provider's consistency contract. Award one point for each evidence-based response and one for explaining why a new key is not the default repair.

A reconciliation response finds a charge with the correct account and key but amount 1,500. Should the worker confirm the 1,200 operation? No. It records a mismatch and requires investigation. It must not silently reinterpret the user's purchase. Write the visible state and next step. The model answer is unresolved payment with no new charge attempt, a recorded mismatch, and a controlled investigation path. Award one point for each element.

Why not keep the HTTP connection open until certainty? A strong answer explains resource limits and independent recovery after client disconnects. Follow up with what happens after provider key expiry. A weak answer retries with a new UUID until it gets a 200 response. Unknown outcomes deserve durable state, bounded recovery, and evidence-based transitions.