[Frontend SDK]: authModal stuck in pending state on idle timeout

[jesswang-dev]

Summary

The authModal fails to reflect the correct connection status when the OAuth process exceeds the WebSocket idle timeout (~120s). The connection is established successfully, but the modal remains stuck in a pending state.

Environment

Self-hosted Nango, using the front-end SDK with the following setup:
NANGO_HOST=https://nango.dev.xxxx.io
NANGO_CONNECT_URL=https://nango-connect.dev.xxxx.io

const nangoRef = useRef<Nango>(new Nango({ host: NANGO_HOST }))
const connect = nangoRef.current.openConnectUI({
  baseURL: NANGO_CONNECT_URL,
  apiURL: NANGO_HOST,
  themeOverride: 'system',
  detectClosedAuthWindow: true, // toggled to test behavior difference
  onEvent: (event: ConnectUIEvent) => {
    if (event.type === 'close') {
      // handle close
    } else if (event.type === 'connect') {
      // handle connect
    }
  },
})

Observed Behavior

When a user starts the OAuth process but takes more than 120 seconds to complete it, the auth modal does not update correctly. The connection is established successfully (it appears in the Nango dashboard), but the modal indicator remains in a pending state indefinitely.
Inspecting the WebSocket traffic shows that the success message is never received in this case.

Normal flow — WebSocket messages:

{"message_type":"connection_ack","ws_client_id":"e0d19324-c39d-4e9d-bf03-c247ce4a7a11"}
{"message_type":"success","provider_config_key":"hubspot","connection_id":"2be9db38-bd6f-4ea2-94fa-78925151805f","is_pending":false}

After idle timeout — WebSocket messages (success message never arrives):

{"message_type":"connection_ack","ws_client_id":"add2e0c2-014b-498b-aa74-7fb0e0783bcf"}

With detectClosedAuthWindow set to the default value false, the modal does not respond (keeps pending) after the connection is successful.

With detectClosedAuthWindow: true, the modal does respond, but the error message shown suggests the connection failed and prompts the user to retry. Even though the connection was already created successfully. This risks creating duplicate connections and is misleading.

Expected Behavior

One of the following resolutions would address this:
Preferred: After the WebSocket closes, the modal should poll or recheck the connection status and update to success if the connection is still open.
Alternative: Extend the WebSocket idle timeout to accommodate slower OAuth flows.
Fallback: If the WebSocket closes before a result is received, close the modal session cleanly so the user can retry without risking a duplicate connection.

Steps to Reproduce

1. Open the auth modal using nango.openConnectUI().
2. Begin the OAuth flow, but pause, keep the modal open for more than 120 seconds
3. Complete the OAuth flow after the wait
4. Observe that the modal remains in a pending state, even though the connection was created successfully

Root Cause Hypothesis

Based on WebSocket traffic, the server-side WebSocket connection closes after ~120 seconds of inactivity before the OAuth flow completes. The front-end modal relies on the WebSocket success message to update its status, so when the socket closes prematurely, the modal has no fallback mechanism to recover the final state.

Contribution

I'm happy to open a PR and would appreciate maintainer guidance before diving in. Based on what I can observe from the outside, the most robust fix seems to be adding a fallback in the front-end SDK: when the WebSocket closes without receiving a success message, poll the connection status endpoint once to check whether the connection was created. If it exists, transition the modal to a success state rather than an error or pending state.

It's also worth noting that detectClosedAuthWindow: true doesn't serve as a workaround here. The property is designed to detect when the user closes the OAuth popup as a cancellation, but it can't distinguish between "user closed the window without finishing" and "window closed after finishing, but the WebSocket had already timed out." In this scenario, it reports a false failure for a user who actually succeeded, which makes the UX worse.

I'm not familiar with how the WebSocket lifecycle is managed on the server side, so if there's a better approach from that perspective, I'm open to guidance.

[linear]

NAN-5269

[rnagulapalle]

i think your root cause is pretty close. i checked the current frontend flow and openConnectUI() just opens Connect UI, while the sdk only moves forward when it gets an explicit success style result back. they also made detectClosedAuthWindow opt in earlier because popup closed detection is not reliable across providers, so turning that on here can easily turn a real success into a fake failure. the part that stood out to me is that if the websocket drops before the success message comes back, there is no good fallback state transition, so pending just hangs, and if the window closes after success you can still get pushed into the wrong error path. feels like the safer fix is exactly a post socket fallback check before showing failure, like recheck connection status once from the session or connection id and only show retry if that check is still negative. otherwise people will retry and create duplicates even though auth already finished.