[🐛 Bug]: OAuth2-proxy authentication causes redirect loop due to /alerts/event/error endpoint

### Environment

- **Keep version:** 0.48.1 (from official Helm chart)
- **Authentication:** OAuth2-proxy with Authentik (OIDC)
- **Kubernetes:** Scaleway Kapsule (nginx-ingress controller)
- **Deployment:** Custom ingress resources (not using chart's built-in ingress due to snippet annotation restrictions)

### Context

We deployed Keep with OAuth2-proxy authentication following the [documentation](https://docs.keephq.dev/deployment/authentication/oauth2proxy-auth). Due to our nginx-ingress configuration not allowing certain snippet annotations (even with `allow-snippet-annotations=true`), we created separate Ingress resources that are functionally equivalent to the chart's template.

After extensive testing, we identified **two related issues** when using OAuth2-proxy with Keep.

---

### Issue 1: Webhooks blocked by OAuth2 redirect

#### Symptom
External systems (Grafana, Prometheus, etc.) sending alerts via webhooks to `/alerts/event` or `/alerts/event/{provider}` were redirected to the SSO login page instead of being processed.

#### Root cause
When all traffic goes through OAuth2-proxy, webhook endpoints that use API key authentication are blocked because:
1. The ingress `auth-url` annotation triggers OAuth2 validation
2. Webhooks don't have OAuth2 tokens, only API keys in headers
3. Keep's internal API key validation never runs because nginx redirects to SSO first

#### Solution
Create a **separate Ingress without OAuth2 annotations** for webhook endpoints:

```yaml
# Webhook endpoints - NO OAuth2, Keep handles API key auth internally
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keep-backend-webhooks
  namespace: keep
  annotations:
    # NO OAuth2 annotations here
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
    - host: keep.example.com
      http:
        paths:
          - path: /v2(/|$)(alerts/event.*)
            pathType: ImplementationSpecific
            backend:
              service:
                name: keep-backend
                port:
                  number: 8080
```

> **Note:** We use `/v2` prefix because `/api` is caught by Next.js frontend.

---

### Issue 2: Redirect loop after successful SSO authentication

#### Symptom
After successfully authenticating via SSO:
1. User is redirected to Keep
2. The `/alerts/feed` page loads briefly
3. User is immediately redirected back to `/signin`
4. Loop continues indefinitely

#### Discovery
Using browser DevTools, we noticed that when `/alerts/feed` loads, the frontend makes a request to:
```
GET /v2/alerts/event/error
```

**Key finding:** Blocking this request in DevTools prevents the redirect loop.

#### Root cause analysis

The webhook bypass regex was **too broad**:

```yaml
path: /v2(/|$)(alerts/event.*)
```

This regex matches:

| Path | Matched | Actual usage |
|------|---------|--------------|
| `/v2/alerts/event` | Yes | Webhook POST - OK without OAuth2 |
| `/v2/alerts/event/grafana` | Yes | Webhook POST - OK without OAuth2 |
| `/v2/alerts/event/netdata` | Yes | Challenge GET - OK without OAuth2 |
| `/v2/alerts/event/error` | Yes | **Frontend GET - NEEDS OAuth2** |

According to the [OpenAPI spec](https://github.com/keephq/keep/blob/main/docs/openapi.json):
- `/alerts/event` (POST) - Webhook ingestion, uses API key
- `/alerts/event/{provider_type}` (POST) - Provider webhook, uses API key
- `/alerts/event/error` (GET) - **Frontend view for displaying event errors, requires user authentication**

#### Problem flow

```
1. User authenticates via SSO successfully
2. User accesses /alerts/feed
3. Frontend calls GET /v2/alerts/event/error
4. Nginx matches webhook bypass regex -> SKIPS OAuth2 auth
5. Backend receives request WITHOUT X-Auth-Request-* headers
6. Backend returns 401/403 (no user identity)
7. Frontend interprets as "not authenticated"
8. Frontend redirects to /signin
9. Loop repeats
```

#### Solution

Use a **negative lookahead** in the regex to exclude `/error`:

```yaml
# Before (too broad)
path: /v2(/|$)(alerts/event.*)

# After (excludes /error)
path: /v2(/|$)(alerts/event(?!/error).*)
```

This regex:
- Matches `/v2/alerts/event` -> rewrite to `/alerts/event`
- Matches `/v2/alerts/event/grafana` -> rewrite to `/alerts/event/grafana`
- Matches `/v2/alerts/event/netdata` -> rewrite to `/alerts/event/netdata`
- Does NOT match `/v2/alerts/event/error` -> goes through OAuth2-protected ingress

---

### Complete working Ingress configuration

Here's our complete, working configuration with 6 separate Ingress resources:

```yaml
---
# 1. Static assets (no auth)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keep-static
  namespace: keep
spec:
  ingressClassName: nginx
  rules:
    - host: keep.example.com
      http:
        paths:
          - path: /_next
            pathType: Prefix
            backend:
              service:
                name: keep-frontend
                port:
                  number: 3000
---
# 2. Webhook endpoints (API key auth, NO OAuth2)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keep-backend-webhooks
  namespace: keep
  annotations:
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
    - host: keep.example.com
      http:
        paths:
          # IMPORTANT: Exclude /error to prevent redirect loop
          - path: /v2(/|$)(alerts/event(?!/error).*)
            pathType: ImplementationSpecific
            backend:
              service:
                name: keep-backend
                port:
                  number: 8080
---
# 3. Main frontend (OAuth2 protected)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keep
  namespace: keep
  annotations:
    nginx.ingress.kubernetes.io/auth-url: "http://oauth2-proxy.keep.svc.cluster.local/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: "https://keep.example.com/oauth2/start?rd=$escaped_request_uri"
    nginx.ingress.kubernetes.io/auth-response-headers: "X-Auth-Request-User,X-Auth-Request-Email,X-Auth-Request-Groups,X-Auth-Request-Access-Token,Authorization"
spec:
  ingressClassName: nginx
  rules:
    - host: keep.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: keep-frontend
                port:
                  number: 3000
---
# 4. Backend API (OAuth2 protected, with URL rewriting)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keep-backend
  namespace: keep
  annotations:
    nginx.ingress.kubernetes.io/auth-url: "http://oauth2-proxy.keep.svc.cluster.local/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: "https://keep.example.com/oauth2/start?rd=$escaped_request_uri"
    nginx.ingress.kubernetes.io/auth-response-headers: "X-Auth-Request-User,X-Auth-Request-Email,X-Auth-Request-Groups,X-Auth-Request-Access-Token,Authorization"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
    - host: keep.example.com
      http:
        paths:
          - path: /v2(/|$)(.*)
            pathType: ImplementationSpecific
            backend:
              service:
                name: keep-backend
                port:
                  number: 8080
---
# 5. WebSocket (OAuth2 protected)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keep-websocket
  namespace: keep
  annotations:
    nginx.ingress.kubernetes.io/auth-url: "http://oauth2-proxy.keep.svc.cluster.local/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: "https://keep.example.com/oauth2/start?rd=$escaped_request_uri"
    nginx.ingress.kubernetes.io/auth-response-headers: "X-Auth-Request-User,X-Auth-Request-Email,X-Auth-Request-Groups,X-Auth-Request-Access-Token,Authorization"
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
    nginx.ingress.kubernetes.io/upstream-hash-by: "$remote_addr"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
    - host: keep.example.com
      http:
        paths:
          - path: /websocket(/|$)(.*)
            pathType: ImplementationSpecific
            backend:
              service:
                name: keep-websocket
                port:
                  number: 6001
---
# 6. OAuth2 Proxy endpoints (no auth on these)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keep-oauth2-proxy
  namespace: keep
spec:
  ingressClassName: nginx
  rules:
    - host: keep.example.com
      http:
        paths:
          - path: /oauth2
            pathType: Prefix
            backend:
              service:
                name: oauth2-proxy
                port:
                  number: 80
```

---

### Keep Helm values (relevant parts)

```yaml
backend:
  env:
    - name: AUTH_TYPE
      value: OAUTH2PROXY
    - name: KEEP_OAUTH2_PROXY_USER_HEADER
      value: X-Auth-Request-Email
    - name: KEEP_OAUTH2_PROXY_ROLE_HEADER
      value: X-Auth-Request-Groups
    - name: KEEP_OAUTH2_PROXY_AUTO_CREATE_USER
      value: "true"
    - name: KEEP_OAUTH2_PROXY_ADMIN_ROLES
      value: admin-group  # Your admin group from IdP
    - name: KEEP_OAUTH2_PROXY_NOC_ROLES
      value: noc-group    # Your NOC group from IdP
    - name: KEEP_API_URL
      value: "https://keep.example.com/v2"

frontend:
  env:
    - name: AUTH_TYPE
      value: OAUTH2PROXY
    - name: KEEP_OAUTH2_PROXY_USER_HEADER
      value: X-Auth-Request-Email
    - name: KEEP_OAUTH2_PROXY_ROLE_HEADER
      value: X-Auth-Request-Groups
    - name: AUTH_TRUST_HOSTED_PROXIES
      value: "true"
    - name: NEXTAUTH_URL
      value: "https://keep.example.com"
    - name: API_URL_CLIENT
      value: "/v2"
    - name: KEEP_API_URL
      value: "https://keep.example.com/v2"
```

---

### OAuth2-proxy configuration (relevant parts)

```yaml
extraArgs:
  provider: oidc
  provider-display-name: "Your IdP"
  oidc-issuer-url: "https://sso.example.com/application/o/keep/"
  redirect-url: "https://keep.example.com/oauth2/callback"
  set-xauthrequest: "true"
  pass-access-token: "true"
  pass-authorization-header: "true"
  scope: "openid profile email groups"
  code-challenge-method: "S256"
  cookie-secure: "true"
  cookie-samesite: "lax"

config:
  configFile: |-
    email_domains = [ "example.com" ]
    upstreams = [ "file:///dev/null" ]
```

---

### Suggestions for Keep

1. **Documentation improvement:** Add a note in the OAuth2-proxy documentation about webhook endpoints needing to bypass OAuth2 authentication

2. **Consider renaming or moving `/alerts/event/error`:** This endpoint name collides with the webhook path pattern. Perhaps moving it to `/alerts/errors` or `/alerts/event-errors` would avoid confusion

3. **Helm chart enhancement:** For users enabling OAuth2-proxy, the chart could automatically create a separate ingress for webhook paths without OAuth2 annotations

---

### Search performed

I searched for similar issues but couldn't find any existing reports about this specific redirect loop behavior with OAuth2-proxy. The closest issues were about general authentication but didn't cover this specific `/alerts/event/error` endpoint issue.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[🐛 Bug]: OAuth2-proxy authentication causes redirect loop due to /alerts/event/error endpoint #5521

Environment

Context

Issue 1: Webhooks blocked by OAuth2 redirect

Symptom

Root cause

Solution

Issue 2: Redirect loop after successful SSO authentication

Symptom

Discovery

Root cause analysis

Problem flow

Solution

Complete working Ingress configuration

Keep Helm values (relevant parts)

OAuth2-proxy configuration (relevant parts)

Suggestions for Keep

Search performed

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Path	Matched	Actual usage
`/v2/alerts/event`	Yes	Webhook POST - OK without OAuth2
`/v2/alerts/event/grafana`	Yes	Webhook POST - OK without OAuth2
`/v2/alerts/event/netdata`	Yes	Challenge GET - OK without OAuth2
`/v2/alerts/event/error`	Yes	Frontend GET - NEEDS OAuth2

[🐛 Bug]: OAuth2-proxy authentication causes redirect loop due to /alerts/event/error endpoint #5521

Description

Environment

Context

Issue 1: Webhooks blocked by OAuth2 redirect

Symptom

Root cause

Solution

Issue 2: Redirect loop after successful SSO authentication

Symptom

Discovery

Root cause analysis

Problem flow

Solution

Complete working Ingress configuration

Keep Helm values (relevant parts)

OAuth2-proxy configuration (relevant parts)

Suggestions for Keep

Search performed

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions