Issues with HTTPS Requests and Routes in APISIX Configuration (Docker Setup)

[FaaSioFlexService]

Hi everyone,

I am currently working on setting up APISIX with Docker for handling HTTP and HTTPS requests, but I’m facing a few issues with serving HTTPS requests and routes defined in my routes.yaml. Here’s a breakdown of my setup and the issues I’m encountering:

Dockerfile_etcd:

# Usa la imagen oficial de etcd como base
  FROM quay.io/coreos/etcd:v3.5.0
  
  # Define el directorio de trabajo
  WORKDIR /etcd
  
  # Configura las variables de entorno para etcd, eliminando ETCD_LISTEN_CLIENT_URLS
  ENV ETCD_LISTEN_PEER_URLS="http://0.0.0.0:2380" \
      ETCD_ADVERTISE_CLIENT_URLS="http://etcd:2379" \
      ETCD_INITIAL_CLUSTER="etcd=http://etcd:2380" \
      ETCD_INITIAL_CLUSTER_STATE="new" \
      ETCD_DATA_DIR="/etcd-data" \
      ETCD_NAME="etcd" \
      ETCD_INITIAL_ADVERTISE_PEER_URLS="http://etcd:2380"  
  ENV ETCD_ENABLE_V2="true" \
      ALLOW_NONE_AUTHENTICATION="yes" \
      ETCD_ADVERTISE_CLIENT_URLS="http://etcd:2379" \
      ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379" \
      ETCD_LOG_LEVEL="debug"
  # Exponer los puertos en los que etcd escucha
  EXPOSE 2379 2380
  
  # Comando de inicio para etcd
  CMD ["etcd"]

Dockerfile:

FROM apache/apisix:latest

USER root

# Create necessary directories for logs and set permissions
RUN mkdir -p /usr/local/apisix/logs && \
    mkdir -p /usr/local/openresty/nginx/logs && \
    mkdir -p /usr/local/openresty/nginx/client_body_temp && \
    chown -R nobody:nogroup /usr/local/openresty/nginx /usr/local/apisix/logs /usr/local/openresty/nginx/logs /usr/local/openresty/nginx/client_body_temp && \
    chmod -R 755 /usr/local/openresty/nginx/logs /usr/local/openresty/nginx/client_body_temp /usr/local/apisix/logs

# Copy certificates and configuration files
COPY ./conf/certs/rootCA.pem /usr/local/apisix/certs/ca-certificates.crt
COPY ./conf/certs/localhost.pem /usr/local/apisix/certs/localhost.pem
COPY ./conf/certs/localhost-key.pem /usr/local/apisix/certs/localhost-key.pem
COPY ./conf/config.yaml /usr/local/apisix/conf/config.yaml
COPY ./conf/routes.yaml /usr/local/apisix/conf/routes.yaml

# Update certificates
RUN apt-get update && apt-get install -y ca-certificates && update-ca-certificates
RUN update-ca-certificates

# Update log level for debug visibility
RUN sed -i 's/log_level: warn/log_level: debug/' /usr/local/apisix/conf/config.yaml

# Expose necessary ports
EXPOSE 9080 9443 9180

# Start APISIX
CMD ["apisix", "start", "-c", "/usr/local/apisix/conf/config.yaml", "-v"]

config. yaml :

apisix:
  basic:
    enable: false          
  config_center: "yaml"
  enable_dev_mode: false
  enable_reuseport: true
  enable_ipv6: true
  log_level: debug       
  enable_http2: true
  show_upstream_status_in_response_header: false
  enable_server_tokens: true
  proxy_mode: "http"
  disable_sync_configuration_during_start: false
  node_listen: 
      - 9080 
      - 9070   
  ssl:
    enable: true
    server_name: localhost
    listen:  
      - ip: "0.0.0.0"    
        port: 9443
        enable_http3: true       
    ssl_protocols: TLSv1.2 TLSv1.3   
    ssl_ciphers: DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384   
    ssl_session_tickets: false  
    ssl_trusted_certificate: /etc/ssl/certs/ca-certificates.crt    
    certs:
        cert: /usr/local/apisix/certs/localhost.pem 
        key: /usr/local/apisix/certs/localhost-key.pem    
        sni: localhost
http:
  enable_access_log: true
  access_log: logs/access.log
nginx_config:
  error_log_level: "info" 
hook_conf:
  enable: false                 
  name: hook_phase               
  log_level: trace                
  is_print_input_args: true    
  is_print_return_value: true   
hook_phase:                     
  apisix:                       
    - http_access_phase         
    - http_header_filter_phase
    - http_body_filter_phase
    - http_log_phasedocker ps -a
routes:
  http: radixtree_host_uri 
  ssl: radixtree_sni   
  path: /usr/local/apisix/conf/routes.yaml  

deployment:    
  role: data_plane  
  role_data_plane:  
    config_provider: yaml  

#deployment:
#  role: traditional
#  role_traditional:
#    config_provider: etcd
#  admin:
#    enable_admin: false
#    admin_key:
#      - name: "admin"
#        key: "edd1c9f0985e76a2"
#        role: admin 
#   allow_admin:                    
#      - 127.0.0.0/24
#      - 172.18.0.1
#    admin_listen:
#        ip: 0.0.0.0
#        port: 9180
#  etcd:
#    host:
#      - "http://etcd:2379"
#    prefix: /apisix
#    tls:
#      cert: /usr/local/apisix/certs/localhost.pem 
#     key: /usr/local/apisix/certs/localhost-key.pem    
#      sni: localhost
 
plugin:
  - name: http-logger
    enable: true
    config:      
      batch: 10                           
      timeout: 5000                       
      log_format: '{"client_ip": "$remote_addr", "method": "$request_method", "uri": "$request_uri", "status": "$status", "request_time": "$request_time", "body": "$request_body"}'
      log_level: "debug"
      log_target: "file"  
      log_file: "/usr/local/apisix/logs/access.log"
      access_log: "/usr/local/apisix/logs/access.log" 
      error_log: "/usr/local/apisix/logs/error.log"
  - name: cors
    enable: true
  - name: proxy-rewrite
    enable: true
  - name: jwt-auth
    enable: true
  - name: openid-connect
    enable: true
  - name: connection-limiting
    enable: true
  - name: redirect-https
    config:
      https_redirect_status_code: 301

routes. yaml:

- uri: /api/v1/
  name: resource-route
  upstream:
    nodes:
      "https://localhost:8081/api/v1/": 1
    type: roundrobin
  methods: ["GET", "POST"]
  plugins:
    - jwt-auth:
        key: "53cRe7k3y!zXy9*@1234Abc"
        algorithms: ["HS256"]

- uri: /register
  name: register-user
  upstream:
    nodes:
      "https://localhost:8081/register": 1
    type: roundrobin
  methods: ["POST","GET"]
  plugins:  
      - cors:
        allow_origin: "https://localhost:9443, https://faasioflex.com:9443"
        allow_methods: "GET, POST"
        allow_headers: "Authorization, Content-Type"
        expose_headers: "X-faasioflex"
        allow_credentials: true
        max_age: 3600

# Ruta para la autenticación de usuarios
- uri: /login
  name: login-user
  upstream:
    nodes:
      "https://localhost:5557": 1  # Redirige a Dex (por defecto, Dex está en localhost:5557)
    type: roundrobin
  methods: ["POST"]
  plugins:
    - openid-connect:
        client_id: backend  # ID de cliente configurado en Dex
        client_secret: ZXhhbXBsZS1hcHAtc2VjcmV0  # Secreto de cliente configurado en Dex
        discovery: "https://localhost:5557/.well-known/openid-configuration"  # URL de configuración de OpenID Connect (Dex)
        token_endpoint_auth_method: "client_secret_basic"  # Método de autenticación para el cliente
        scope: ["openid", "profile", "email", "phone", "nickname", "name"]  # Alcances solicitados

- uri: /app/*
  name: your-api-route
  upstream:
    nodes:
      "https://localhost:5557": 1
    type: roundrobin
  methods: ["GET", "POST"]
  plugins:
    - openid-connect:
        client_id: backend  # ID de cliente configurado en Dex
        client_secret: ZXhhbXBsZS1hcHAtc2VjcmV0  # Secreto de cliente configurado en Dex
        discovery: "https://localhost:5557/.well-known/openid-configuration"  # URL de descubrimiento de Dex
        token_endpoint_auth_method: "client_secret_basic"  # Método de autenticación para el cliente
        scope: ["openid", "profile", "email", "phone", "nickname", "name"]  # Alcances que se solicita en el proveedor

- uri: /functions/register
  name: register-functions
  upstream:
    nodes:
      "https://localhost:8081/functions/register": 1
    type: roundrobin
  methods: ["POST"]
  plugins:
    - jwt-auth:
        key: "53cRe7k3y!zXy9*@1234Abc"
        algorithms: ["HS256"]

- uri: /functions/activate
  name: activate-functions
  upstream:
    nodes:
      "https://localhost:8081/functions/activate": 1
    type: roundrobin
  methods: ["POST"]
  plugins:
    - jwt-auth:
        key: "53cRe7k3y!zXy9*@1234Abc"
        algorithms: ["HS256"]
    - cors:
        allow_origin: "https://localhost:9443, https://faasioflex.com:9443"
        allow_methods: "GET, POST"
        allow_headers: "Authorization, Content-Type"
        expose_headers: "X-faasioflex"
        allow_credentials: true
        max_age: 3600

- uri: /functions/execute
  name: execute-functions
  upstream:
    nodes:
      "https://localhost:8081/functions/execute": 1
    type: roundrobin
  methods: ["POST"]
  plugins:
    - jwt-auth:
        key: "53cRe7k3y!zXy9*@1234Abc"
        algorithms: ["HS256"]
    - connection-limiting:
        conn_limit: 10  # Limita a 10 conexiones simultáneas
        burst: 5  # Permite un "pico" adicional de 5 conexiones
        default_conn_delay: 10  # Si se alcanza el límite, se espera 10 ms antes de aceptar una nueva conexión
    - cors:
        allow_origin: "https://localhost:9443, https://faasioflex.com:9443"
        allow_methods: "GET, POST"
        allow_headers: "Authorization, Content-Type"
        expose_headers: "X-faasioflex"
        allow_credentials: true
        max_age: 3600

- uri: /functions/delete
  name: execute-functions
  upstream:
    nodes:
      "https://localhost:8081/functions/delete": 1
    type: roundrobin
  methods: ["POST"]
  plugins:
    - jwt-auth:
        key: "53cRe7k3y!zXy9*@1234Abc"
        algorithms: ["HS256"]
    - cors:
        allow_origin: "https://localhost:9443, https://faasioflex.com:9443"
        allow_methods: "GET, POST"
        allow_headers: "Authorization, Content-Type"
        expose_headers: "X-faasioflex"
        allow_credentials: true
        max_age: 3600

- uri: /*
  name: catch-all-route
  upstream:
    nodes:
      "http://backend_service:8000": 1  # Usa HTTP interno si APISIX maneja la conversión
    type: roundrobin
  plugins:
    redirect:
      http_to_https: true  # Aplica solo cuando la solicitud entra por HTTP (9080)
  enable_websocket: true

# Ruta HTTP /status
- uri: /status
  name: status-http-route
  upstream:
    nodes:
      "https://localhost:8081": 1  # Redirige a tu backend en https://localhost:8081
    type: roundrobin
  methods: ["GET"]
  plugins: []  # Sin plugins, solo proxy inverso

# Ruta HTTPS /status (igual que la anterior)
- uri: /status
  name: status-https-route
  upstream:
    nodes:
      "https://localhost:8081": 1  # Redirige a tu backend en https://localhost:8081
    type: roundrobin
  methods: ["GET"]
  plugins: []  # Sin plugins, solo proxy inverso

- uri: /heartgo
  name: status-route
  methods: ["GET"]
  upstream:
    type: roundrobin
    nodes:
      "https://localhost:9443": 1
  plugins:
    - response-rewrite:
        body: '{"status": "ok"}'
        status_code: 200

- uri: /heartgo
  name: status-route
  methods: ["GET"]
  upstream:
    type: roundrobin
    nodes:
      "http://localhost:9080": 1
  plugins:
    - response-rewrite:
        body: '{"status": "ok"}'
        status_code: 200

- uri: /hello
  name: hello-route
  upstream:
    type: roundrobin
    nodes:
      - host: google.com  # Mantienes la respuesta de google para verificar
        port: 80
        weight: 1
  methods: ["GET"]  # Responde a GET

- uri: /heart
  name: heart-route
  upstream:
    type: roundrobin
    nodes:
      "127.0.0.1:8000": 1  # Respondemos en localhost en el puerto 8000
  methods: ["GET"]  # Responde a GET
- uri: /test
  name: id-test
  upstream:
    nodes:
      "${{HOST_IP}}:${{PORT}}": 1
    type: roundrobin
- uri: /id/1
  name: id-route
  plugins:
    - response-rewrite:
        body: "hello\n"  # Responde con el texto "hello\n"
        status_code: 200  # Código 200 para indicar éxito
  methods: ["GET"]  # Responde a GET

script shesll to setup docker is:

#!/bin/bash

# Nombre de la imagen y del contenedor
IMAGE_NAME="apisix"
CONTAINER_NAME="apisix"
ETCD_CONTAINER_NAME="etcd"
NETWORK_NAME="faasioflexnetwork"
ETCD_DOCKERFILE="Dockerfile_etcd"
               
# Crear la red faasioflexnetwork si no existe
if ! docker network inspect $NETWORK_NAME >/dev/null 2>&1; then
    echo "La red $NETWORK_NAME no existe. Creando la red..."
    docker network create $NETWORK_NAME
else
    echo "La red $NETWORK_NAME ya existe."
fi

# Eliminar contenedores detenidos, si existen
STOPPED_CONTAINERS=$(docker ps -a -q -f status=exited)
if [ -n "$STOPPED_CONTAINERS" ]; then
    docker rm $STOPPED_CONTAINERS
fi

container_id=$(docker ps -q -f name=$ETCD_CONTAINER_NAME)
if [ -z "$container_id" ]; then
    echo "Container ID is empty"
else
    echo "Container ID: $container_id"
fi


# Verificar si el contenedor etcd ya existe
if [ $(docker ps -a -q -f name=$ETCD_CONTAINER_NAME) ]; then
    echo "El contenedor $ETCD_CONTAINER_NAME ya existe. Deteniéndolo y eliminándolo..."
    docker stop $ETCD_CONTAINER_NAME
    docker rm $ETCD_CONTAINER_NAME
fi

# Eliminar la imagen etcd si existe
if [ $(docker images -q $ETCD_CONTAINER_NAME) ]; then
    echo "La imagen de etcd ya existe. Eliminándola..."
    docker rmi -f $ETCD_CONTAINER_NAME
fi

# Construir la imagen Docker para etcd desde el Dockerfile_etcd
echo "Construyendo la imagen Docker para etcd desde $ETCD_DOCKERFILE..."
docker build -f $ETCD_DOCKERFILE -t $ETCD_CONTAINER_NAME .

# Verificar si la construcción de la imagen fue exitosa
if [ $? -ne 0 ]; then
    echo "Error al construir la imagen de etcd."
    exit 1
fi

# Ejecutar el contenedor de etcd (Esperar a que se inicie completamente)
echo "Creando y ejecutando el contenedor de etcd..."
docker run -d --name $ETCD_CONTAINER_NAME --network $NETWORK_NAME --publish 2379:2379 $ETCD_CONTAINER_NAME /usr/local/bin/etcd

# Esperar hasta que etcd esté completamente arrancado (puedes cambiar el tiempo de espera si es necesario)
echo "Esperando a que etcd esté completamente en funcionamiento..."
sleep 10  # Aumenté el tiempo de espera para asegurarse de que etcd esté arrancado correctamente

# Verificar que etcd está en ejecución
ETCD_STATUS=$(docker ps -q -f name=$ETCD_CONTAINER_NAME)
if [ -z "$ETCD_STATUS" ]; then
    echo "Error: El contenedor $ETCD_CONTAINER_NAME no se inició correctamente."
    exit 1
else
    echo "Contenedor $ETCD_CONTAINER_NAME en ejecución..."
fi

# Verificar si el contenedor apisix ya existe
if [ $(docker ps -a -q -f name=$CONTAINER_NAME) ]; then
    echo "El contenedor $CONTAINER_NAME ya existe. Deteniéndolo y eliminándolo..."
    docker stop $CONTAINER_NAME
    docker rm $CONTAINER_NAME
fi

# Eliminar la imagen de APISIX si existe
if [ $(docker images -q $IMAGE_NAME) ]; then
    echo "La imagen $IMAGE_NAME ya existe. Eliminándola..."
    docker rmi -f $IMAGE_NAME
fi

# Construir la imagen de APISIX desde el Dockerfile
echo "Construyendo la imagen Docker para APISIX desde el Dockerfile..."
docker build -t $IMAGE_NAME .

# Verificar si la construcción de la imagen fue exitosa
if [ $? -ne 0 ]; then
    echo "Error al construir la imagen $IMAGE_NAME."
    exit 1
fi

# Crear y ejecutar el contenedor de APISIX
echo "Creando y ejecutando el contenedor $CONTAINER_NAME..."
echo  "docker run -d -p 9080:9080 -p 9443:9443 -p 9180:9180 --name $CONTAINER_NAME --network $NETWORK_NAME    $IMAGE_NAME"
docker run -d -p 9080:9080 -p 9443:9443  -p 9180:9180 --name $CONTAINER_NAME --network $NETWORK_NAME    $IMAGE_NAME

# Verificar si el contenedor de APISIX se está ejecutando
CONTAINER_STATUS=$(docker ps -q -f name=$CONTAINER_NAME)
if [ -z "$CONTAINER_STATUS" ]; then
    echo "Error: El contenedor $CONTAINER_NAME no se inició correctamente."
    echo "Mostrando los logs del contenedor..."
    docker logs $CONTAINER_NAME
    exit 1
else
    echo "Contenedor $CONTAINER_NAME en ejecución..."
    docker ps
fi

# Verificar que los contenedores estén conectados a la red
echo "Verificando que los contenedores estén conectados a la red $NETWORK_NAME..."
docker network inspect $NETWORK_NAME | grep -A 10 "Containers"

Problem:
HTTPS Requests Not Working: Despite setting up SSL certificates, I am unable to serve HTTPS requests on port 9443. The requests just time out or fail.

Routes Not Being Executed: The routes defined in my routes.yaml are not being triggered. Specifically, requests to /api/v1/ do not get routed properly to the upstream services defined in routes.yaml.

Things I’ve Tried:
Double-checking the paths for certificates and ensuring they are being copied correctly in the Dockerfile.
Confirming that the config.yaml and routes.yaml files are correctly loaded into the APISIX container.
Setting the logging level to debug to get more insights, but I’m still not seeing any useful logs related to routing or SSL issues.

jogugil@PHOSKI:~/smartiaservices_projects/faasioflex/src$ curl -v http://127.0.0.1:9080/hello

• Trying 127.0.0.1:9080...
• Connected to 127.0.0.1 (127.0.0.1) port 9080

GET /hello HTTP/1.1
Host: 127.0.0.1:9080
User-Agent: curl/8.5.0
Accept: /

< HTTP/1.1 404 Not Found
< Date: Tue, 11 Feb 2025 15:23:11 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Server: APISIX/3.11.0
<
{"error_msg":"404 Route Not Found"}

• Connection #0 to host 127.0.0.1 left intact
  jogugil@PHOSKI:~/smartiaservices_projects/faasioflex/src$ curl -v http://127.0.0.1:9080/heart
• Trying 127.0.0.1:9080...
• Connected to 127.0.0.1 (127.0.0.1) port 9080

GET /heart HTTP/1.1
Host: 127.0.0.1:9080
User-Agent: curl/8.5.0
Accept: /

< HTTP/1.1 404 Not Found
< Date: Tue, 11 Feb 2025 15:23:15 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Server: APISIX/3.11.0
<
{"error_msg":"404 Route Not Found"}

• Connection #0 to host 127.0.0.1 left intact
  jogugil@PHOSKI:~/smartiaservices_projects/faasioflex/src$ curl -v https://127.0.0.1:9443/heart
• Trying 127.0.0.1:9443...
• Connected to 127.0.0.1 (127.0.0.1) port 9443
• ALPN: curl offers h2,http/1.1
• TLSv1.3 (OUT), TLS handshake, Client hello (1):
• CAfile: /etc/ssl/certs/ca-certificates.crt
• CApath: /etc/ssl/certs
• TLSv1.3 (IN), TLS alert, internal error (592):
• OpenSSL/3.0.13: error:0A000438:SSL routines::tlsv1 alert internal error
• Closing connection
  curl: (35) OpenSSL/3.0.13: error:0A000438:SSL routines::tlsv1 alert internal error
  jogugil@PHOSKI:~/smartiaservices_projects/faasioflex/src$ curl -v https://localhost:9443/heart
• Host localhost:9443 was resolved.
• IPv6: ::1
• IPv4: 127.0.0.1
• Trying [::1]:9443...
• Connected to localhost (::1) port 9443
• ALPN: curl offers h2,http/1.1
• TLSv1.3 (OUT), TLS handshake, Client hello (1):
• CAfile: /etc/ssl/certs/ca-certificates.crt
• CApath: /etc/ssl/certs
• TLSv1.3 (IN), TLS alert, internal error (592):
• OpenSSL/3.0.13: error:0A000438:SSL routines::tlsv1 alert internal error
• Closing connection
  curl: (35) OpenSSL/3.0.13: error:0A000438:SSL routines::tlsv1 alert internal error
  Docker Logs:

I’ve been checking the logs but can’t see any errors pointing to the SSL issue or routing failure.
If anyone has encountered a similar issue or has any suggestions on what might be wrong with the configuration, I’d greatly appreciate the help!