gunicorn fix

Author: rempairamore

.github/workflows/docker-build.yml

@@ -2,9 +2,9 @@ name: Docker Build and Push
 
 on:
   push:
-    branches: [ main, master ]
+    branches: [ main, master, gunicorn-fix ]
   pull_request:
-    branches: [ main, master ]
+    branches: [ main, master, gunicorn-fix ]
 
 env:
   SERVIZIO_OC: oc_sparql

Dockerfile

@@ -33,4 +33,10 @@ EXPOSE 8080
 
 # Start the application
 # The Python script will now read environment variables for SPARQL configurations
-CMD ["python3", "sparql_oc.py"]
+CMD ["gunicorn", \
+     "-w", "2", \
+     "--worker-class", "gevent", \
+     "--worker-connections", "800", \
+     "--timeout", "1200", \
+     "-b", "0.0.0.0:8080", \
+     "sparql_oc:application"]

README.md

@@ -73,6 +73,10 @@ When static sync is enabled (via `--sync-static` or `SYNC_ENABLED=true`), the ap
 
 ## Running Options
 
+### Local Development
+
+For local development and testing, the application uses the built-in web.py HTTP server.
+
 The application supports the following command line arguments:
 
 - `--sync-static`: Synchronize static files at startup and enable periodic sync (every 30 minutes)
@@ -95,6 +99,20 @@ python3 sparql_oc.py --sync-static --port 8085
 
 The Docker container is configured to run with `--sync-static` enabled by default.
 
+### Production Deployment (Docker)
+
+When running in Docker/Kubernetes, the application uses **Gunicorn** as the WSGI HTTP server for better performance and concurrency handling:
+
+- **Server**: Gunicorn with gevent workers
+- **Workers**: 2 concurrent worker processes
+- **Worker Type**: gevent (async) for handling thousands of simultaneous requests
+- **Timeout**: 1200 seconds (to handle long-running SPARQL queries)
+- **Connections per worker**: 800 simultaneous connections
+
+The Docker container automatically uses Gunicorn and is configured with static sync enabled by default.
+
+> **Note**: The application code automatically detects the execution environment. When run with `python3 sparql_oc.py`, it uses the built-in web.py server. When run with Gunicorn (as in Docker), it uses the WSGI interface.
+
 ### Dockerfile
 
 You can change these variables in the Dockerfile:
@@ -112,13 +130,11 @@ ENV BASE_URL="sparql.opencitations.net" \
     SYNC_ENABLED="true"
 
 # Install system dependencies required for Python package compilation
-# We clean up apt cache after installation to reduce image size
 RUN apt-get update && \
     apt-get install -y \
     git \
     python3-dev \
-    build-essential && \
-    apt-get clean
+    build-essential
 
 # Set the working directory for our application
 WORKDIR /website
@@ -133,6 +149,11 @@ RUN pip install -r requirements.txt
 # Expose the port that our service will listen on
 EXPOSE 8080
 
-# Start the application
-# The Python script will now read environment variables for SPARQL configurations
-CMD ["python3", "sparql_oc.py"]
+# Start the application with gunicorn for production
+CMD ["gunicorn", \
+     "-w", "2", \
+     "--worker-class", "gevent", \
+     "--worker-connections", "800", \
+     "--timeout", "1200", \
+     "-b", "0.0.0.0:8080", \
+     "sparql_oc:application"]

docker_version.txt

@@ -1 +1 @@
-1.3.6sync
+1.3.8sync-gunitest

requirements.txt

@@ -39,3 +39,5 @@ web.py
 SPARQLWrapper
 gitpython
 APScheduler==3.10.1
+gunicorn==21.2.0 
+gevent==23.9.1

sparql_oc.py

@@ -45,6 +45,7 @@
 # URL Mapping
 urls = (
     "/", "Main",
+    "/static/(.*)", "Static",
     "/meta", "SparqlMeta",
     '/favicon.ico', 'Favicon',
     "/index", "SparqlIndex"
@@ -74,6 +75,9 @@
 # App Web.py
 app = web.application(urls, globals())
 
+# WSGI application
+application = app.wsgifunc()
+
 def sync_static_files():
     """
     Function to synchronize static files using sync_static.py
@@ -219,6 +223,36 @@ class SparqlMeta(Sparql):
     def __init__(self):
         Sparql.__init__(self, env_config["sparql_endpoint_meta"],
                        "meta", "/meta")
+        
+class Static:
+    def GET(self, name):
+        """Serve static files"""
+        static_dir = "static"
+        file_path = os.path.join(static_dir, name)
+
+        if not os.path.exists(file_path):
+            raise web.notfound()
+
+        # Content types
+        ext = os.path.splitext(name)[1]
+        content_types = {
+            '.css': 'text/css',
+            '.js': 'application/javascript',
+            '.png': 'image/png',
+            '.jpg': 'image/jpeg',
+            '.jpeg': 'image/jpeg',
+            '.gif': 'image/gif',
+            '.svg': 'image/svg+xml',
+            '.ico': 'image/x-icon',
+            '.woff': 'font/woff',
+            '.woff2': 'font/woff2',
+            '.ttf': 'font/ttf',
+        }
+
+        web.header('Content-Type', content_types.get(ext, 'application/octet-stream'))
+
+        with open(file_path, 'rb') as f:
+            return f.read()
 
 
 # Run the application