Auto restart not working

[MrCivsteR]

When the server get the error below, it should invoke auto restart.
ENGINE (F): Application hangs (force crash) 64 s

The server process in the container is still running but nobody can connect. Anyone connected gets disconnected.

Error log file attached
error.log

[MrCivsteR]

Current workaround is to run a script with cron every minute. The script will check docker logs every 10 seconds and check for application hang. Will restart the container if logs show appropriate error.

#!/bin/sh

# Define the container name
CONTAINER_NAME="arma-reforger"

# Define the search string
SEARCH_STRING="Application hangs (force crash)"

# Define the number of iterations and delay
ITERATIONS=5
DELAY=10

# Check if the container is running
if ! docker ps --filter "name=^/${CONTAINER_NAME}$" --filter "status=running" --format "{{.Names}}" | grep -q "^${CONTAINER_NAME}$"; then
  echo "The container '$CONTAINER_NAME' is not running. Exiting."
  exit 1
fi

i=1
while [ $i -le $ITERATIONS ]
do
  echo "Iteration $i: Checking logs for container '$CONTAINER_NAME'..."

  # Check the last 10 log entries for the search string
  if docker logs --tail 10 "$CONTAINER_NAME" 2>&1 | grep -q "$SEARCH_STRING"; then
    echo "Issue detected: '$SEARCH_STRING' found in the logs."
    echo "Restarting the container: $CONTAINER_NAME..."
    
    # Restart the container
    docker restart "$CONTAINER_NAME"
    
    if [ $? -eq 0 ]; then
      echo "Container $CONTAINER_NAME restarted successfully."
    else
      echo "Failed to restart container $CONTAINER_NAME."
    fi
    
    # Exit the script after restarting the container
    exit 0
  else
    echo "No issues found in the last 10 log entries of $CONTAINER_NAME."
  fi

  # If this is not the last iteration, wait for the specified delay
  if [ $i -lt $ITERATIONS ]; then
    echo "Waiting for $DELAY seconds before the next check..."
    sleep $DELAY
  fi
  
  i=$((i + 1))
done

echo "Script completed after $ITERATIONS iterations. No issues detected."

[soda3x]

Maybe we can put this in a health check or restart policy rather than rely on arma reforgers logic

[MrCivsteR]

Since health check runs inside the container, this would involve capturing the stdout of the ArmaReforgerServer process

[soda3x]

It's ok, I'll look into a solution