Production readiness audit

[tpendragon]

PUL IT has put together a checklist for production applications.

The work of this ticket is to go through the checklist and evaluate which of the criteria an application meets, and to create tickets for any criteria which an application does not meet.

Infrastructure

• All environments can be fully build using a playbook in Princeton Ansible (Infrastructure as Code)
• There is only one application on the server (One Application Per Server)
• There is a staging environment (Staging Environment)
• There are at least two servers per environment (Minimum 2 Servers Per Environment)
• Can be deployed with a single command (probably via capistrano) (Low Barrier to Deployment)
• Backup and Restore
  • Solr
  • Postgres
• Solr index uses a replication factor of 3 (Index Replication)

Security

• Main dependencies (Ruby, PHP, etc.) are under support (Use Dependencies Under Support)
• Automatic dependency checking for upstream dependencies (e.g. Dependabot) (Use Dependencies Under Support)
• Web ports are only accessible to the load balancer, not the world (Firewall)
• Secrets are in Ansible Vault (Secrets Management)
• We use OIT supported Single Sign-On for users to authenticate (Single Sign-on for Authentication / Authorization)

Monitoring

• Alerts are set up in Datadog, CheckMK, or Honeybadger (CheckMK preferred over Datadog) (Alerts Requiring Response)
• There are not many false alarms with the alerts (Alerts Requiring Response)
• We store metrics on the state of the system, either in Datadog, CheckMK, or Honeybadger (CheckMK preferred over Datadog) (Diagnostic Metrics)

User Interfaces

• WCAG AA Compliant (WCAG AA Compliance)
• WCAG Compliance covered by CI (WCAG AA Compliance)
• Gathers Web Analytics that measure the goals and success criteria of that application (Web Analytics)

Code Practices

• Main branch is deployable (Main Branch Always Deployable)
• There are automated tests that run on pull requests and merges to main (Automated Tests)
• Code coverage is measured and displayed in README (Code Coverage)
• Code review required for merge to main
• Application is configured via Environment Variables, preferably set up via Princeton Ansible (Configuration via Environment Variables)
• There is a README with access to documentation that includes
  • What the application is for
  • How to run the application
  • How to deploy the application
  • How to run the application's tests
  • Any other activities a developer would likely want to know about